Hello Everyone 👋

Raman Sharma Here. I hope you all are doing good and safe. I'm a tech enthusiast, passionate developer and a creator of Fullyworld Web Tutorials YouTube channel from India 🇮🇳.

Here, I am presenting Voice Controllable Website, a website prototype that is fully voice controllable.

Introduction

As we know that this is an era of AI (Artificial Intelligence) .

Artificial Intelligence is an approach to make a computer smarter and human life easier. AI assists in every area of our life, whether we are reading emails, giving directions, doing a web search and a lot more...

Applications of AI :

• Gaming:- AI is also used in gaming applications to get a large number of possible positions based on deep knowledge in strategic, sports games, e.g, FIFA, Chess, river crossing, etc.

• Natural Language Processing:- Interact with the computer that understands natural language spoken by humans.

• Intelligent Robots:- Robots are able to perform the instructions given by a human.

• Speech Recognition:- There are some AI-based speech recognition systems that have the ability to hear and express sentences and understand their meanings while a person talks to it. For example Siri, Crotona, Google assistant, etc...

• and a lot more...

Insipiration, Problem and Idea 💡

AI is being used everywhere, even in web development. In web development, there is a voice search, voice assistance, chatbot and web search.

This is increasing the user's experience.

So, there is one idea that came to my mind, why not create a full voice-controlled website?

Wait! what? where are you going?

Don't you wanna know why and how I have created it and how it will be helpful for humans?

Well then, let's see

Voice-controlled systems have changed the way of interaction of human with computer and other systems.

As, it is fast, fun and makes communication more real.

Learning is more efficient and interactive if you can make use of voice-enabled features especially while learning languages and music.

Speech Recognition allow the user to interact with the computer and make hand-free requests.

So, How it is Beneficial?

A voice automated application will be more helpful for human.

1. It is of great interest to the people.

2. It follows the hand free commands feature.

3. It can be more useful for visually disabled people as it can be controlled and operated by one's voice

4. It will give a response for every command of the user visually as well as audibly.

5. It will give the right way to do the thing audibly as well as visually if some command or something went wrong.

6. It will give every instruction to the user.

Thus, voice-controlled web applications are very advantageous to control and operate the functionalities of web applications.

How does it actually work?

Users make hand-free request -> computer process them -> it give the results based on the user's request

There are some AI Platforms available online for voice control system, e.g, Alan AI, Snips, Wonder Voice and more...

These platforms either provide software or APIs to enable voice functionality in our websites or software.

Ya I know, you have a question in your mind, if these platforms are already available then why I have created this prototype?

Don't worry about that, let's see why I have created this.

Why I have created this?

As these ai platforms are already available but these are only limited as free and some are paid also.

Let's take the example of Alan AI, This platform is very good as it provides API and with help of that we can make any feature of our website voice controllable but, this is not open source.

Its free version only provides 10000 interactions and the other is paid and some other ai platforms only provide software and they are also not free and open source.

so, I got an idea that why not create an open-source library for everyone that can help to make every website fully functionally voice controllable.

That's why I have created a prototype of a fully functioning voice controllable website to show how a fully voice controllable website will look.

Checkout this concept in the video both visually and audibly given below

Now, Let's Talk about my website, how its look , its features and working

Let's see a Demo

Features ⚡

control your website with voice commands

This prototype is having 8 features

• Voice Controlled Navigation
• Voice Controlled Scrolling
• Voice Controlled Video Selection
• Voice Controlled Pagination
• Voice Controlled Form Filling And Submission
• Voice Controlled Website History
• Voice Controlled Video Controls
• Voice Controlled Search

Wow 😲

Not Only This,

This also give response for every command with its result and even give directions on complex commands.

That's Interesting!

Although some commands are hard coded but many of the commands are dynamic and all the results are dynamic.

All searching, video selection , video controls, form filling and submission, navigate history, scrolling is dynamic.

As, I haven't used Machine Learning Here so many of the audible responses from the website that you will listen are hard coded messages but, results are very dynamic those you will see visually.

So do you want to know how i have created this and what type of technologies i have used?

Well then... get ready🔥

Development process

1. Understanding the Requirements

At first, I have carefully understood all the things that I wanted to implement and examined all the aspects of this project.

2. Designing Layout

This was an important part of my development process as User Interface is something that can help in gaining user's attention. Now I have to choose the good Layout and good color combinations

In this phase of the project, the challenge that I have faced is that I cant able to choose the good layout and theme for the website, I was getting confused between the layout position and colors so decided to make sample parts first. And at the last, I have joined all the parts and made a good layout and interface that looks smooth and simple.

3. Implementation Part

That was the most difficult and important phase which requires a lot of hard work and patience to complete.

So, I have completed this phase by divide it into serval sub-parts

- Tech stack used: 👩‍💻

These were the tech stacks that i have used.

• React Js (Main Framework)
• YouTube API (To get My YouTube Channel Videos Dynamically)
• Redux (For Videos Store)
• Speech Recognition API (To enable voice recognition in website for commands)
• React-speech-kit (To Generate Voice of Website)
• React-youtube (To render youtube videos in youtube video player instead of iframe to enable video voice controls dynamically based on commands)
• JavaScript
• Redux-thunk (To get videos youtube api and directly send to redux)
• React-router-dom (v5 used for routing)[i will update it to v6 in next improvement update]
• Netlify hosting

- Making of Website Interface:

After the designing phase now it's time to make things functional means now I have to implement the design accordingly so that is will be easy to user as well as for a developer to add or give commands.

So, here first I have designed the working interface that is below in the picture.

- Adding Voice commands and responses according to user commands:

After creating the interface the task is to accept voice commands and give responses accordingly both audibly and visually.

That was one of the difficult phases in making this project. I have faced many problems in this phase.

To recognise voice commands i have used speech recognition API that is provided by the JavaScript

Ya i know adding voice recognition with speech recognition API is the easy task but giving responses and rendering videos and doing all dynamic stuff was not that easy.

So, I was facing challenge here so that how should i render videos dynamically with voice commands as all voice commands are handled in main file of website and i have to render that in component.

So, to solve this issue i have used props , redux and State concept of react.

By this it was easy to manage all the states.

So, I have coded all the commands with it.

These are the commands which it supports:-

Stop Taking Voice Commands

Navigation Commands

Form Filling and Submission Commands

Open Video Commands - Home Page

Open Video Commands - Home Page Dialog Box

Open Video Commands - Videos Page

Controlling Video Commands

Open Next/Prev page of pagination - videos page

Go Back/Next Route/History of the application commands

Scrolling commands

Search with voice commands

Open Commands Table command

These all Commands are also mentioned in instruction screen that will come first when you will open website

As this is all done, so now came the most challenging Part

It was to give the responses audibly also after giving the result

What challenge i have faced here?

So, The challenge was that , i was using speech synthesis API to give response audibly but that was not at all working.

my website was not even speaking

I was in the need of hook for react to make website speakable.

So, To solve this issue I have searched over the internet and explored so many websites, then, I found one library... do you know which library I have found?? 🤓

I found react-speech-kit

This is the awesome library

just you have to install this library using command

npm i react-speech-kit
or
yarn add react-speech-kit

Then you have to import it

import { useSpeechSynthesis } from 'react-speech-kit';

You have to import speak function from useSpeechSynthesis hook

const { speak } = useSpeechSynthesis();

And, Then boom you have to use this function to make your website speakable

speak({ text: "I am awesome!" })

- Deployment: 🔌

As it is react so it is easy to deploy.

I have deployed it on Netlify through my website Github repo.

So what's next❓

There are so many things that have to added in this project like:

• Making the experience more smoother
• Adding Voice controlled authentication
• Adding Machine Learning into the project so that is can understand many things on its own.
• and lot more...

But, the most important thing is to turn this project into an open source library as there is not any this type of library present which is open source and free.

Important Links 🔗

• Live Preview :- Voice controllable website deployed on netlify

• Github Repo :- Voice Controlled Website Repo

Checkout My YouTube Channel Connect on LinkedIn

Feel Free to Contribute in this project as it is open source, i would really appreciate your contribution to make this website more better

And at the last, I would like to thank HashNode and Netlify for providing us such an amazing platform and for conducting such a motivational Hackathon where people from around the globe join and build something useful and productive for Open Source. It doesn't matter if someone won or lose all that matters is that every error and struggle comes with new Learnings. 😊

Thank you for Reading 😇