Building Voice-Activation Blocks

This tutorial explains the steps involved in building a web-based Voice Activation Block to know temperature & humidity readings from the Temperature & Fire Monitoring & Alerting Tinker Block Application. You will also learn how to build a mobile-app based Voice Activation Block to control the Buzzer Block remotely. Specifically, you will learn how to use voice commands to interact with a Tinker Block Application and have the response announced to you in voice.

Pre-requisites:

Before building Voice-Activation Block, please make sure to build:

Temperature Block, Fire Block, Buzzer Block, Processor Block and Power Block
IoT-based Server Block
IoT-based Device Control Block
Device Integration Block

Approach 1: Building a Web-based Voice-activated Block to view data from Sensor Block

1. Open "Notepad" text editing application on your computer.

2. Copy the below Java Script code and paste it into the Notepad editor.

Click here to view the Java Script code...

<html>

<head>

<title>Tinker Blocks - Voice Activated Application</title>

</head>

<body>

<b> <H1> <blue> Tinker Blocks - Voice Activated Application </blue> </H1> </br>

Tap the "Speech" button and ask for the Sensor reading you wish to know through voice command. For e.g., say "What is the temperature now", "Tinker Blocks, tell me the humidity now".

<button onclick="runSpeechRecog()" style="border: transparent;padding: 0 0.5rem;">Speech </button>

</div>

const outputDiv = document.getElementById('output');

runSpeechRecog = () => {

document.getElementById("output").innerHTML = "Loading text...";

var output = document.getElementById('output');

var action = document.getElementById('action');

let recognization = new webkitSpeechRecognition();

recognization.onstart = () => {

action.innerHTML = "Listening...";

}

recognization.onresult = (e) => {

var transcript = e.results[0][0].transcript;

output.innerHTML = transcript;

if (transcript.includes("temperature")) {

$.getJSON('https://api.thingspeak.com/channels/99999/feed/last.json?api_key=XXXXXXXXXXXXXXXX', function(data) {

var p = data.field1;

outputDiv.textContent = "The temperature now is : " + p + " Degrees Celsius";

const utterance = new SpeechSynthesisUtterance(outputDiv.textContent);

utterance.lang = 'en-US'; // Set the language

speechSynthesis.speak(utterance);

});

}

if (transcript.includes("humidity")) {

$.getJSON('https://api.thingspeak.com/channels/99999/feed/last.json?api_key=XXXXXXXXXXXXXXXX', function(data) {

var p = data.field2;

outputDiv.textContent = "The humidity now is : " + p + " %";

const utterance = new SpeechSynthesisUtterance(outputDiv.textContent);

utterance.lang = 'en-US'; // Set the language

speechSynthesis.speak(utterance);

});

}

output.classList.remove("hide")

action.innerHTML = "";

}

recognization.start();

}

</script>

</center>

</body>

</html>

3. Edit the code as follows:

- Replace 99999 with the Channel ID of your ThingSpeak Channel
- Replace XXXXXXXXXXXXXXXX with the Read API Key of your ThingSpeak Channel.
- Change "field1" to the field number to which you have configured temperature sensor in the ThingSpeak Channel Settings tab. Similarly, change "field2" to the field number to which you have configured temperature sensor in the ThingSpeak Channel Settings tab.
- Change "temperature" to the name of parameter that your sensor gives. For e.g., humidity, motion status, fire status, distance, etc.

4. Save the file as "Voice Activation.html".

5. Open the Voice Activation.html file in your web-browser. To open the file in your web-browser, go to your File Explorer, select the file, right-click on the file-name and choose "Google Chrome" or "Microsoft Edge" to open.

6. You will see a "Speech" button as shown in the below diagram.

7. Click the "Speech" button. It will ask for you to grant permission to your computer's microphone. Grant the permission by click the button "Allow this time".

8. The Java Script application is now ready to listen to your voice command.

9. Use your voice to speak a sentence such as "What is the current temperature?" or "What is the temperature now?", "Give me the latest value of the temperature sensor".

10. The Java script responds by not only displaying the current temperature, but also announcing the same through the computer's speaker, which you can hear loudly.

Approach 2: Building a Mobile-based Voice-activated Block to turn on an Actuator Block

Go to MIT App Inventor and click to "Create Apps". Sign-in with your Google Account.
Once you sign-in, click on "New project" button to create new project and give it a name such as "Voice Activated Mobile App".
Once the new project is created, open the "User Interface" section of the panel on the left side of the screen.
Within the "User Interface" section, drag and drop a "Label" element into the blank mobile screen shown in the designer. Re-name the label component as "Buzzer Control" and change the label's text to "Buzzer Control: ".
Drag and drop a "Button" element into the blank mobile screen shown in the designer. Re-name the button component as "SendCommandButton" and change the button's title as "Say a command".
Drag and drop a "Label" element into the blank mobile screen shown in the designer. Re-name the label component as "Status" and change the label's text as "Status: ".
Open the "Media" section of the panel on the left side of the screen.
Within the "Media" section, drag and drop "SpeechRecognizer" component into the blank mobile screen shown in the designer. Re-name the component as "SpeechRecognizerComponent".
Open the "Connectivity" section of the panel on the left side of the screen.
Within the "Connectivity" section, drag and drop "Web" component into the blank mobile screen shown in the designer. Re-name the component as "CommandWeb".
This completes the design of the mobile app screen. The same will be displayed as shown below:

12. Next, let's add a few blocks of code. Click on "Blocks" button option in the top-right corner of the window.

13. Add the following code blocks to allow the user to click the button and start speech recognition:

14. Next, add the following (independent) blocks of code to recognize the speech to "turn on" or "turn off" the Buzzer device, convert it into a text, and send a TALKBACK request to ThingSpeak Server using the text. In the below example, we replace any space characters in the speech transcript with an underscore ("_"). In addition, we also capitalize the text.

For e.g., if the user utters "turn on", the text is converted into "TURN_ON" and sent as part of the TALKBACK command to ThingSpeak Server to turn-on the Buzzer device. If the user utters "turn off", the text is converted into "TURN_OFF" and sent as part of the TALKBACK command to ThingSpeak Server to turn-off the Buzzer device.

15. In the URL field of the code block, key-in the following:

https://api.thingspeak.com/talkbacks/99999/commands.json?api_key=XXXXXXXXXXXXXXXX

Where 99999 is the Talkback ID and XXXXXXXXXXXXXXXX is the Talkback API key which you created in ThingSpeak earlier. Change them to the Talkback ID and TalkBack API Key that you have configured in ThingSpeak IoT Platform.

16. Next, we add the following blocks of code to receive the response from ThingSpeak Server and indicate the status of the TALKBACK command to the user. If the TALKBACK command was successfully sent to ThingSpeak Server, a "Command sent successfully" status is displayed on the screen. If not, a "Command sending failed" message is displayed on the screen.

17. The Mobile app is now complete and is ready for testing. Create an Android "apk" file by clicking the "Build" --> "Android APK (.apk)" menu option.

18. Once the build is complete, you will see a QR code displayed on the screen. Scan this QR code from a Android-based Smartphone.

19. The App downloads to your smartphone. If necessary, change the settings on your Smartphone to "Allow downloading an App from an unknown source".

20. Once the app downloads to your Smartphone, install the same and open the app. It will appear as seen in the screen-shot shown below-left.

21. When the App opens, tap the "Say a Command". The App now starts listening to your voice.

22. Say "Turn on" and notice the status displayed below the button. Make sure it shows "Success...".

23. Again, tap the "Say a Command". The App starts listening to your voice.

24. Say "Turn off" and notice the status displayed below the button. Make sure it shows "Success...".

25. If you open the TALKBACK view page in ThingSpeak.com, you will see the TURN_ON or TURN_OFF commands added in response to your voice command in the App.

26. If you connect a NodeMCU and upload the Arduino code to read the TALKBACK command, it will read the TALKBACK command and turn on or turn off the Buzzer device connected to NodeMCU.

You can download the above Mobile App by clicking this button:

Download App

You can download the Project file for the above Mobile App by clicking this button:

Download App Project