Building a Voice-Driven Chatbot Front End

Every piece exists now: Lesson 2’s mic button transcribes speech into the widget’s input field, Lesson 4’s local Piper service turns a reply back into audio. This lesson connects both to Course 11’s actual chat widget, the one with a toggle button, message panel, and REST-backed sendMessage(), so it becomes something a visitor can speak to and hear from, without touching anything about how that widget already talks to wp_ai_client_prompt() on the backend.

Step 1: Auto-submit after a transcript arrives

Lesson 2 filled the input field but left the send button for the visitor to press. For a voice-first flow, submitting automatically once recognition ends is what makes it feel like a conversation instead of a two-step chore:

// File: assets/js/voice-input.js (extending Lesson 2's recognition.addEventListener('result', ...))
recognition.addEventListener( 'result', function ( event ) {
	var transcript = event.results[ 0 ][ 0 ].transcript;
	input.value = transcript;

	var form = document.getElementById( 'site-chat-form' );
	form.dispatchEvent( new Event( 'submit', { cancelable: true } ) );
} );

Course 11’s existing form.addEventListener( 'submit', ... ) handler runs exactly as it already did, appending the visitor’s bubble and calling sendMessage(), it has no idea whether the text arrived by typing or by voice, which is exactly the point.

Step 2: Speak the assistant’s reply back

Course 11’s sendMessage() eventually calls appendMessage( 'assistant', text ) once a real reply comes back from the REST endpoint. Hook into that same point to also speak it, using Lesson 4’s local Piper service:

// File: assets/js/voice-output.js
function speakReply( text ) {
	if ( ! window.siteChatWidget || ! window.siteChatWidget.voiceEnabled ) {
		return;
	}

	fetch( window.siteChatWidget.ttsUrl, {
		method: 'POST',
		headers: {
			'Content-Type': 'application/json',
			'X-WP-Nonce': window.siteChatWidget.nonce,
		},
		body: JSON.stringify( { text: text } ),
	} )
		.then( function ( response ) {
			return response.json();
		} )
		.then( function ( data ) {
			if ( data.audio_url ) {
				new Audio( data.audio_url ).play();
			}
		} )
		.catch( function ( error ) {
			console.error( 'Text-to-speech playback failed:', error );
		} );
}

// File: inc/rest-tts-endpoint.php
add_action( 'rest_api_init', function () {
	register_rest_route( 'my-plugin/v1', '/speak', array(
		'methods'             => 'POST',
		'callback'            => 'my_plugin_handle_speak_request',
		'permission_callback' => '__return_true',
	) );
} );

function my_plugin_handle_speak_request( WP_REST_Request $request ) {
	$text = sanitize_textarea_field( $request->get_param( 'text' ) );

	if ( ! $text ) {
		return new WP_Error( 'missing_text', 'No text provided.', array( 'status' => 400 ) );
	}

	$url = my_plugin_speak_text( $text ); // Lesson 4's function

	if ( is_wp_error( $url ) ) {
		return $url;
	}

	return array( 'audio_url' => $url );
}

Calling speakReply() right where Course 11’s appendMessage( 'assistant', text ) already runs is the one edit that ties this lesson into that widget’s existing code, no restructuring of the message flow required.

Step 3: A visible state for listening and speaking

// File: assets/js/voice-output.js (continued)
function speakReply( text ) {
	// ...fetch as above, then:
	var audio = new Audio( data.audio_url );
	document.getElementById( 'site-chat-widget' ).classList.add( 'is-speaking' );
	audio.addEventListener( 'ended', function () {
		document.getElementById( 'site-chat-widget' ).classList.remove( 'is-speaking' );
	} );
	audio.play();
}

Step 4: Let voice be opt-in, not a replacement

// File: assets/js/chat-widget.js (a small addition to Course 11's markup)
'<label class="site-chat-voice-toggle">' +
'  <input type="checkbox" id="site-chat-voice-enabled" />' +
'  Enable voice replies' +
'</label>'

document.getElementById( 'site-chat-voice-enabled' ).addEventListener( 'change', function ( e ) {
	window.siteChatWidget.voiceEnabled = e.target.checked;
} );

Text-only visitors, and anyone who simply prefers reading over audio playing on a page, should be able to turn this off with one click, speakReply()’s own guard clause at the top already checks this flag before doing anything.

Test it: a full spoken round trip

Click the mic, ask a real question out loud, and confirm three things happen in order: the transcript appears in the input and submits itself, the assistant’s text reply appears in the message panel exactly as Course 11 built it, and, with the voice toggle on, you actually hear that reply spoken back through Lesson 4’s Piper service or the SpeechSynthesis fallback.

Recap

Nothing about Course 11’s chat widget changed structurally. Lesson 2’s transcript now auto-submits through the widget’s existing form handler, and a new speakReply() call, hooked in exactly where the widget already appends an assistant message, plays Lesson 4’s synthesized audio back. A visible listening and speaking state, and a one-click way to turn voice off entirely, keep this honest about being an addition to a working text widget, not a replacement for it.

Resources & further reading

• Building the Chat Widget UI, Course 11
• SpeechSynthesisUtterance, MDN Web Docs, developer.mozilla.org
• Autoplay guide for media and Web Audio APIs, MDN Web Docs, developer.mozilla.org