Thursday, December 19, 2013

Just Speak Updated With More Hands-Free Functionality

    *Just Speak*: Controlling Android By Voice

1 JustSpeak: Controlling Android By Voice

JustSpeak is an Android Accessibility Service that enables voice control of your Android device. Once enabled, you can activate on-screen controls, launch installed applications, and trigger other commonly used Android actions using spoken commands.

1.1 Enabling Just Speak

Once installed, you can enable the Just Speak service under Settings->Accessibility on your Android device. Enabling the service is an action that only needs to be performed once as Just Speak will be automatically restarted when the phone is rebooted.

Just Speak is an Accessibility Service that uses Acessibility APIs on the platform to augment Android's user interface. Just Speak augments the Android user interface with voice-input control; other Accessibility Services like TalkBack provide spoken feedback. Note that Just Speak can be used either by itself, or in conjunction with other platform Accessibility Services such as TalkBack.

1.2 Initiating Voice Recognition

Once Just Speak is installed and enabled on your device, you can initiate voice commands by either performing an up-swipe from the Home button (if your device has soft keys) or by performing multiple taps on the Home button (if your device has hard keys). Note that this is the same gesture that activates Google Now on devices running the stock version of Android; When using Just Speak , you can get to Google Now by saying Launch Google Now.

Successful invocation of Just Speak starts voice recognition; this is indicated by playing an auditory icon (accompanied by vibration if available) and a visual overlay. Depending on the settings enabled in Just Speak, as well as other Accessibility Services, received voice input may be spoken and/or displayed after voice recognition has completed. As an example, TalkBack can work in conjunction with Just Speak to speak the recognized commands. Just Speak supports both local and global commands as described in later sections of this document.

1.3 Cancelling Voice Recognition

Voice recognition can be stopped by performing the same action used to initiate voice recognition (either performing an up-swipe from the Home button or by performing multiple clicks of the Home button). Just Speak is also programmed to stop listening if no voice input is received after a specific amount of time. Finally, an overlay, that covers the entire screen, is displayed whenever voice recognition is active; Clicking anywhere on this overlay will dismiss the overlay and terminate voice recognition.

1.4 Global Voice Commands

Just Speak supports a set of global commands that are available on any screen. These global commands include:

Command         Utterance                     Synonym Action                        
Open            Open <Installed Application> Launch, Run Launch Application            
Recent          Recent Apps                        Recent Applications Recent Applications           
Quick Settings Quick Settings                Open Quick Settings         
Toggle WiFi Switch WiFi (On/Off)   Toggle WiFi
Toggle Bluetooth Switch Bluetooth (On/Off)   Toggle Bluetooth
Toggle Tethering Switch Tethering (On/Off)   Toggle Tethering
Home            Go Home                         Return to home screen         
Back            Go Back                         Return to previous screen     
Notifications   Open Notifications              Open notifications shade
Easy Labels Easy Labels   Display Easy Labels

Note that voice commands are flexible in that action words may be substituted with synonymous verbs (eg: “launch” in place of “open”). In addition, voice commands can be formulated as complete sentences, e.g., "Please open GMail".

In addition, Just Speak provides the following spoken aliases as a means of triggering commonly used applications:

Utterance      Action                     
Browser        Launch default Web browser
Web            Launch default Web Browser
OK Google Now Launch Google Now          
Search         Launch Voice Search        
Voice Search   Launch Voice Search        

1.5 Local Voice Commands

In addition to the global commands that are available all screens, Just Speak allows you to interact with on-screen controls in a variety of ways.

Command   Utterance             Synonym Action                                 
Activate Click <control name> Click, Tap Activate control by its on-screen name
Scroll    Scroll Up/Down        Forward, Backward Scroll  e.g., Lists.                   
Switch   Switch On/Off Toggle Toggle Switches
Long Press Long Press Long Click, Long Tap Long Press on on-screen controls
Check Check   Check, Uncheck Toggle CheckBox values

1.6 Labeling On-Screen Control

When using Just Speak, the text labels that appear next to on-screen controls determine the set of available local commands. For many controls, such as images, checkboxes, and switches, there may be no visible text to associate with the control. In these instances, Just Speak uses underlying Accessibility Metadata provided by the application developer to construct relevant labels; note that this metadata is also used by Accessibility Services such as TalkBack to meaningfully speak on-screen controls.

We leverage the visual overlay that indicates that Just Speak is active to visually display this additional metadata — this serves as a hint as to what you can say to activate the available controls. These overlay labels take one of two forms:

  1. Controls receives a centered label in the simple case

where there are no actionable child controls that need additional labeling.

  1. Where the control itself has actionable children,

Just Speak displays a labled frame around the actionable children.

1.7 Chaining Commands

Just Speak can be configured to take multiple voice commands at once and perform them sequentially. This chaining works with both local and global commands, performing the preceding action after the previous action has been executed. Commands are chained via simple connectives such as “and” and “then”. An example of this would be

“Click confirm and then go home”.

In this case, Just Speak would click the “Confirm” control (assuming it’s present) and upon completion of that task, return to the Home screen.

1.8 Easy Labels

Another configurable setting in Just Speak is the ability to replace the labels associated with on-screen controls with easy labels. These are phonetic labels that are designed to be unambiguous and are displayed in the overlay over their respective controls, easily identifying what to say to interact with a specific control. Note that phonetic labels can be temporarily activated or deactivated via the Just Speak global command Easy Labels.

1.9 Persistent Overlay

To aid users with limited dexterity, Just Speak also provides the option to make the overlay persistent, capturing all touch events received by the Android device. This effectively removes all traditional interactions a user can have with their device, replacing it with Just Speak functionality. The benefit of this is that the entire device essentially becomes a button, toggling between initiating and terminating voice recognition. In addition to this, overlay labeling is always present, allowing you to constantly be aware of what you can say to Just Speak.

1.10 Alternative Means Of Initiating Voice Recognition

We are continuing to experiment with alternative ways of initiating voice recognition. Toward this end, Just Speak enables you to configure an NFC tag that can then be tapped to initiate voice control; see “Just Speak -> Settings” for configuring an NFC tag.

Date:      2013-08-06 Tue

Author:    T.V Raman

Created: 2013-12-18 Wed 13:50

Emacs (Org mode 8.2.3a)