Just Speak Updated With More Hands-Free Functionality

    *Just Speak*: Controlling Android By Voice

1 JustSpeak: Controlling Android By Voice

JustSpeak is an Android Accessibility Service that enables voice control of your Android device. Once enabled, you can activate on-screen controls, launch installed applications, and trigger other commonly used Android actions using spoken commands.

1.1 Enabling Just Speak

Once installed, you can enable the Just Speak service under Settings->Accessibility on your Android device. Enabling the service is an action that only needs to be performed once as Just Speak will be automatically restarted when the phone is rebooted.

Just Speak is an Accessibility Service that uses Acessibility APIs on the platform to augment Android's user interface. Just Speak augments the Android user interface with voice-input control; other Accessibility Services like TalkBack provide spoken feedback. Note that Just Speak can be used either by itself, or in conjunction with other platform Accessibility Services such as TalkBack.

1.2 Initiating Voice Recognition

Once Just Speak is installed and enabled on your device, you can initiate voice commands by either performing an up-swipe from the Home button (if your device has soft keys) or by performing multiple taps on the Home button (if your device has hard keys). Note that this is the same gesture that activates Google Now on devices running the stock version of Android; When using Just Speak , you can get to Google Now by saying Launch Google Now.

Successful invocation of Just Speak starts voice recognition; this is indicated by playing an auditory icon (accompanied by vibration if available) and a visual overlay. Depending on the settings enabled in Just Speak, as well as other Accessibility Services, received voice input may be spoken and/or displayed after voice recognition has completed. As an example, TalkBack can work in conjunction with Just Speak to speak the recognized commands. Just Speak supports both local and global commands as described in later sections of this document.

1.3 Cancelling Voice Recognition

Voice recognition can be stopped by performing the same action used to initiate voice recognition (either performing an up-swipe from the Home button or by performing multiple clicks of the Home button). Just Speak is also programmed to stop listening if no voice input is received after a specific amount of time. Finally, an overlay, that covers the entire screen, is displayed whenever voice recognition is active; Clicking anywhere on this overlay will dismiss the overlay and terminate voice recognition.

1.4 Global Voice Commands

Just Speak supports a set of global commands that are available on any screen. These global commands include:

Command	Utterance	Synonym	Action
Open	Open <Installed Application>	Launch, Run	Launch Application
Recent	Recent Apps	Recent Applications	Recent Applications
Quick Settings	Quick Settings	Open	Quick Settings
Toggle WiFi	Switch WiFi (On/Off)		Toggle WiFi
Toggle Bluetooth	Switch Bluetooth (On/Off)		Toggle Bluetooth
Toggle Tethering	Switch Tethering (On/Off)		Toggle Tethering
Home	Go Home		Return to home screen
Back	Go Back		Return to previous screen
Notifications	Open Notifications		Open notifications shade
Easy Labels	Easy Labels		Display Easy Labels

Note that voice commands are flexible in that action words may be substituted with synonymous verbs (eg: “launch” in place of “open”). In addition, voice commands can be formulated as complete sentences, e.g., "Please open GMail".

In addition, Just Speak provides the following spoken aliases as a means of triggering commonly used applications:

Utterance	Action
Browser	Launch default Web browser
Web	Launch default Web Browser
OK Google Now	Launch Google Now
Search	Launch Voice Search
Voice Search	Launch Voice Search

1.5 Local Voice Commands

In addition to the global commands that are available all screens, Just Speak allows you to interact with on-screen controls in a variety of ways.

Command	Utterance	Synonym	Action
Activate	Click <control name>	Click, Tap	Activate control by its on-screen name
Scroll	Scroll Up/Down	Forward, Backward	Scroll  e.g., Lists.
Switch		Switch On/Off Toggle	Toggle Switches
Long Press	Long Press	Long Click, Long Tap	Long Press on on-screen controls
Check	Check	Check, Uncheck	Toggle CheckBox values

1.6 Labeling On-Screen Control

When using Just Speak, the text labels that appear next to on-screen controls determine the set of available local commands. For many controls, such as images, checkboxes, and switches, there may be no visible text to associate with the control. In these instances, Just Speak uses underlying Accessibility Metadata provided by the application developer to construct relevant labels; note that this metadata is also used by Accessibility Services such as TalkBack to meaningfully speak on-screen controls.

We leverage the visual overlay that indicates that Just Speak is active to visually display this additional metadata — this serves as a hint as to what you can say to activate the available controls. These overlay labels take one of two forms:

1. Controls receives a centered label in the simple case

where there are no actionable child controls that need additional labeling.

1. Where the control itself has actionable children,

Just Speak displays a labled frame around the actionable children.

1.7 Chaining Commands

Just Speak can be configured to take multiple voice commands at once and perform them sequentially. This chaining works with both local and global commands, performing the preceding action after the previous action has been executed. Commands are chained via simple connectives such as “and” and “then”. An example of this would be

“Click confirm and then go home”.

In this case, Just Speak would click the “Confirm” control (assuming it’s present) and upon completion of that task, return to the Home screen.

1.8 Easy Labels

Another configurable setting in Just Speak is the ability to replace the labels associated with on-screen controls with easy labels. These are phonetic labels that are designed to be unambiguous and are displayed in the overlay over their respective controls, easily identifying what to say to interact with a specific control. Note that phonetic labels can be temporarily activated or deactivated via the Just Speak global command Easy Labels.

1.9 Persistent Overlay

To aid users with limited dexterity, Just Speak also provides the option to make the overlay persistent, capturing all touch events received by the Android device. This effectively removes all traditional interactions a user can have with their device, replacing it with Just Speak functionality. The benefit of this is that the entire device essentially becomes a button, toggling between initiating and terminating voice recognition. In addition to this, overlay labeling is always present, allowing you to constantly be aware of what you can say to Just Speak.

1.10 Alternative Means Of Initiating Voice Recognition

We are continuing to experiment with alternative ways of initiating voice recognition. Toward this end, Just Speak enables you to configure an NFC tag that can then be tapped to initiate voice control; see “Just Speak -> Settings” for configuring an NFC tag.

Date:      2013-08-06 Tue

Author:    T.V Raman

Created: 2013-12-18 Wed 13:50

Emacs 24.3.50.1 (Org mode 8.2.3a)

Validate

JustSpeak: Control All Aspects Of Android By Voice

JustSpeak: Controlling Android By Voice

Table of Contents

• 1 JustSpeak: Controlling Android By Voice
  • 1.1 Starting JustSpeak
  • 1.2 Initiating Voice Commands
  • 1.3 Global Voice Commands
  • 1.4 Local Voice Commands
  • 1.5 Chaining Commands
  • 1.6 Alternative Means Of Triggering JustSpeak

1 JustSpeak: Controlling Android By Voice

JustSpeak is an Android Accessibility Service that enables voice control of your Android device. Once enabled, you can activate on-screen controls, launch installed applications, and trigger other commonly used Android actions using spoken commands.

1.1 Starting JustSpeak

Once installed, you can start JustSpeak under Settings->Accessibility on your Android device — you only need to do this once, i.e., JustSpeak will be automatically restarted when you reboot your phone.

JustSpeak is an Accessibility Service i.e., once started, it uses Acessibility APIs on the platform to augment Android's user interface. JustSpeak augments the Android user interface with voice-input control; other Accessibility Services like TalkBack provide spoken feedback. Note that JustSpeak can be used either by itself, or in conjunction with other platform Accessibility Services such as TalkBack.

1.2 Initiating Voice Commands

Once started, JustSpeak registers itself as an Assistant on your device; you can initiate voice commands by swiping up from the bottom of the screen. Note that this is the same gesture that activates Google Now on devices running the stock version of Android. When using JustSpeak, you can get to Google Now by saying Google. Successful invocation of voice-control produces an auditory icon, indicating that your device is ready to listen. The auditory tone, along with an optional vibration (if available) you hear is your cue to start speaking; JustSpeak displays a visual overlay on the right-edge of the screen to indicate that it is listening. the recognized utterance is displayed visually, — additionally, it is spoken out if TalkBack is active. JustSpeak supports two types of voice control commands as described below.

1.3 Global Voice Commands

JustSpeak supports a set of global commands — these are global in that they are available on any screen. Global commands include:

Command	Utterance	Action
Open	Open <Installed Application>	Launch Application
Recent	Recent	Recent Applications
Quick Settings	Quick Settings	Launch Quick Settings
Toggle WiFi	Switch WiFi (On/Off)	Toggle WiFi
Toggle Bluetooth	Switch Bluetooth (On/Off)	Toggle Bluetooth
Toggle Tethering	Switch Tethering (On/Off)	Toggle Tethering
Home	Go Home	Return to home screen
Back	Go Back	Return to previous screen
Notifications	Open Notifications	Pull down notifications shade

Note that voice commands are flexible in that you can use a number of synonyms for verbs such as open e.g., launch. In addition, voice control commands can be formulated as full sentences, e.g., "Please open GMail".

List Of Synonyms For Open:

• open
• go to
• show
• display

In addition, JustSpeak provides the following spoken aliases as a means of triggering commonly used applications:

Utterance	Action
Browser	Launch default Web browser
Web	Launch default Web Browser
OK Google Now	Launch Google Now
Search	Launch Voice Search
Voice Search	Launch Voice Search

1.4 Local Voice Commands

In addition to the global voice commands that are available on any screen, JustSpeak lets you activate on-screen controls in a variety of ways.

Command	Utterance	Action
Activate	Click <control name>	Activate control by its on-screen name
Toggle	Turn on/off <toggle>	Toggle on/off switch
Scroll	Scroll Up/Down	Scroll e.g., Lists.

List Of Synonym For Activate:

• touch
• click on
• press
• activate
• open
• push
• tap

1.5 Chaining Commands

Note that you can issue a sequence of commands via a single utterance by using simple connectives such as then or and when speaking; as an example, you can say:

Quick Settings then  WiFi

to open "Quick Settings" and then click the "WiFi" item that appears within quick settings. This is an example of chaining together a global command followed by a local command that becomes available after the global command has executed successfully. Similarly, you can say:

Open GMail, then compose

to launch GMail, and immediately start composing a new message.

1.6 Alternative Means Of Triggering JustSpeak

We are continuing to experiment with alternative ways of initiating voice control. Toward this end, JustSpeak enables you to configure an NFC tag that can then be tapped to initiate voice control; see JustSpeak->Settings for configuring an NFC tag. On Android 4.3 and above, you can also trigger JustSpeak by long-pressing both volume buttons at the same time.

Date: 2013-08-06 Tue

Author: T.V Raman

Org version 7.9.3f with Emacs version 24

Validate XHTML 1.0

Jelly Bean: Accessibility Gestures Explained

1 Jelly Bean: Accessibility Gestures Explained

This article details accessibility gestures in Jelly Bean and is a follow-up to Jelly Bean Accessibility Explained. It gives a conceptual overview of the 16 possible gestures and describes how they are used. The interaction behavior described here holds for all aspects of the Android user interface, including interactive Web pages within Chrome and the Android Web Browser.

1.1 Conceptual Overview Of The Gestures

Placing a finger on the screen speaks the item under the finger by first placing Accessibility Focus on that item. Moving the finger triggers touch exploration which moves Accessibility Focus.

To generate any of the Accessibility Gestures discussed below, one moves the finger much faster — how much faster is something we will tune over time, and if needed, make user customizable.

To remember the gestures, think of the four directions, Up, Down, Left and Right. In addition to these four basic navigational gestures, we defined an additional 12 gestures by picking pairwise combinations of these directional flicks, e.g., Left then Down — this gives a total of 16 possible gestures. In what follows, left then down means swipe left, and continue with a down flick. Note that in performing these additional gestures, speed matters the most. As an example, it is not essential that you make a perfect Capital L when performing the Down then Right gesture for instance; speed throughout the gesture, as well as ensuring that the finger moves some distance in each direction is key to avoid these being misinterpreted as basic navigational gestures.

1.2 Accessibility Focus And Accessibility Gestures

Accessibility Focus is moved using the four basic directional gestures. For now we have aliased Left with Up, and Down with Right; i.e., both Left and Up move to the previous item, whereas Down and Right move to the next item. Note that this is not the same as moving with a physical D-Pad or keyboard on Android; the Android platform moves System Focus in response to the D-Pad. Thus, moving with a D-Pad or trackball moves you through the various interactive controls on the screen; moving Accessibility Focus via the Accessibility Gestures moves you through everything on the screen.

1.3 Accessibility Gestures For Common Actions

In addition to the basic navigation describe above, we define the following gestures for common actions:

Navigation Granularity

You can increase or decrease navigation granularity by rapidly stroking Up then Down or Down then Up.

Scrolling Lists

You can scroll a list forward by rapidly stroking Right then Left; the reverse, i.e., Left then Right scrolls the list backward by a screenful.

1.4 User Configurable Gestures

Gestures Down then Left, Up then Left, Down then Right and Up then Right are user configurable; their default assignments are shown below.

Back

Gesture Down then Left is the same as pressing the Back button.

Home

Up then Left has the same effect as pressing the Home button.

Status Bar

Gesture Up then Right opens Status Notifications.

Recent

Down then Right has the same effect as pressing the Recent Applications button.

1.5 Summary

Gestures for manipulating and working with Accessibility Focus are an evolving part of the Android Accessibility; we will continue to refine these based on user experience. At this point, you are probably saying:

But wait, you said 16 gestures, but only told us the meanings of 12 of them

You are correct — we have left ourselves some gestures to use for future features.

Jelly Bean Accessibility Explained:Touch Exploration Augmented By Gestures

1 Jelly Bean Accessibility Explained:Touch Exploration Augmented By Gestures

We announced a number of accessibility enhancements in Android Jelly Bean — see our Google IO 2012 announcements and our Android Accessibility talk from I/O 2012. This article gives a user-centric overview of the Jelly Bean interaction model as enabled by touch exploration and navigational gestures. Note that as with every release, Android Accesssibility continues to evolve, and so as before, what we have today is by no means the final word.

1.1 High-Level Concepts

First, here's some shared vocabulary to ensure that we're all talking of the same thing when explaining Jelly Bean access:

Random Access

Enable user to reach any part of the on-screen UI with equal ease. We enabled this as of ICS with touch exploration.

Deterministic Access

Enable user to reliably land on a desired item on the screen. We enable this in Jelly Bean with linear navigation.

Accessibility Focus

The item that the user most recently interacted with — either via touch exploaration or linear navigation receives accessibility focus.

Activation

User can activate item having accessibility focus by double-tapping anywhere on the screen.

1.2 Sample Interaction Scenarios We Enable

The combination of random access via touch exploration, backed up by linear navigation starting from the point the user just explored enables users to:

• Touch explore an application to understand its screen layout,
• Use muscle memory to quickly touch parts of the display to access familiar application screens,
• Use linear navigation to reach the desired item when muscle memory is wrong by a small amount.

As an example, when using the Google Play Store, I can use muscle memory with touch exploration to find the Search button in the top action bar. Having found an application to install, I can once again use muscle memory to roughly touch in the vicinity of the Install button; If what I touch is not the Install button, I can typically find it with one or two linear navigation steps. Having found the Install button, I can double-tap anywhere on the screen.

The same use case in ICS where we lacked Accessibility Focus and linear navigation would have forced me to use touch exploration exclusively. In instances where muscle memory worked perfectly, this form of interaction was highly effective; but in our experience, it also tended to lead to breakdowns and consequent user frustration in instances where users almost found the control they were looking for but missed by a small amount.

Having introduced accessibility focus and linear navigation in Jelly Bean, we decided to eliminate the ICS requirement that the user tap on or near a control to activate it — we now enable users to activate the item with accessibility focus by tapping anywhere on the screen. To eliminate spurious taps --- especially on tablets, we made this a double-tap rather than a single tap. Note: based on user experience, we may optionally bring back single tap at some point in the future as an end-user customization.

1.3 Summary

Android Accessibility continues to move forward with Jelly Bean, and will continue to evolve rapidly along with the platform. Please use the Eyes-Free Google Group to provide constructive feedback on what works or doesn't work for you --- what is most effective is to objectively describe a given use case and your particular experience.

What's New In Google Accessibility From Google I/O 2012

Google IO 2012: What's New With Google Access

1 Google IO 2012: What's New From Google Access

We showcased a number of exciting advances in accessibility on Android and Chrome during IO 2012. With these advances, blind and low-vision users can leverage Google applications and services on Android and Chrome to collaborate effectively with their peers and to obtain on-the-go access. Advances include out-of-the-box access on Android (JellyBean), a new set of gestures that enable fluent interaction on touch-screen devices, Braille support on Android, an extension framework for ChromeVox, and a new, high-quality voice for use with Web applications on Chrome.

1.1 Enhanced Android Accessibility In JellyBean:

• Accessibility on Android can be enabled by long-pressing with two fingers (4 seconds) the setup screen to enable out-of-the-box access for blind users.
• Touch exploration has been enhanced with simple gestures that enable users navigate on-screen contents.
• JellyBean provides a set of Accessibility Actions that can be called from any AccessibilityService such as TalkBack; it also provides early support for Braille displays.
• Touch exploration and gesture navigation both set AccessibilityFocus —

double-tapping anywhere on the screen activates the item with AccessibilityFocus .

• TalkBack now has a sister service BrailleBack for providing Braille support on Android.

• Chrome on Android is now accessible and supports the latest in Web Access standards.

With these enhancements in Android access, blind users can use a combination of touch exploration and navigational gestures to access any part of the Android user interface.

As an example, I typically use the Android Play Store by touching the screen around the area where I expect a specific control; quick flicks of the finger then immediately get me to the item I want. With these touch gestures in place, I now use touch exploration to learn the layout of an application; with applications that I use often, I use a combination of muscle memory and gesture navigation for rapid task completion.

1.2 Chrome OS On Chrome Books And Chrome Box

Chrome OS comes with ChromeVox pre-configured — ChromeVox is our Web Accessibility solution for blind users. With the new high-quality voice that is being released on the Chrome Webstore, ChromeVox now provides smooth spoken feedback in Chrome on all desktop environments. Finally, we announced a flexible extension framework that enables Web developers leverage ChromeVox from within and outside of their own Web applications to provide rich, contextual spoken feedback.

1.3 Developer Tools For Ensuring Accessibility

To help developers better leverage Web Accessibility, we are releasing a new Accessibility Audit tool that enables Web developers detect and fix commonly occuring accessibility errors. This tool has been integrated into Chrome's Developer Tools and helps Web developers ensure accessibility while working within their normal workflow.

1.4 Accessibility Related Presentations At Google I/O 2012

Catch these on Youtube in the next week if you weren't able to attend I/O this week.

• Android Accessibility (T. V. Raman,Peter Lundblad, Alan Viverette and Charles Chen).
• Advanced Web Accessibility (Rachel Shearer, Dominic Mazzoni and Charles Chen).
• What's New In JellyBean: Android Team.
• JellyBean announcement in the Wednesday keynote.

Date: 2012-06-21 Thu

Author: T.V Raman

Org version 7.8.11 with Emacs version 24

Validate XHTML 1.0

Accessible GMail On Android — Eyes-Free Email On The Go

1 Accessible GMail On Android — Eyes-Free Email On The Go

I've been using Android as my primary smart phone since late 2008, and the level of email access I've had on Android in the past has always been a source of frustration. About a year ago, I first started accessing my email on Android with K9-Mail — that helped me bridge some of the accessibility gaps on the platform.

Over the last few months, our friends over in GMail Mobile have been adding accessibility support to the GMail client on Android. What is truly exciting is that this support is being added to existing releases of Android including Froyo (Android 2.2) and GingerBread (Android 2.3). This means that GMail on Android is now accessible on existing devices — get the update from Market and give it a spin.

1.1 Typical Usage Pattern

Here is my typical usage pattern when accessing my corporate email at Google. Note that the volume of email I receive on this account is extremely high, and includes many mailing lists that I typically do not read while on a mobile device. To limit how much email I download to the mobile device, and to ensure that I attend to the most pressing email messages while on the go I do the following:

• I have defined a GMail filter that assigns label to-mobile to messages I want to access when on the go.
• Typically, this includes email addressed directly to me, and other priority items.
• I launch GMail to open to this label.
• I quickly skim through the list of displayed messages to here the subject and a quick overview of the message.
• If I decide to read the complete message, I select that message via the trackball on my Nexus One to hear the message in its entirety.
• And finding an email thread I am looking for is just one click away — press the search button, and use up/down to navigate your search history.

See our help center documentation for additional details.

Author: T.V Raman <raman@google.com>

Date: 2011-08-10 Wed

HTML generated by org-mode 6.30c in emacs 23

Leveraging Android Access From Google IO 2011

You can watch our Google IO 2011 on Levarging Android Access APIs. The main take-aways from the talk:

• Android Access is easy --- the framework does most of the heavy-lifting.
• Implementing Android Access does not mean you take a performance hit.
• Accessibility is really about expanding the reach of your application.

Implementing accessibility within your application and thereby ensuring that it is usable in a wide variety of end-user scenarios will benefit your application --- both in terms of the number of users you gain, as well as how often your users use your application.

TalkBack Refreshed: Accessible On-Screen Keyboard And More ...

Android Access: TalkBack Refreshed

1 Android Access: TalkBack Refreshed

The latest enhancements to TalkBack now brings Android Accessibility to devices without a physical keyboard. Many of these enhancements also improve the overall TalkBack experience on all devices.

1.1 Highlights

• New TalkBack Keyboard.
• On-screen talking keyboard enables text entry via the touch screen.
• Text review provides spoken feedback when moving the cursor by character, word, sentence, or paragraph.
• Virtual D-Pad for navigating the Android user interface.
• Global TalkBack commands enable one-click access to oft-used commands.

1.2 TalkBack Keyboard

The TalkBack Keyboard is an Accessible Input Method (Accessible IME) that when activated enables you to enter and review text via the touch screen. To use this feature, you need to first activate the TalkBack keyboard via the Language and Keyboard option in the Settings menu. Next, customize the TalkBack Keyboard to taste via the TalkBack Keyboard Settings option --- here, you can customize additional features including auditory feedback as you type. Finally, open your favorite editing application, long-press on an edit field, and select TalkBack keyboard as your default IME. Note that you need do this only once; once the TalkBack keyboard has been made the default, it persists across reboots.

1.3 Entering Text On The Touch Screen

TalkBack keyboard is an on-screen keyboard that supports touch exploration along with synchronized spoken and auditory feedback. This means you can now enter text when using devices that don't sport a physical keyboard.

But wait, there's more here than meets the finger at first touch. Once you have activated the TalkBack Keyboard, you can switch the keyboard among three states by long-pressing the volume up/down buttons:

Hidden

The TalkBack keyboard is not displayed.

Navigating

You get access to an on-screen virtual D-Pad, along with Back, Home, Search, and Menu buttons.

Typing

An on-screen qwerty keyboard.

My preferred means of using the keyboard is to turn on auditory feedback from within TalkBack Keyboard Settings, as well as having SoundBack active. In this mode, you hear keys as you explore the keyboard along with an auditory icon; picking up your finger types the last key you explored. Typing produces a distinctive key-click.

The on-screen keyboard occupies the bottom 1/3 of your screen. While entering text, explore and find the top row, then move above it to hear what you have typed so far.

1.4 Reviewing Text By Character, Word, Sentence Or Paragraph

You can now navigate and review text by character, word, sentence or paragraph. Use a two-finger tap to move forward through these navigation levels; a two-finger double tap moves in the reverse direction. Once you have selected your preferred mode of navigation, you can use Up/Down on the physical track-ball/D-Pad, or alternatively, flick up or down on the virtual D-Pad to move forward or backward through the text being reviewed.

Note that text review works when the TalkBack keyboard is in either/navigating/ or typing mode; personally, I find it less error-prone on keyboard-less devices to first switch to navigating mode when reviewing text, since it is easy to inadvertently enter spurious text otherwise.

1.5 Using The On-Screen Virtual D-Pad

Placing the TalkBack keyboard in navigating mode provides an on-screen virtual D-Pad --- this is especially useful on devices that do not have a physical D-Pad or track-ball on the front of the device. When active, the virtual D-Pad occupies the bottom one-third of the screen, and fast-flicks in that area has the same effect as moving with a D-Pad or track-ball. Tapping anywhere within the virtual D-Pad is the same as clicking with the track-ball.

The corners of the virtual D-Pad also provides Back, Home, Search and Menu buttons --- these are especially useful on devices that lack explicit physical or capacitive buttons for these common Android actions. You can explore the virtual D-pad by moving your finger around the D-Pad area; crossing the top-edge of this area provides haptic and auditory feedback that can be used as an orientation aid in finding the virtual buttons on the corners.

1.6 Global Commands

In addition, selecting the TalkBack Keyboard as your default input method enables a set of global commands that can be accessed from your physical keyboard --- eventually, we will make these available via the soft keyboard as well. Here are a list of the current commands:

Command	Description	Key
Battery	Speaks the current battery level	menu + B
Time	Speaks the current date and time	menu + T
Connectivity	Speaks the connectivity state of each connection: WiFi, 3G, etc	menu + O
Repeat	Repeats the last TalkBack utterance	menu + R
Spell	Spells the last TalkBack utterance	menu + S

These shortcuts are listed in the Accessibility Preferences application where they can be edited. You can choose between menu and search for the modifier, and any letter on the keyboard for the letter.

1.7 Summary

All of these features work on Android 2.2 and above. In addition, TalkBack makes WebView accessible in Honeycomb --- look for a separate announcement about accessibility enhancements that are exclusive to the Honeycomb release in the coming weeks.

Author: T.V Raman

Date: 2011-03-16 Wed

HTML generated by org-mode 7.4 in emacs 24

Eyes-Free Shell Refreshed

We just refreshed Eyes-Free Shell on Android Market with a long-overdue set of improvements that have been waiting to launch. Here is a brief summary of user-visible changes:

User Customizable Home Screen

You can now add additional pages of short-cuts to the home screen. You can flip through these pages of short-cuts by tapping the left or right edge of the screen. Pressing the menu key within a page of shortcuts allows you to customize the short-cuts on that page; it also provides controls for inserting new short-cut pages.

One-Click Uninstall

The default way of managing applications in Android requires many clicks through nested menus --- this is especially true when uninstalling applications. The Eyes-Free Shell now lets you uninstall applications by pressing menu while in the applications list .

I18N

Spanish and Chinese strings for Eyes-Free Shell.

And many more underlying changes too numerous to fit in this margin. Speak, Listen, And Enjoy!

Intersection Explorer --- Now Intersections Sound Even Better

We just updated Intersection Explorer on Android Market. This version improves on the initial launch by providing more intuitive descriptions for intersections, I'll include some examples below:

T-Intersection

Minor Street ends in Main Street to form a T-Intersection Depending on where you explore from, you hear:

• Currently at Minor Street ends in Main Street
• Currently at right on to Minor Street from Main Street.
• Currently at left on to Minor Street from Main Street.

Plus-Intersection

Given the 4-way intersection of Castro Street and El Camino, you hear one of the following depending on the direction you're exploring:

• Currently at Castro Street crosses El Camino
• Currently at El Camino crosses Castro Street.

And a lot more than will fit this margin --- explore, share and enjoy!

Walking About With A Talking Android

1 Walking About With a Talking Android

I have long relied on spoken directions from Google Maps on the desktop. As I access more and more of my online world through my Android phone, Google's recent announcement of GMM4.5 enhanced with walking directions means that I now have superior functionality to what I have enjoyed at my desk --- but now with the added benefit of having it all in my pocket!

Inclusion of step-by-step walking directions on Android now allows me to specify a destination on my TalkBack enabledeyes-free Android device, and have these spoken to me as I walk. But wait, there's more!

We're launching a new member of our Eyes-Free family of programs for Android --- WalkyTalky that goes hand-in-hand with spoken walking directions from Google Maps to better navigate the physical world. In addition,application Intersection Explorer allows me to explore the layout of streets using touch before venturing out with WalkyTalky.

1.1 WalkyTalky

WalkyTalky is an Android application that speaks the address of nearby locations as you pass them. It also provides more direct access to the walking directions component of Google Maps. With WalkyTalky installed, you can:

• Launch WalkyTalky to specify a destination,
• Either specify the destination by address, or pick from favorites or recently visited locations,
• And in addition to spoken walking directions,
• Hear street addresses as you walk by.

These spoken updates, in conjunction with the walking directions that are spoken by Google Maps help me navigate the physical world as efficiently as I navigate the Internet.

1.2 Intersection Explorer

Often, I like exploring a neighborhood to learn the layout of the streets before actually venturing out with my trusty companion,Hubbell Labrador, and this is where Intersection Explorer comes into its own. Using this application, I can explore any neighborhood on Google Maps via touch exploration.

1.2.1 How It Works

• Intersection Explorer starts off at the user's current location.
• One can change the start position by entering an address, to do this, press menu and click on new location.
• Once the map has loaded, touching the screen speaks the streets at the nearest intersection.
• Moving one's finger along a compass direction, and then tracing a circle speaks each street at that intersection along with the associated compass direction.
• Presence of streets is cued by a slight vibration as one traces the circle.
• Lifting up the finger when on a street moves in that direction to the next intersection, speaks the distance moved, and finally speaks the newly arrived-at intersection.

1.3 Summary

Together, Intersection Explorer and WalkyTalky, in conjunction with Walking Directions from Google Maps brings a new level ofaccess to my physical world. I use these tools in conjunction with other Maps-based applications such as the Places Directory on Android --- this is another application from the Google Maps team that works fluently with TalkBack on Android to help me find nearby attractions or other locations of interest.

So next time you take your trusty Android out for a walk, make sure to give these new tools a spin --- you can report back on your experience via our Eyes-Free Group.

Applications WalkyTalky and Intersection Explorer can be downloaded from the Android Market.Share And Enjoy, and as usual, remember, The Best Is Yet To Come!

Author: T.V Raman

Date: 2010-09-09 Thu

HTML generated by org-mode 7.01 in emacs 24

QR Code for WalkyTalky:



QR Code for Intersection Explorer:

TalkBack, Eyes-Free Shell Refreshed --- Now With End-User Settings

$

We are pushing out a series of updates via Android Market for TalkBack and the Eyes-Free Shell. Here is a brief overview of end-user visible changes.

Accessibility Preferences

Going by the principle of things should just work as expected , we have long resisted giving in to having a complex set of user preference settings for TalkBack and friends --- in my experience, if you introduce such a settings menu early on, we as software engineers tend to punt on all complex decisions by turning each question into a complex user-facing dialog. That said, it is now time to gradually introduce end-user settings for some aspects of the various accessibility tools.

Accessibility Preferences

Welcome new application AccessibilityPreferences to Android. What this application does:

• From an end-user perspective, it provides a single place where you will find preference settings corresponding to each accessibility tool you have installed on your phone.
• For developers of accessibility tools, it provides a simple means of registering a custom program for managing end-user preferences for that tool.

TalkBack installs its user preferences under this tool. You can tweak a number of settings that affect TalkBack behavior including:

• Control whether TalkBack speaks when the screen is off --- useful to silence status messages when you have the phone turned off.
• Control whether TalkBack speaks when ringer volume is set to 0, i.e., phone is in silent mode.
• Control whether the proximity sensor is used to shut off speech.

Over time, we'll add more settings here as appropriate --- but expect us to be conservative with respect to how many settings show up.

Updates To The Eyes-Free Shell

Here is a summary of updates to the Eyes-Free Shell:

• Changes the proximity sensor logic so that it is only active when the shell is active; this should be more battery efficient
• Fixes a race condition bug that can trigger when the shell is being exited as an application is being installed/removed

TalkBack

Here is a summary of changes to TalkBack:

• TalkBack now includes application-specific speech strategies for some popular applications. This provides context-sensitive spoken feedback.
• Applications that have such speech strategies defined include Facebook, Stitcher and GoogleVoice amongst others.
• Implements a settings screen that can be used with Accessibility Preferences
• Available settings:
   
  1. Ringer Volume (Speak at all ringer volumes, No speech in Silent Mode, No speech in Vibrate and Silent Mode)
   
  2. Screen Status (Allow speech when screen is off, No speech when screen is off)
   
  3. Speak Caller ID (checked/not checked)
   
  4. Proximity Sensor (checked/not checked)

In addition, TalkBack introduces the ability to add application-specific plugins --- expect to see more advancement here in future releases.

AccessibilityPreferences Hints For Developers

If you're a developer of an AccessibilityService, you need to:

• Implement a preferences screen for your application.
• Implement this with intent filter:

              
  
              <intent-filter>            
  
                  <action android:name="android.intent.action.MAIN" />            
  
                  <category android:name="android.accessibilityservice.SERVICE_SETTINGS" />            
  
              </intent-filter>            
  
          
  

Share And Enjoy,

Eyes-Free Review: Droid2 From MOT

Here is a quick eyes-free access overview of the MOT Droid2.

Hardware

1. The device has a pull-out keyboard, and the buttons are much more tactile than the original Droid.
2. The device also has dropped the hard-to-use D-Pad from the original Droid in favor of PC-style arrow keys.
3. There is once again no dedicated number row at the top.
4. The capacitive buttons on the front of the device appear in a different order from the original Droid --- with the device in portrait mode, reading left to right you have: Menu, home, back, and search.
5. In addition, MOT ships a voice search application on the device that is triggered by pressing a special microphone button -- it's worth learning the position of this key, since voice-search can be useful --- and more importantly, if you're relying on spoken feedback, hitting this button leads to the phone falling inexplicably silent.

Software

If you look under accessibility, you'll find an application called Voice Readouts from MOT. This appears to be a screenreader analogous to TalkBack, though in my experience, it did not produce spoken feedback in many instances. That said, this application collaborates well with TalkBack --- and after installing TalkBack from the Android Market (note: the Droid2 does not come with TalkBack bundled) -- you can activate both TalkBack and VoiceReadout for an optimal experience.

VoiceReadout appears to have a preliminary version of touch-exploration. With VoiceReadout active, a single tap speaks the item under the finger; a double-tap activates that item. Note that moving the finger around on the display does not appear to trigger touch exploration; also, touch exploration appears to be available in only some contexts.

Instances where touch exploration appears to be active

1. Settings application.
2. Portions of Android Market.

In general, touch exploration appears to be available in ListView.

In addition, the Droid2 also includes a low-vision accessibility tool called Zoom Mode ( look for it under Settings -> Accessibility ) this tool provides a magnification lens.

Summary

All in all, the Droid2 appears to be one of the better choices for eyes-free use from among the presently available crop of Android phones. Touch exploration, though preliminary, is nice to see on the platform, and the bundled low-vision magnification aid is a nice touch. Voice Readouts is also a great example of an Android accessibility service done right in that it co-exists peacefully with other screenreaders like TalkBack to provide an optimal end-user experience. To users not familiar with adaptive technologies in general, this might not sound like a big deal --- but users of PC screenreders have long been familiar with the need to have only one screenreader turned on. As we transition to modern platforms like Android, it's useful to remind ourselves that screenreaders can in fact co-exist, with each tool providing something useful to create an overall experience that is greater than the sum of the parts.

Welcoming Loquendo Susan To Android (FroYo)

Android 2.2 (AKA FroYo) introduces many platform enhancements, and one that I find particularly relevant is the ability to plug-in additional Text-To-Speech engines. What this means from an end-user point of view:

• Android comes with a set of built-in voices since Android 1.6 --- these are the Pico voices for English, French, Italian, German and Spanish.
• With the Text-ToSpeech plug-in mechanism in place, we can now add new engines to the platform.
• The first such add-on was ESpeak, which brings support for many of the world's languages.
• And now, vendors are able to sell high-quality add-on voices via the Android Market.
• Loquendo Susan is the first commercially available voice for Android. Users running FroYo can buy this voice on the Android Market. Thanks to the plug-in mechanism, once you buy a new voice, you can switch all your talking applications to use the newly installed voice --- see instructions below.

Activating And Using Newly Installed Voices

Goto Settings → Voice Input And Output → Text To Speech Settings. First, activate the newly installed voice by clicking the corresponding checkbox item for that voice. Next, go to Default Engine in the Text To Speech Settings menu, and make the newly installed voice your default engine. Finaly, if you want all applications to use the new voice, check option Always use my settings

With this in place, my Nexus and Droid both speak using Loquendo Susan --- thus turning my Android into into a truly pleasant eyes-free device.

Stitcher And TalkBack: The World In My Ears

Shortwave Radio --- and DXing was one my hobbies growing up--- I spent many hours listening to far-off radio stations ---and in the process developed a love for languages. Fast forward to the late 90's, and one could now listen to radio stations from all around the world on the Internet --- but this time without the hiss and static of shortwave propogation. But there was a catch --- you needed to be at your computer to listen to these stations. At home, I solved this problem by setting up a set of living room speakers connected to the computer in my office-bedroom; with a wireless keyboard, this brought Internet radio to my living room.

Fast-forward to the next decade, and I now have the Internet in my pocket in the form of a smart phone. I recently discovered Stitcher on the Android Market --- and it got me the final mile to having ubiquitous access to Internet Radio!

Using Stitcher With TalkBack

There is little more to say other than try it out! .Stitcher on Android is a simple Android application that worksout of the box with TalkBack. Once you install stitcher fromMarket, use the arrow keys or trackball on your phone to browse through the various categories. Clicking on stations launchesplayback immediately. Note that for now, the stop buttonin the player is not navigable by the trackball --- I have gotten used to hitting it by dead-reckoning since it always appears in afixed position. In the last few weeks, stitcher hasreplaced StreamFuriously , my previous Internet Radio solution on Android.

So here's to happy listening!A brief note on the title of this post --- The World In My Ears was also the title of abook on DXing by Arthur Cushen from New Zealand --- I remember hearing his voice in the 80's on the BBC's World Service.

An Eyes-Free View Of Android At The Google IO Sandbox

Google IO 2010 is playing home to over 5,000 attendees in San Francisco this week. A number of Google Access engineers are at the conference consuming and producing information --- here is a brief view of some of the exciting bits seen on the Android show floor from an eyes-free perspective.

Hardware And New Devices From An Eyes Free Perspective

Many of the phone manufacturers were showing off their latest devices on the show floor --- visit the Android Sandbox at Google IO to see these first hand. Charles and I walked through the various displays Wednesday (May 19) afternoon to test drive these devices first-hand --- given the large number of Android devices coming out every week, this was a unique opportunity to see many of these devices for the first time. Here are some highlights:

• All devices were running Android 1.6 or later, and consequently, Settings/Accessibility was available on every device. Having worked on this for the last 2 years, it's extremely gratifying to see phone manufacturers including accessibility in their devices.
• We found one device from Motorola where we couldn't find the accessibility setting --- the booth representative promised to check after we pointed this out --- waiting to hear back.
• My favorite device was the LG Ally --- check this device out if you get a chance.
  • Device to be sold by Verizon.
  • Device has an elegant tactual feel.
  • Front of the device sports hardware answer/hangup buttons.
  • The pull-out qwerty keyboard is a pleasure to use --- I would rate this one of the best designed cell phone keyboards I've seen.
• Android devices continue to show up in many shapes and sizes --- re-emphasizing that there is a device for everyone. This makes it even more important to choose a device that meets your particular needs.

Software --- Android Applications Galore

We also visited the various vendors showing off their latest Android applications. What was gratifying was that even though most of these developers had paid little thought to eyes-free use --- and were blissfully unaware of the existence of an Android Accessibility API, their applications worked for the most part with Accessibility enabled. Where there were gaps, we were able to show developers what they needed to do --- everyone was extremely receptive. Below is a brief summary of what we saw --- and a shout-out to all the friendly developers we met:

Where

This is a very accessible application I have been using for a while --- the developers were thrilled to hear that it was accessible since they had made no special effort.

Aloqua

A competing application to Where with a very slick visual UI. This application doesn't raise the appropriate Access Events at present because it's a custom UI. When we first talked to their lead developer he was extremely hesitant saying I dont want to change my custom UI. However, I could hear his face light up when we said You dont need to change your look and feel --- you just need to set a couple of custom Java properties (specifically, property ContentDescription

Pandora

Another favorite of mine that works well with access --- except --- the player controls are unlabeled. I showed them the application in action on my Droid --- looking forward to seeing this application become even more usable.

NPR News

There are many NPR tools on the Android Market --- NPR News is the official application. The application was originally written by a Googler and Open Sourced --- I have been using it for about 4 months and it's completely accessible. It could do with some power-user shortcut keys to make it even more efficient.

MLB At Bat

I had originally played with this application during last year's World Series; at the time, the application was quite usable with TalkBack. I'm happy to report that nothing has regressed --- the application still continues to to work well, except for a couple of glitches with unlabled player controls. The booth representatives had actually heard of accessibility --- and were receptive to fixing the remaining issues.

Summary: The light-weight design of the Android Access layer has proven valuable in making sure that it makes it on to every device. The minimal set of responsibilities the API places on developers has meant that a large number of Android applications are accessible out of the box.

Audio Books On Android --- Thanks Librivox!

In my previous article , I alluded to an Audio Books application forAndroid. I did not go into much detail on the application itselfbecause I felt it deserved an article of its own.So heregoes!

In Praise Of Librivox

If you aren't familiar with the Librivox project, please visit Librivox.org to see the wonderful work that that project is doing. Androidapplication AudioBooks brings the wonders of Librivoxto Android --- now, you can carry all 30,000 audio books andcounting in your pocket and access them anywhere .Here are some highlights:

• Browse, and quickly play available audio books. You canbrowse by several criteria.
• Books you listen to get downloaded to your device and areavailable for offline listening.
• All books provide a table of contents, allowing you to jumpto a specific portion of a book.
• 90% of the application user interface is completelyaccessible with TalkBack --- see below for missing accessfeatures.

The only glitche with using application AudioBooks with the Android Access API is that the player controls withinthe audio-book player are presently missing contentdescriptions --- this is Android-API speak to say that thecontrols are images with missing labels. So the first time youuse this app, you'll need someone to tell you the buttons ---alternatively just experiment to discover theirfunctions. There are pause, play, rewind and forward buttons ---if the friendly folk who developed this application stumble uponthis post, please get in touch, and I can show you what you needto add to your code to make the eyes-free experience evensmoother.

Happy Listening --- And Share And Enjoy!

Using Android Market Eyes-Free

The Android Market is a treasure-trove of applications --- many of which work out of the box with Android's Access API, and as a result, the freely available screenreaders on the platform. Working with Market can be initially daunting, given the large collection applications; additionally, there are a couple of spots in the workflow that need access improvements. While we get those fixes pushed, here is a step-by-step overview of using Android Market with TalkBack, including the work-arounds for moving over some of the afore-mentioned hurdles.

Android Market: A Brief Overview

Rather than giving a detailed explanation of all of Android Market's user interface, I'll sketch my day-to-day mode of using Market --- personally, I find task-oriented help guides far more usable.

Task: Find Application

• I typically launch Android Market from within the Applications list in the Eyes-Free shell. On my Droid, I typically do this with the keyboard already opened since I know I'll be typing very soon.
• I press the Search capacitive button on the bottomright of the display to bring up the search tool. Note that Market can sometime take a few seconds to launch depending on your network --- TalkBack should announce Market when it's ready.
• Type a search query --- as an example, try audio books
• Use the D-Pad arrow keys (up/down) to navigate the list of results. TalkBack speaks each entry as you move through the list.
• Find one you like; for this example, we'll use one of my favorite Market applications --- AudioBooks from project Librivox.
• Press the Enter key on the keyboard to open this application
• This takes you to a screen that lists a short description, and comments from various users on the application. The install button is on the bottom of this screen.
• And here comes the sticking point in the Market UI that we're working on fixing; when you cursor through this list, you dont always get to the install button.But no fear, you can still install the application!
• While we work on creating and pushing the fix for the above, I typically install applications by tapping the screen where the install button appears. The bad news is that Ipresently do this by dead reckoning; the good news is that the install button always appears at a consistent spot. The easiest way to learn to do this is to have someone put your finger on the button the first time, and then learn its position relative to the pull-out keyboard. While we know that this is not an ideal eyes-free experience, this little trick opens up a treasure-trove of applications.
• Tap the install button, and you come to the permissions screen. Cursor to the OK button, and press Enter Depending on the layout of that screen, you may once again need to use dead-reckoning. At this point, I routinely click those on-screen buttons, rather than wasting time attempting to cursor to the button.
• And voila, the AudioBooks application should download and install!

Task: Browse Market

In addition to searching, you can also browse the Market for available applications, use the cursor keys on the D-Pad for browsing. Once selected, installing an application follows the same workflow as above.

And The Best Is Yet To Come

Once installed, you can try out the application by pulling down the status bar. Look for the next posting in this series for details on using application AudioBooks --- it is one of my all time Market favorites.

Eyes-Free: TalkBack And Shell Improvements

Here is a brief summary of updates to Android's eyes-free tools--- including TalkBack, and the Eyes-Free Shell from the last two weeks.

TalkBack

• Speech during a phone call is now re-enabled.
• Turning the screen on/off is spoken. This announcement includes the ringer mode/volume.
• Changes in the the ringer mode - silent, vibrate, and normal are now announced.
• Unlocking the phone is announced.
• Other Android applications can programmatically discover if TalkBack is enabled.

Eyes-Free Shell

Now that applications can programmatically discover whetherTalkBack has been enabled, configuring Eyes-Free shell to become your default home screen has become a lot easier. In a nutshell,if you are a TalkBack user and install Eyes-Free shell, hitting the Home button will bring up the eyes-free shell, ---no configuration needed. Note that you can always get to the default Android home screen by long-pressing the Back button.

Share And Enjoy

Eyes-Free Updates: Marvin And TalkBack Simplified

We routinely push updates to our access tools on Android; users get these updates automatically via Android Market updates. We just pushed out updated versions of TalkBack, our Open Source screenreader for Android, and Marvin, the Eyes-Free shell. Here is a brief summary of these updates:

• Android applications can now programmatically discover if TalkBack is running, thanks to the latest changes in TalkBack. From an end-user perspective, this means that you no longer need to configure Eyes-Free shell via EyesFreeConfig to be the default home. If you run TalkBack, and have EyesFree Shell installed, then pressing Home automatically gives you the EyesFree Shell. Remember, you can always get to the default Android Home by long-pressing Back.
• EyesFree Shell now includes a touch-based shortcuts manager. Until now, shortcuts needed to be explicitly configured by editting an XML file on the SDCard. With the recent EyesFree update, you can interactively define short-cuts via a touch-based ShortCuts manager. By default, we have assigned shortcut 1 to the ShortCuts manager; so to invoke this new feature, do:
  1. Stroke left (4 using stroke dialer notation) to enter the shortcuts screen.
  2. Stroke up and to the left (1 using stroke-dialer notation) to invoke application ShortCuts Manager.
  3. Use the trackball/D-Pad to configure each of the 8 available shortcuts.

Marvin: We hope this gives some minimal relief to the pain in all the diodes on your left side.