Android-Accessibility（Android 8.0以上）

摘要： Accessibility Overview Accessible design allows users of all abilities to navigate, understand, and use your UI successfully.Android Acces...

Accessibility Overview

Accessible design allows users of all abilities to navigate, understand, and use your UI successfully.Android Accessibility的目的在於讓所有的使用者都能更方便的使用Android裝置，不僅為殘障人士提供了便利，更是方便了all users，比如你在開車，在做飯的時候。

說個題外話，看Google I/O 2017或2018開發者大會視訊，講解Accessibility in Android的Product Manager（Patrick Clary）and Technical Program Manager（Victor Tsaran）是兩位殘障人士，由衷的敬佩。

Impact of Accessibility

• Increase your app's reach.
• Improve your app's versatility.

全世界約有15%的殘障人士，如果能使你的APP more Accessibility，那麼會有更多的人使用；另外Accessibility不僅僅方便了殘障人士，也能方便所有人，比如你在開車的時候也能用語音代替手勢或點選行為。也有可以做一些外掛，比如微信搶紅包什麼的。

Build accessibility services

在分析內部的AccessibilityService之前，我們先來看看構建accessibility service基礎知識，先有個巨集觀上的認識，再去分析內部實現。

Manifest declarations and permissions

Accessibility service declaration

Accessibility Service也是一個服務，需要在AndroidManifest.xml中宣告，注意這裡必須有BIND_ACCESSIBILITY_SERVICE permission，讓Android System知道可以繫結。

<application>
<service android:name=".MyAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
android:label="@string/accessibility_service_label">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
</service>
</application>

Accessibility service configuration

Accessibility Service還需要配置資訊，指定服務處理時的accessibility events型別和關於服務的附加資訊。配置資訊可以通過AccessibilityServiceInfo類的setServiceInfo()方法配置。Android4.0後，可以在Manifest中包含 < meta-data> 元素。

For example：

AndroidManifest.xml

<service android:name=".MyAccessibilityService">
...
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/accessibility_service_config" />
</service>

< project_dir>/res/xml/accessibility_service_config.xml

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:description="@string/accessibility_service_description"
android:packageNames="com.example.android.apis"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFlags="flagDefault"
android:accessibilityFeedbackType="feedbackSpoken"
android:notificationTimeout="100"
android:canRetrieveWindowContent="true"
android:settingsActivity="com.example.android.accessibility.ServiceSettingsActivity"
/>

屬性的解釋

Accessibility service methods

自己寫的Accessibility Service必須從AccessibilityService繼承，然後再重寫裡面的方法，這些方法都是被Android System呼叫。

onServiceConnected()
onAccessibilityEvent(), onInterrupt()
onUnbind()

Register for accessibility events

Accessibility service configuration一個很重要的功能是告訴指定Accessibility Service可以處理哪些Accessibility Event，這些event可以靠兩種方式判別：

• Package Names
• Event Types

Android Framework可能會把AccessibilityEvent分發給多個Accessibility Service，前提是這些Accessibility Service有不同的Feedback Types。如果多個Accessibility Service的Feedback Types相同，只有第一個註冊的Accessibility Service會接收到這個Accessibility Event。

Accessibility volume

Android 8.0 (API level 26) 及以上，包含了STREAM_ACCESSIBILITY volume category，可以獨立於裝置上其他的聲音，只控制accessibility service的聲音輸出。通過呼叫AudioManager例項的adjustStreamVolume() 方法調節accessibility service的音量。

For example：

import static android.media.AudioManager.*;

public class MyAccessibilityService extends AccessibilityService {
private AudioManager mAudioManager =
(AudioManager) getSystemService(AUDIO_SERVICE);

@Override
public void onAccessibilityEvent(AccessibilityEvent accessibilityEvent) {
AccessibilityNodeInfo interactedNodeInfo =
accessibilityEvent.getSource();
if (interactedNodeInfo.getText().equals("Increase volume")) {
mAudioManager.adjustStreamVolume(AudioManager.STREAM_ACCESSIBILITY,
ADJUST_RAISE, 0);
}
}
}

Accessibility shortcut

使用者可以通過長按兩個音量鍵來啟用和禁用他們喜歡的Accessibility Service。

Accessibility button

導航欄的右側包括一個Accessibility button。當用戶按下這個按鈕時，可以根據螢幕上顯示的內容呼叫幾個啟用的Accessibility Service。Accessibility Service需要addFLAG_REQUEST_ACCESSIBILITY_BUTTON flag（android:accessibilityFlags ），然後呼叫registerAccessibilityButtonCallback() 。

For example：

private AccessibilityButtonController mAccessibilityButtonController;
private AccessibilityButtonController
.AccessibilityButtonCallback mAccessibilityButtonCallback;
private boolean mIsAccessibilityButtonAvailable;

@Override
protected void onServiceConnected() {
mAccessibilityButtonController = getAccessibilityButtonController();
mIsAccessibilityButtonAvailable =
mAccessibilityButtonController.isAccessibilityButtonAvailable();

if (!mIsAccessibilityButtonAvailable) {
return;
}

AccessibilityServiceInfo serviceInfo = getServiceInfo();
serviceInfo.flags
|= AccessibilityServiceInfo.FLAG_REQUEST_ACCESSIBILITY_BUTTON;
setServiceInfo(serviceInfo);

mAccessibilityButtonCallback =
new AccessibilityButtonController.AccessibilityButtonCallback() {
@Override
public void onClicked(AccessibilityButtonController controller) {
Log.d("MY_APP_TAG", "Accessibility button pressed!");

// Add custom logic for a service to react to the
// accessibility button being pressed.
}

@Override
public void onAvailabilityChanged(
AccessibilityButtonController controller, boolean available) {
if (controller.equals(mAccessibilityButtonController)) {
mIsAccessibilityButtonAvailable = available;
}
}
};

if (mAccessibilityButtonCallback != null) {
mAccessibilityButtonController.registerAccessibilityButtonCallback(
mAccessibilityButtonCallback, null);
}
}

Fingerprint gestures

Accessibility Service可以響應另一種輸入機制，即在裝置的指紋感測器上進行定向滑動（向上、向下、左、右）。配置一個Service來接收這些互動的回撥，需要完成以下步驟：

1. 宣告USE_FINGERPRINT permission 和CAPABILITY_CAN_REQUEST_FINGERPRINT_GESTURES capability.
2. 設定FLAG_REQUEST_FINGERPRINT_GESTURES flag（android:accessibilityFlags ）.
3. 註冊函式registerFingerprintGestureCallback() .

For example:

AndroidManifest.xml

<manifest ... >
<uses-permission android:name="android.permission.USE_FINGERPRINT" />
...
<application>
<service android:name="com.example.MyFingerprintGestureService" ... >
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/myfingerprintgestureservice" />
</service>
</application>
</manifest>

myfingerprintgestureservice.xml

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
...
android:accessibilityFlags=" ... |flagRequestFingerprintGestures"
android:canRequestFingerprintGestures="true"
... />

MyFingerprintGestureService.java

import static android.accessibilityservice.FingerprintGestureController.*;

public class MyFingerprintGestureService extends AccessibilityService {
private FingerprintGestureController mGestureController;
private FingerprintGestureController
.FingerprintGestureCallback mFingerprintGestureCallback;
private boolean mIsGestureDetectionAvailable;

@Override
public void onCreate() {
mGestureController = getFingerprintGestureController();
mIsGestureDetectionAvailable =
mGestureController.isGestureDetectionAvailable();
}

@Override
protected void onServiceConnected() {
if (mFingerprintGestureCallback != null
|| !mIsGestureDetectionAvailable) {
return;
}

mFingerprintGestureCallback =
new FingerprintGestureController.FingerprintGestureCallback() {
@Override
public void onGestureDetected(int gesture) {
switch (gesture) {
case FINGERPRINT_GESTURE_SWIPE_DOWN:
moveGameCursorDown();
break;
case FINGERPRINT_GESTURE_SWIPE_LEFT:
moveGameCursorLeft();
break;
case FINGERPRINT_GESTURE_SWIPE_RIGHT:
moveGameCursorRight();
break;
case FINGERPRINT_GESTURE_SWIPE_UP:
moveGameCursorUp();
break;
default:
Log.e(MY_APP_TAG,
"Error: Unknown gesture type detected!");
break;
}
}

@Override
public void onGestureDetectionAvailabilityChanged(boolean available) {
mIsGestureDetectionAvailable = available;
}
};

if (mFingerprintGestureCallback != null) {
mGestureController.registerFingerprintGestureCallback(
mFingerprintGestureCallback, null);
}
}
}

Multilingual text to speech

text-to-speech (TTS) service可以在一個文字塊識別和使用多種語言，要啟用這種功能，需要將LocaleSpan物件中的所有字串封裝起來

For example：

TextView localeWrappedTextView = findViewById(R.id.my_french_greeting_text);
localeWrappedTextView.setText(wrapTextInLocaleSpan("Bonjour!", Locale.FRANCE));

private SpannableStringBuilder wrapTextInLocaleSpan(
CharSequence originalText, Locale loc) {
SpannableStringBuilder myLocaleBuilder =
new SpannableStringBuilder(originalText);
myLocaleBuilder.setSpan(new LocaleSpan(loc), 0,
originalText.length() - 1, 0);
return myLocaleBuilder;
}

Take action for users

從Android 4.0起，Accessibility Service可以代替使用者做出Action，比如改變焦點（焦點就是當前正在處理事件的位置，比如有多個text輸入框，同一時間內只能有一個輸入框可以輸入），模擬點選，模擬手勢等等。

Listen for gestures

Accessibility Service可以監聽特定的手勢，然後代替使用者做出反應。需要設定flagsFLAG_REQUEST_TOUCH_EXPLORATION_MODE

public class MyAccessibilityService extends AccessibilityService {
@Override
public void onCreate() {
getServiceInfo().flags = AccessibilityServiceInfo.FLAG_REQUEST_TOUCH_EXPLORATION_MODE;
}
...
}

Continued gestures

可以通過Path 表示手勢的路徑，然後用GestureDescription.StrokeDescription 構造手勢。

For example：

// Simulates an L-shaped drag path: 200 pixels right, then 200 pixels down.
private void doRightThenDownDrag() {
Path dragRightPath = new Path();
dragRightPath.moveTo(200, 200);
dragRightPath.lineTo(400, 200);
long dragRightDuration = 500L; // 0.5 second

// The starting point of the second path must match
// the ending point of the first path.
Path dragDownPath = new Path();
dragDownPath.moveTo(400, 200);
dragDownPath.lineTo(400, 400);
long dragDownDuration = 500L;
GestureDescription.StrokeDescription rightThenDownDrag =
new GestureDescription.StrokeDescription(dragRightPath, 0L,
dragRightDuration, true);
rightThenDownDrag.continueStroke(dragDownPath, dragRightDuration,
dragDownDuration, false);
}

Use accessibility actions

可以通過getSource() 得到node，然後呼叫performAction做出Action。

public class MyAccessibilityService extends AccessibilityService {

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
// get the source node of the event
AccessibilityNodeInfo nodeInfo = event.getSource();

// Use the event and node information to determine
// what action to take

// take action on behalf of the user
nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SCROLL_FORWARD);

// recycle the nodeInfo object
nodeInfo.recycle();
}
...
}

Use focus types

可以用AccessibilityNodeInfo.findFocus() 查詢node有哪些元素具有Input Focus or Accessibility Focus，還可以使用focusSearch() 選擇Input Focus。最後使用performAction(AccessibilityNodeInfo.ACTION_SET_ACCESSIBILITY_FOCUS) 設定Accessibility Focus。

Gather information

Accessibility services also have standard methods of gathering and representing key units of user-provided information, such as event details, text, and numbers.

Get event details

• AccessibilityEvent.getRecordCount() 或getRecord(int)
• AccessibilityEvent.getSource() - 返回一個AccessibilityNodeInfo 物件

Process text

Hint text

• isShowingHintText() andsetShowingHintText()
• getHintText()

Locations of on-screen text characters

• refreshWithExtraData()

Standardized one-sided range values

一些AccessibilityNodeInfo 物件用AccessibilityNodeInfo.RangeInfo 的例項表示UI元素的範圍值。

Float.NEGATIVE_INFINITY
Float.POSITIVE_INFINITY

Build an accessibility service

這一部分會介紹構建一個accessibility service的flow，從應用程式接收到的資訊，然後該資訊反饋給使用者。

Create your accessibility service

create class

package com.example.android.apis.accessibility;

import android.accessibilityservice.AccessibilityService;
import android.view.accessibility.AccessibilityEvent;

public class MyAccessibilityService extends AccessibilityService {
...
@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
}

@Override
public void onInterrupt() {
}

...
}

AndroidManifest.xml

<application ...>
...
<service android:name=".MyAccessibilityService">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
. . .
</service>
...
</application>

Configure your accessibility service

告訴Android System，你想怎麼執行，何時執行，你想對何種AccessibilityEvent做出迴應，Service是否要監聽所有Application，還是特定的Application，使用哪種feedback types。

有兩種方法，一種是setServiceInfo(android.accessibilityservice.AccessibilityServiceInfo) ，然後重寫onServiceConnected() 。

For example：

@Override
public void onServiceConnected() {
// Set the type of events that this service wants to listen to.Others
// won't be passed to this service.
info.eventTypes = AccessibilityEvent.TYPE_VIEW_CLICKED |
AccessibilityEvent.TYPE_VIEW_FOCUSED;

// If you only want this service to work with specific applications, set their
// package names here.Otherwise, when the service is activated, it will listen
// to events from all applications.
info.packageNames = new String[]
{"com.example.android.myFirstApp", "com.example.android.mySecondApp"};

// Set the type of feedback your service will provide.
info.feedbackType = AccessibilityServiceInfo.FEEDBACK_SPOKEN;

// Default services are invoked only if no package-specific ones are present
// for the type of AccessibilityEvent generated.This service *is*
// application-specific, so the flag isn't necessary.If this was a
// general-purpose service, it would be worth considering setting the
// DEFAULT flag.

// info.flags = AccessibilityServiceInfo.DEFAULT;

info.notificationTimeout = 100;

this.setServiceInfo(info);

}

另一種方法是使用xml方法

For example：

<accessibility-service
android:accessibilityEventTypes="typeViewClicked|typeViewFocused"
android:packageNames="com.example.android.myFirstApp, com.example.android.mySecondApp"
android:accessibilityFeedbackType="feedbackSpoken"
android:notificationTimeout="100"
android:settingsActivity="com.example.android.apis.accessibility.TestBackActivity"
android:canRetrieveWindowContent="true"
/>

同時在AndroidManifest.xml中新增< meta-data>，假設XML file 在res/xml/serviceconfig.xml

<service android:name=".MyAccessibilityService">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data android:name="android.accessibilityservice"
android:resource="@xml/serviceconfig" />
</service>

Respond to accessibility events

當監聽有AccessibilityEvent時，使用onAccessibilityEvent(AccessibilityEvent) 方法做出迴應。用getEventType() 獲取AccessibilityEvent Type，getContentDescription() 提取與觸發事件的檢視有關的標籤文字。

For example:

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
final int eventType = event.getEventType();
String eventText = null;
switch(eventType) {
case AccessibilityEvent.TYPE_VIEW_CLICKED:
eventText = "Clicked: ";
break;
case AccessibilityEvent.TYPE_VIEW_FOCUSED:
eventText = "Focused: ";
break;
}

eventText = eventText + event.getContentDescription();

// Do something nifty with this text, like speak the composed string
// back to the user.
speakToUser(eventText);
...
}

Query the view hierarchy for more context

有時候，需要得到檢視的相關資訊，可以檢視檢視的層次關係，為了做到這一點，需要現在xml中配置。

android:canRetrieveWindowContent="true"

使用getSource() 獲得AccessibilityNodeInfo 物件，當接收到一個AccessibilityEvent時，它會做以下事情：

1.抓住該事件檢視的父節點。

2.在該檢視（父節點）尋找label and check box作為子檢視。

3.如果它找到了它們，就建立一個字串來向用戶報告，指示標籤，以及是否檢查了它。

4.如果在遍歷檢視層級時返回null（不管什麼時候），那麼該方法就會放棄。

For example：

// Alternative onAccessibilityEvent, that uses AccessibilityNodeInfo

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {

AccessibilityNodeInfo source = event.getSource();
if (source == null) {
return;
}

// Grab the parent of the view that fired the event.
AccessibilityNodeInfo rowNode = getListItemNodeInfo(source);
if (rowNode == null) {
return;
}

// Using this parent, get references to both child nodes, the label and the checkbox.
AccessibilityNodeInfo labelNode = rowNode.getChild(0);
if (labelNode == null) {
rowNode.recycle();
return;
}

AccessibilityNodeInfo completeNode = rowNode.getChild(1);
if (completeNode == null) {
rowNode.recycle();
return;
}

// Determine what the task is and whether or not it's complete, based on
// the text inside the label, and the state of the check-box.
if (rowNode.getChildCount() < 2 || !rowNode.getChild(1).isCheckable()) {
rowNode.recycle();
return;
}

CharSequence taskLabel = labelNode.getText();
final boolean isComplete = completeNode.isChecked();
String completeStr = null;

if (isComplete) {
completeStr = getString(R.string.checked);
} else {
completeStr = getString(R.string.not_checked);
}
String reportStr = taskLabel + completeStr;
speakToUser(reportStr);
}

以上基本是Android官網的學習資料，下一篇會學習AccessibilityService的原始碼，看看內部是怎麼實現的。