CTS介紹

CTS簡介：Compatibility Test suite系列相容測試
google定義了一個相容性規範(Compatible Definition),而CTS就是為了確保某個測試符合該規範。從而基於的應用程式能在基於同一個api版本的裝置上面執行。通過CTS測試的device可以獲得android的商標，並且享受android market的許可權

一般CTS測試有三項：Cts,Gts,Ctsverify，CTS測試工具可以在官方網上下載（），選擇對應Android版本的CDD文件（介紹CTS測試），CTS測試包下載（測試包會不斷更新），將下載的CTS測試包解壓到相應資料夾裡面應包含：android-cts資料夾裡面包含Docs、Repository、Tools三個資料夾，

Android6.0的測試包和Android7.0的CTS測試包已經有些不同，而Media Files最新為CTS Media 1.2.，

測試環境

·        linux系統（最好是64位ubuntu10.04）

·        需正確配置JavaJDK（1.7版本以上）和Android SDK 和ADB，確保系統和手機能夠正常的連線

  下載java  jdk 1.8   地址 :

下載android sdk 

android官網下載sdk,注意現在官網不推薦單獨安裝sdk，都是和Android Studio繫結在一起，單獨的sdk比較難找，放在網頁的最下面。

下載完成後 ,需要在下載相應的api

配置環境變數 ,這裡配置全域性的環境變數 

$sudo vim /etc/profile

#set java environment

JAVA_HOME=/usr/java/jdk1.8.0_181
JRE_HOME=/usr/java/jdk1.8.0_181/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

export ANDROID_HOME=/home/ctstest/androidsdk/android-sdk-linux
export PATH=$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$PATH

更新下環境變數

source /etc/profile

檢視配置是否正確  

java 

adb devices

測試前準備

·        下載需要進行測試的軟體版本   -------使用者版本

·        連線IPV6翻牆網路

·        插入可用的SD卡，容量最好大於1G

·        插入可用的Sim卡，如果為雙卡版本，需確保卡一的卡槽有可用的Sim卡.

·        寫入合法的IMEI號，需要寫入號碼     ----- 卡要寫號碼

·        檢查手機是否設定了CTS標誌位  -------- cts標示位置

·        手機恢復出廠設定（Settings→Backup andreset→Factory data reset）

·        確保SD中有足夠的空間（大於1G），可將SD格式化清空。

·        確保手機裡沒有Google賬號或者其他賬號存在

·        確保日期和時間的正確(開了Wifi或者資料連線以後會自動更新時間，時區選預設的)，設定太平洋時間gtm：-8

·        設定螢幕為常亮(Settings→Display→Sleep→Never)

·        設定無鎖屏(Settings→Security→Set up screenlock→None)

·        手機恢復出廠設定（Settings→Backup andreset→Factory data reset）

·        2.1_R2到4.2_R4之前，安裝CtsDelegatingAccessibilityService.apk,
(在android-cts/repository/testcases下，可用adb install安裝)，並進入Settings→Accessibility開啟Delegating Accessibility Service.

·         

·        安裝CtsDeviceAdmin.apk,(在android-cts/repository/testcases下可用adbinstall CtsDeviceAdmin.apk),並進入Settings→Security→Deviceadministrators開啟裡面的兩個包

·         

·        將媒體檔案(android-cts-media-1.2.zip解壓出來的檔案),名稱為test的檔案，拷貝到手機外部SD卡上，可使用指令碼拷貝(在android-cts-media-1.0目錄下執行bash copy_media.shall)

·         

·        確保開發者選項中的USB debugging,Stay awake,Allowmock locations三個選項已勾選。

·        設定系統語言為美式英語(Settings→Language&input→Language)

·        設定系統輸入法為谷歌預設輸入法
(Settings→Language&input→Default→English(US))

·        開啟Wifi，開啟資料連線，開啟GPS

·        確保USB連線穩定

執行測試

整體測試

測試之前手機需要安裝CtsDeviceAdmin.apk在終端中輸入：
    cts/android-cts/repository/testcases$adbinstall     CtsDeviceAdmin.apk
    在Settings-Security-Deviceadministrators下勾選
    Android Devices Manage
    android.deviceadmin.cts.CtsDevicesAdminReceiver
    android.deviceadmin.cts.CtsDevicesAdminReceiver2

l  用adb devices檢視手機是否連上

n  進入測試環境

l  進入CTS測試

[email protected]:~/cts/6.0_r14/android-cts$cd tools/

[email protected]:~/cts/6.0_r14/android-cts/tools$./cts-tradefed

Android CTS 6.0_r14 build:3623550

03-17 16:46:58 I/DeviceManager: Detectednew device c23392e

l  cts-tf > runcts --plan CTS --disable-reboot  //開始執CTS整個測試包

l cts-tf> help   使用幫助文件

Run:

  runcts --plan test_plan_name: run a test plan

  runcts --package/-p : run a CTS test package

  runcts --class/-c [--method/-m] : run a specific test class and/ormethod

  run cts --continue-session session_ID: run allnot executed tests from a previous CTS session

  runcts [options] --serial/-s device_ID: run CTS on specified device

  runcts [options] --shards number_of_shards: shard a CTS run into given number ofindependent chunks, to run on multiple devices inparallel

  runcts --help/--help-all: get more help on running CTS

List:

 l/list d/devices: list connected devices and their state

 l/list packages: list CTS test packages

 l/list p/plans: list CTS test plans

  l/list i/invocations: list invocations aka CTStest runs currentlyin progress

 l/list c/commands: list commands: aka CTS test run commands currently inthe queue waiting to be allocated devices

 l/list r/results: list CTS results currently present in the repository

Add:

  addderivedplan --plan plane_name --session/-s session_id -r[pass/fail/notExecuted/timeout]: derive a plan from the given session

Dump:

 d/dump l/logs: dump the tradefed logs for all running invocations

Options:

  --disable-reboot: Do not reboot device after running some amount of tests.

l  cts-tf > l r     檢視失敗專案  成功專案 ，未執行專案

Session Pass  Fail  Not Executed Start time           Plan name  Device serial(s) 

0       2743  130   3555         2017.01.18_16.46.14  CTS1       24ade972

l cts-tf> run cts --continue-session 0 --disable-reboot

 繼續執行未執行的專案

cts-tf > l r

Session Pass  Fail  Not Executed Start time           Plan name  Device serial(s) 

0       2743  130   3555         2017.01.18_16.46.14  CTS1       24ade972          

l cts-tf> add derivedplan --plan testname -s 0 -r fail   

l cts-tf> run cts --plan testname      執行失敗專案

l 多臺機器同時執行

Android6.0測試需要IPV6環境，可以用命令run cts –plan CTS –skip-preconditions
Android6.0測試需要的時間比較長，google提供了多臺手機一起測試的功能：
r un cts –plan CTS –shards 手機數量 –skip-preconditions

 比如我需要3臺機器一起測試就可以輸入命令：run cts –plan CTS –shards 3–skip-preconditions
    另外：在終端中輸入help可以參看cts的相關命令
    run cts –plan test_plan_name 測試一個plan
    run cts –packages/-p packages_name 單獨測試一個package
    run cts –class/-c class_name–method/-m method_name 測試一個類和方法
    如：run cts -candroid.hardware.camera2.cts.BurstCaptureTest -m testYuvBurst

l i 命令檢視執行時間 cts-tf > l i
l r 檢視測試結果 cts-tf > l r

CTS v1 console command reference

Table1. This table summarizes the CTS v1 console commands for varioususes.

Host	Description
help	Display a summary of the most commonly used commands
help all	Display the complete list of available commands
exit	Gracefully exit the CTS console. Console will close when all currently running tests are finished
Run	Description
run cts	Run the specified tests and displays progress information. One of --plan, --package, --class or --continue-session needs to be specified The CTS console can accept other commands while tests are in progress If no devices are connected, the CTS desktop machine (or host) will wait for a device to be connected before starting tests If more than one device is connected, the CTS host will choose a device automatically
--plan <test_plan_name>	Run the specified test plan
--package/-p <test_package_name>  [--package/-p <test_package2>...]	Run the specified test packages
--class/-c <class_name> [--method/-m <test_method_name>	Run the specified test class and/or method
--continue-session	Run all not executed tests from previous CTS session; the sessions testResult.xml will be updated with the new results
--shards <number_of_shards>	Shard a CTS run into given number of independent chunks, to run on multiple devices in parallel
--serial/-s <deviceID>	Run CTS on the specific device
-t <class_name>#<test_method_name>	Run a specific test method
--force-abi 32|64	On 64-bit devices, run the test against only the 32-bit or 64-bit ABI
List	Description
list packages	List all available test packages in the repository
list plans	List all available test plans in the repository
list invocations	List 'run' commands currently being executed on devices
list commands	List all 'run' commands currently in the queue waiting to be assigned to devices
list results	List CTS results currently stored in repository
list devices	List currently connected devices and their state 'Available' devices are functioning, idle devices, available for running tests 'Unavailable' devices are devices visible via adb, but are not responding to adb commands and won't be allocated for tests 'Allocated' devices are devices currently running tests
Add	Description
add derivedplan --plan <plan_name>  --result/-r  [pass | fail | timeout | notExecuted]  [--session/-s <session_id>]	Create a plan derived from given result session; use this option to rerun reports and validate test issues

For Android 7.0 or later, you'll use CTS v2.

CTS v2 console command reference

Table1. This table summarizes the CTS V2 console commands for varioususes.

Host	Description
help	Display a summary of the most commonly used commands
help all	Display the complete list of available commands
version	Show the version.
exit	Gracefully exit the CTS console. Console will close when all currently running tests are finished.
Run	Description
run cts	Run the default CTS plan (that is, the full CTS invocation). The CTS console can accept other commands while tests are in progress. If no devices are connected, the CTS desktop machine (or host) will wait for a device to be connected before starting tests. If more than one device is connected, the CTS host will choose a device automatically.
--plan <test_plan_name>	Run the specified test plan.
--module/-m <test_module_name>  [--module/-m <test_module2>...]	Run the specified test module or modules. For example, run cts --module CtsGestureTestCases executes the gesture test module (this can be shortened to run cts -m Gesture). run cts -m Gesture --test android.gesture.cts.GestureTest#testGetStrokesruns the specific package, class, or test.
--module/-m <test_module_name> -- test <test_name>	Run the specified module and test. For example, run cts -m Gesture --test android.gesture.cts. GestureTest#testGetStrokes runs the specific package, class, or test.
--retry	Retry all tests that failed or were not executed from the previous sessions. Use list results to get the session id.
--shards <number_of_shards>	Shard a CTS run into given number of independent chunks, to run on multiple devices in parallel.
--serial/-s <deviceID>	Run CTS on the specific device.
--include-filter <module_name>  [--include-filter <module2>...]	Run only with the specified modules.
--exclude-filter <module_name>  [--exclude-filter <module2>...]	Exclude the specified modules from the run.
--log-level-display/-l <log_level>	Run with the minimum specified log level displayed to STDOUT. Valid values: [VERBOSE, DEBUG, INFO, WARN, ERROR, ASSERT].
--abi <abi_name>	Force the test to run on the given ABI, 32 or 64. By default CTS runs a test once for each ABI the device supports.
--logcat, --bugreport, and --screenshoot-on-failure	Give more visibility into failures and can help with diagnostics.
--device-token	Specifies a given device has the given token eg. --device-token 1a2b3c4d:sim-card..
--skip-device-info	Skips collection of information about the device. Note: do not use this option when running CTS for approval.
--skip-preconditions	Bypasses verification and setup of the device's configuration, such as pushing media files or checking for Wi-Fi connection.
List	Description
list modules	List all available test modules in the repository.
list plansor list configs	List all available test plans (configs) in the repository.
list invocations	List 'run' commands currently being executed on devices.
list commands	List all 'run' commands currently in the queue waiting to be assigned to devices.
list results	List CTS results currently stored in repository.
list devices	List currently connected devices and their state. 'Available' devices are functioning, idle devices, available for running tests. 'Unavailable' devices are devices visible via adb, but are not responding to adb commands and won't be allocated for tests. 'Allocated' devices are devices currently running tests.
Dump	Description
dump logs	Dump the tradefed logs for all running invocations.
ADD	Description    ps:
a/add s/subplan	Create a subplan used to create a subplan
--session <session_id>	The session used to create a subplan
--result-type <status>	Which result to include in the subplan
--name/-n <subplan_name>	The name of the new subplan
Cts-tf> run cts --retry  sessionId --retry-type FAILED/NOT_EXECUTED 執行未完成的任務 ，輸出結果

    測試模組下的單個模組

     測試模組 

      比如我們要測試 CtsMediaTestCases  ，我們知道這個模組下面有連個模組 ：

      armeabi-v7a CtsMediaTestCases

      如果要整體這是這個模組 ，  可以使用

       cts-tf > run cts -m CtsMediaTestCases

     要是測試模組下的其中一個小模組 ，我們就需要新增一個

--abi <abi_name>

Force the test to run on the given ABI, 32 or 64. By default CTS runs a test once for each ABI the device supports.

        cts-tf > run cts -m CtsMediaTestCases -a  armeabi-v7a

 android 8.1 CTS 測試  

1. 首先當然還是要下載測試套件  https://source.android.com/compatibility/cts/downloads                                        Android 8.1 is the release of the development milestone code-named Oreo-MR1.The source code for the following tests can be synced with the'android-cts-8.1_r5' tag in the open-source tree.  
2. 當然還是少不了CTS Media Files 的  CTS Media 1.4    （有4G大小了）
3. 後面的手機配置基本上都是大同小異 ，這裡就不在重複
4. 這裡按照CTS v2 console command reference 相關命令來執行 
5. 執行命令 run cts   預設是執行 Run the default CTS plan (that is, the full CTS invocation).
6. run cts --shards <number_of_shards>   : 表示在幾個裝置上執行  run cts --shards 2  ,代表在2個裝置上執行
7. 最後執行完成後就會給你有提示 ，坐等測試結果就好了 （記得不要開啟整個測試的結果 ，有時間會很大，導致瀏覽器卡死，所以還是開啟失敗的html）

l  測試結束後         /repository/results/2016.05.27_21.29.17/testResult.xml使用瀏覽器開啟，檢視測試的結果

可以進入android-cts/repository/results/2016.05.27_21.29.17輸入命令： sudo gedit testResult.xml 在這裡必須使用命令開啟，否則不能儲存，可以看到測試結果：

 最近因為測試 的需要,接觸到了Android cts測試 ,以上是最近整理的一點東西 ,如果有不對的地方,也請看到的大神指點一二,謝謝 !

FAQ

l  修改瞭如下內容  ，正常測試沒有問題，可不需要修改

增加了環境變數 ，使用的是6.0的aapt ，這是要測試的相應的版本的aapt

1 。[email protected]:~# gedit .bashrc

exportANDROID_HOME=/home/liangcheng/android-sdk-linux      #/*SDK所在的路徑*/

exportPATH=$PATH:/home/liangcheng/android-sdk-linux/build-tools/23.0.3

l  手機在linux上可以通過adb devices 看到 ，但是不彈出連線時間的對話方塊

   需要檢視 develop options 中 Select usb Configuration 是否是在MTP模式上