最近開發了總有同事抱怨UiAutomator有些介面的空間無法識別，以至於部分功能自動化指令碼開發被Block，對此我研究了騰訊之前釋出的一個adbui庫，這個庫是python版的，其中有一個通過ocr的方式來解決UIA開發過程中控制元件不能識別的問題，於是就想把這個功能移植到UIA的公共庫裡面。

【注意事項】

1.  該介面需要傳送http請求和騰訊的雲伺服器進行通訊，因此，使用該介面時，需保證資料流量或者Wi-Fi連線。

2.   該介面容易受到網速等不可逆因素影響，導致響應速度變慢，所以不適合大規模使用。

3.   該介面如果使用資料流量，會產生一定程度上流量費用。

長話短說，上程式碼.

首先生成簽名工具類:

public class TencentSign {
    /**
     * 生成 Authorization 簽名欄位
     *
     * @param appId
     * @param secretId
     * @param secretKey
     * @param expired
     * @return
     * @throws Exception
     */
    public static String appSign(long appId, String secretId, String secretKey,
                                 long expired) throws Exception {
        long now = System.currentTimeMillis() / 1000;
        int rdm = Math.abs(new Random().nextInt());
        String plainText = String.format("a=%d&k=%s&e=%d&t=%d&r=%d&u=%s&f=", appId,
                secretId, expired, now, rdm, "xx");
        byte[] hmacDigest = HmacSha1(plainText, secretKey);
        byte[] signContent = new byte[hmacDigest.length + plainText.getBytes().length];
        System.arraycopy(hmacDigest, 0, signContent, 0, hmacDigest.length);
        System.arraycopy(plainText.getBytes(), 0, signContent, hmacDigest.length,
                plainText.getBytes().length);
        return Base64Encode(signContent);
    }

    /**
     * 生成 base64 編碼
     *
     * @param binaryData
     * @return
     */
    public static String Base64Encode(byte[] binaryData) {
        return Base64.encodeToString(binaryData, Base64.NO_WRAP);
    }

    /**
     * 生成 hmacsha1 簽名
     *
     * @param binaryData
     * @param key
     * @return
     * @throws Exception
     */
    public static byte[] HmacSha1(byte[] binaryData, String  key) throws Exception {
        Mac mac = Mac.getInstance("HmacSHA1");
        SecretKeySpec secretKey = new SecretKeySpec(key.getBytes(), "HmacSHA1");
        mac.init(secretKey);
        byte[] HmacSha1Digest = mac.doFinal(binaryData);
        return HmacSha1Digest;
    }

    /**
     * 生成 hmacsha1 簽名
     *
     * @param plainText
     * @param key
     * @return
     * @throws Exception
     */
    public static byte[] HmacSha1(String plainText, String key) throws Exception {
        return HmacSha1(plainText.getBytes(), key);
    }
}
 

Http傳送請求，根據adbui python庫中定義的header以及data的格式。

public class HttpUtils {
   
    /**
     * post方式請求伺服器(https協議)
     *
     * @param url
     *            請求地址
     * @param content
     *            引數
     * @return
     * @throws NoSuchAlgorithmException
     * @throws KeyManagementException
     * @throws IOException
     */
    public static String post(String url, String sign, String content)
            throws NoSuchAlgorithmException, KeyManagementException, IOException {
        URL console = new URL(url);
        HttpURLConnection conn = (HttpURLConnection) console.openConnection();
        conn.setDoInput(true);
        conn.setDoOutput(true);
        conn.setRequestMethod("POST");
        conn.setUseCaches(false);
        conn.setInstanceFollowRedirects(true);
        //根據adbui python庫填寫的Header引數
        conn.addRequestProperty("Content-Type", "text/json");
        conn.setRequestProperty("Authorization", sign);
        conn.connect();
        DataOutputStream out = new DataOutputStream(conn.getOutputStream());
        out.write(content.getBytes("UTF-8"));
        out.flush();
        out.close();
        if (conn.getResponseCode() != HttpURLConnection.HTTP_OK)
            throw new RuntimeException("Request url failed ...");
        BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(),"UTF-8"));
        String result = "";
        String getLine;
        while ((getLine = in.readLine()) != null)
            result += getLine;
        in.close();
        return result;
    }

}
 

獲取UI控制元件位置的的實現，實現過程中在base64編碼那塊入坑多次

public class TencentOCR {


    public static List getUisByOcr(String text) throws Exception {
        Common.takeScreenshot("ocrshot");
        StringBuffer path = new StringBuffer(Environment.getExternalStorageDirectory().getPath());
        path.append("/screenshot/ocrshot.png");
        String result = getTextFromImage(path.toString());
        JSONObject ocrResult = new JSONObject(result);
        List<UI> uis = new ArrayList();
        JSONArray items = ocrResult.getJSONArray("items");
        for (int i = 0; i < items.length(); i++) {
            int sameCount = 0;
            JSONObject item = items.getJSONObject(i);
            String itemString = item.getString("itemstring");
            if (itemString.contains(text)) {
//                print("itemString-->" + itemString);
                JSONObject itemcoord = item.getJSONObject("itemcoord");
                UI ui = new UI(itemcoord.getInt("x"), itemcoord.getInt("y"),
                        itemcoord.getInt("x") + itemcoord.getInt("width"),
                        itemcoord.getInt("y") + itemcoord.getInt("height"),
                        itemcoord.getInt("width"), itemcoord.getInt("height"));
                uis.add(ui);
            }
        }
        return uis;
    }
    /**
     * 查詢指定text的UI位置
     *
     * @param text 需要查詢的text
     * @return 返回UI位置
     */
    public static UI getUiByOcr(String text) throws Exception {
        List<UI> uilist = getUisByOcr(text);
        if(!uilist.isEmpty())
            return uilist.get(0);
        return null;
    }

    public static UI getUiByOcr(String text, int index) throws Exception {
        List<UI> uilist = getUisByOcr(text);
        if(!uilist.isEmpty())
            return uilist.get(index);
        return null;
    }

    /**
     * 識別圖中文字資訊
     *
     * @param imagePath 圖片路徑
     * @return 返回ocr識別結果
     */
    public static String getTextFromImage(String imagePath) throws Exception {
        if (TextUtils.isEmpty(imagePath))
            return null;
        long expired = System.currentTimeMillis() / 1000 + 2592000;
        //得到Authorization
        String sign = TencentSign.appSign(YouTuHttpContants.APP_ID,
                YouTuHttpContants.SECRET_ID,
                YouTuHttpContants.SECRET_KEY,
                expired);
        String image = image2Base64(imagePath);
        JSONObject jsonObject = new JSONObject();
        jsonObject.put("app_id", String.valueOf(YouTuHttpContants.APP_ID));
        jsonObject.put("session_id", "");
        jsonObject.put("image", image);
        String result = HttpUtils.post(YouTuHttpContants.OCR_URL, sign,
                jsonObject.toString());
        return result;
    }

    /**
     * 將圖中進行base64格式編碼
     *
     * @param imagePath 圖片路徑
     * @return
     */
    public static String image2Base64(String imagePath) {

        InputStream is = null;
        byte[] data = null;
        String result = null;
        try {
            is = new FileInputStream(imagePath);
            data = new byte[is.available()];
            is.read(data);
            result = Base64.encodeToString(data, Base64.DEFAULT);
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (is != null) {
                try {
                    is.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
        return result;
    }

    public static class UI {

        private int x1;
        private int y1;
        private int x2;
        private int y2;
        private int width;
        private int height;
        private String text;

        public UI(int x1, int y1, int x2, int y2, int width, int height) {
            this.x1 = x1;
            this.y1 = y1;
            this.x2 = x2;
            this.y2 = y2;
            this.width = width;
            this.height = height;
            this.text = null;
        }

        public void setText(String text) {
            this.text = text;
        }

        public String getText() {
            return text;
        }

        public void click() {
            int x = x1 + width / 2;
            int y = y1 + height / 2;
            mDevice.click(x, y);
        }
    }
}