1. 程式人生 > >重要的軟體測試度量和度量指標(1)——附帶例子和圖表

重要的軟體測試度量和度量指標(1)——附帶例子和圖表

  在軟體專案中,最重要的就是測量質量,成本, 專案和流程的有效性。如果沒有測量,專案不能算真正的完成。 這篇文章,我們將會結合例子和圖表—學習軟體測試度量和測量指標以及怎樣在軟體測試流程中使用它們。
  這裡有一句名言:“我們無法測量我們不能控制的東西”。


這裡寫圖片描述

  其中, 可控制的專案指的是:一個專案管理者/領導從測試計劃中能發現偏差並且有效的採取行動。根據專案需要生成測量指標是實現令被測軟體達到要求的重要的指標。

什麼是軟體測量指標?

  軟體測量指標是指對一個擁有指定屬性的系統、 系統元件或過程處在何種程度的定量測量。

可以被定義為“測量的標準”

  軟體指標被用於測量專案的質量。簡單的說,指標是一個被用於描述一個屬性的單位,是一種測量的尺度。
  假設,在一般情況下,公斤是一個描述重量的屬性的指標,同樣的,在軟體工程中,在成千行的程式碼中有多少問題被發現?這裡問題的數量是一個測量,而程式碼行數是另一個測量。 指標就是由這兩個測量定義的。

  測試指標舉例:

  • 模組中有多少缺陷存在?
  • 人均執行多少測試用例?
  • 測試覆蓋率是多少?
  • 軟體測試測量的是什麼?

  測量是產品或過程的一些屬性的量化指示,如程度、數量、尺寸、容量,規模等。

  測試度量舉例: 缺陷的總數。
  請參考下圖來清楚的瞭解測量與指標的區別。


這裡寫圖片描述

為什麼要有測試指標?

  制定軟體測試指標是軟體測試經理最重要的責任。   測試指標往往能決定下一階段的活動,比如:預估未來專案的成本和進度。
  1. 瞭解促進專案的成功需要何種改進。
  2. 對待改善的過程或是技術做出決策。

軟體測試指標的重要性:

  正如前面所說的,測試指標是測量軟體質量最重要的因素。 現在,我們如何通過指標來測量軟體的質量呢?   假設,如果一個專案沒有一個指標,測試分析師所作的工作的質量要怎麼被定量?   例如:一個測試分析師需要做:
  1. 為 5 個需求設計測試用例
  2. 執行設計的測試用例
  3. 記錄缺陷以及需求和執行失敗的相關用例
  4. 缺陷解決後, 重新驗證缺陷,再次執行相應的失敗的用例
  在以上方案中,如果指標沒有被採用,那測試分析師完成的工作量將不客觀, 並且測試報告無法通過適當資訊反映出他的工作/專案的狀態。   如果專案採用指標, 則通過適當的資料可以確切的反映出他的工作狀態。 **即在測試報告中,我們可以確定(打印出):**
  1. 每個需求設計多少個測試用例?
  2. 有多少測試用例沒有設計(設計不完善)?
  3. 有多少測試用例被執行?
  4. 多少用例通過/失敗/中斷?
  5. 多少測試用例沒有被執行?
  6. 有多少缺陷被發現,這些缺陷的嚴重程度是什麼?
  7. 有多少失敗的測試用例是因為同一個缺陷等。
  以專案需求為基礎我們可以有更多的指標來詳細的瞭解專案的進度而不僅僅是以上所述。   **基於以上的指標,測試經理將會理解以下所提到的要點:**   a) 完成工作的百分比   b) 有待完成的工作的百分比   c) 完成剩餘工作所需的時間   d) 專案是否按照計劃的時間進行或者滯後等。   基於指標,如果專案沒有按照計劃的時間完成, 經理則需要向客戶或是其他股東提出警示,解釋滯後的原因,避免最後為時已晚的意外。

指標的生命週期:

➢ 分析:
  • 識別指標
  • 定義指標
➢ 溝通:
  • 給利益相關者以及測試團隊解釋指標的需求
  • 培訓測試團隊在處理測試指標時需要獲取的關鍵資料點
➢ 評估:
  • 獲取、驗證資料
  • 使用獲取的資料計算指標值
➢ 報告:
  • 報告中衍生出有效結論
  • 將報告發送給給利益相關者和各位代表
  • 記錄利益相關者的反饋

手動測試指標的型別:

  測試指標主要分為兩個類別: 計算指標和基礎指標。   **基礎指標**是由測試分析師在測試用例編寫和執行期間收集的資料派生來的指標。   這個資料將貫穿整個測試周期。 收集的資料譬如為, 某一專案所開發的測試案例的總數、 需要被執行的 測試用例的數量或是通過/失敗/受阻的測試用例計數。   **計算指標**是通過收集基礎指標資料而派生的。這些指標通常由測試經歷為測試報告等目的而進行追蹤。

軟體測試指標的例子:

這裡來舉一個例子來計算各種各樣被用於軟體測試報告中的指標:

以下表格中的資料來自真正參與到測試中的測試分析師:

序號 測試指標 個數
1 需求個數 5
2 平均每個需求所寫的測試用例個數 20
3 總的測試用例個數 100
4 被執行的用例的個數 65
5 通過的用例個數 30
6 失敗的用例個數 26
7 中斷的用例的個數 9
8 沒有被執行的用例個數 35
9 確定的缺陷數 30
10 嚴重的缺陷數 6
11 優先順序較高的缺陷數 10
12 一般的缺陷數 6
13 輕微的缺陷數 8

原文如下:

In software projects, it is most important to measure the quality, cost and effectiveness of the project and the processes. Without measuring these, project can’t be completed successfully.

In today’s article we will learn with examples and graphs – Software test metrics and measurementsand how to use these in software testing process.

There is a famous statement: “We can’t control things which we can’t measure”.


這裡寫圖片描述

Here controlling the projects means, how a project manager/lead can identify the deviations from the test plan ASAP in order to react in the perfect time. Generation of test metrics based on the project needs is very much important to achieve the quality of the software being tested.

What are Software Testing Metrics?

A Metric is a quantitative measure of the degree to which a system, system component, or process possesses a given attribute.

Metrics can be defined as “STANDARDS OF MEASUREMENT”.

Software Metrics are used to measure the quality of the project. Simply, Metric is a unit used for describing an attribute. Metric is a scale for measurement.
Suppose, in general, “Kilogram” is a metric for measuring the attribute “Weight”. Similarly, in software, “How many issues are found in thousand lines of code?”, here No. of issues is one measurement & No. of lines of code is another measurement. Metric is defined from these two measurements.

Test metrics example:

  • How many defects are existed within the module?
  • How many test cases are executed per person?
  • What is the Test coverage?
  • What is Software Test Measurement?

Measurement is the quantitative indication of extent, amount, dimension, capacity, or size of some attribute of a product or process.

Test measurement example: Total number of defects.
Please refer below diagram for clear understanding of the difference between Measurement & Metrics.


這裡寫圖片描述

Why Test Metrics?

Generation of Software Test Metrics is the most important responsibility of the Software Test Lead/Manager. Test Metrics are used to take the decision for next phase of activities such as, estimate the cost & schedule of future projects.
  1. Understand the kind of improvement required to success the project
  2. Take decision on process or technology to be modified etc.

Importance of Software Testing Metrics:

As explained above, Test Metrics are the most important to measure the quality of the software. Now, how can we measure the quality of the software by using Metrics? Suppose, if a project does not have any metrics, then how the quality of the work done by a Test analyst will be measured? **For Example:** A Test Analyst has to,
  1. Design the test cases for 5 requirements
  2. Execute the designed test cases
  3. Log the defects & need to fail the related test cases
  4. After the defect is resolved, need to re-test the defect & re-execute the corresponding failed test case.
In above scenario, if metrics are not followed, then the work completed by the test analyst will be subjective i.e. the test report will not have the proper information to know the status of his work/project. If Metrics are involved in the project, then the exact status of his/her work with proper numbers/data can be published. **I.e. in the Test report, we can publish:**
  1. How many test cases have been designed per requirement?
  2. How many test cases are yet to design?
  3. How many test cases are executed?
  4. How many test cases are passed/failed/blocked?
  5. How many test cases are not yet executed?
  6. How many defects are identified & what is the severity of those defects?
  7. How many test cases are failed due to one particular defect? etc.
**Based on the project needs we can have more metrics than above mentioned list, to know the status of the project in detail.** Based on the above metrics, test lead/manager will get the understanding of the below mentioned key points. a) %ge of work completed b) %ge of work yet to be completed c) Time to complete the remaining work d) Whether the project is going as per the schedule or lagging? etc. Based on the metrics, if the project is not going to complete as per the schedule, then the manager will raise the alarm to the client and other stake holders by providing the reasons for lagging to avoid the last minute surprises.

Metrics Life Cycle:

➢ Analsis
  • Identifiation of the Metrics
  • Define the identified Metrics
➢ Communicate
  • Explain the need of metric to stakeholder and testing tean
  • Educate the testing team about the data points need to be captured for processing the metric
➢ Evaluation
  • Capture & verify the data
  • Calculating the metric(s) value using the data capyured
➢ Report
  • Develop the report with effective conclsion
  • Didtribute report to the stakeholder and respective representative
  • Take feedback from stakeholder

Types of Manual Test Metrics:

Testing Metrics are mainly divided into 2 categories. Base Metrics Calculated Metrics

Base Metrics:

Base Metrics are the Metrics which are derived from the data gathered by the Test Analyst during the test case development and execution. This data will be tracked throughout the Test Life cycle. I.e. collecting the data like, Total no. of test cases developed for a project (or) no. of test cases need to be executed (or) no. of test cases passed/failed/blocked etc.

Calculated Metrics:

Calculated Metrics are derived from the data gathered in Base Metrics. These Metrics are generally tracked by the test lead/manager for Test Reporting purpose.

Examples of Software Testing Metrics:

Let’s take an example to calculate various test metrics used in software test reports: Below is the table format for the data retrieved from the test analyst who is actually involved in testing:
S.NO Testng Metric Data retrieved during test case development&execution
1 NO.of Requirements 5
2 Avg.NO.of Test cases written per Requirement 20
3 Total no.of Test cases written for all requirements 100
4 Total no.of Test cases Executed 65
5 No.of Test cases Passed 30
6 No.of Test cases Failed 26
7 No.of Test cases Blocked 9
8 No.of Test cases unexecuted 35
9 No.of Test Defects identified 30
10 Critical Defects count 6
11 High Defects count 10
12 Medium Defects count 6
13 Low Defects count 8