重要的軟體測試度量和度量指標(1)——附帶例子和圖表
在軟體專案中,最重要的就是測量質量,成本, 專案和流程的有效性。如果沒有測量,專案不能算真正的完成。 這篇文章,我們將會結合例子和圖表—學習軟體測試度量和測量指標以及怎樣在軟體測試流程中使用它們。
這裡有一句名言:“我們無法測量我們不能控制的東西”。
其中, 可控制的專案指的是:一個專案管理者/領導從測試計劃中能發現偏差並且有效的採取行動。根據專案需要生成測量指標是實現令被測軟體達到要求的重要的指標。
什麼是軟體測量指標?
軟體測量指標是指對一個擁有指定屬性的系統、 系統元件或過程處在何種程度的定量測量。可以被定義為“測量的標準”
軟體指標被用於測量專案的質量。簡單的說,指標是一個被用於描述一個屬性的單位,是一種測量的尺度。
假設,在一般情況下,公斤是一個描述重量的屬性的指標,同樣的,在軟體工程中,在成千行的程式碼中有多少問題被發現?這裡問題的數量是一個測量,而程式碼行數是另一個測量。 指標就是由這兩個測量定義的。
測試指標舉例:
- 模組中有多少缺陷存在?
- 人均執行多少測試用例?
- 測試覆蓋率是多少?
- 軟體測試測量的是什麼?
測量是產品或過程的一些屬性的量化指示,如程度、數量、尺寸、容量,規模等。
測試度量舉例: 缺陷的總數。
請參考下圖來清楚的瞭解測量與指標的區別。
為什麼要有測試指標?
制定軟體測試指標是軟體測試經理最重要的責任。 測試指標往往能決定下一階段的活動,比如:預估未來專案的成本和進度。- 瞭解促進專案的成功需要何種改進。
- 對待改善的過程或是技術做出決策。
軟體測試指標的重要性:
正如前面所說的,測試指標是測量軟體質量最重要的因素。 現在,我們如何通過指標來測量軟體的質量呢? 假設,如果一個專案沒有一個指標,測試分析師所作的工作的質量要怎麼被定量? 例如:一個測試分析師需要做:- 為 5 個需求設計測試用例
- 執行設計的測試用例
- 記錄缺陷以及需求和執行失敗的相關用例
- 缺陷解決後, 重新驗證缺陷,再次執行相應的失敗的用例
- 每個需求設計多少個測試用例?
- 有多少測試用例沒有設計(設計不完善)?
- 有多少測試用例被執行?
- 多少用例通過/失敗/中斷?
- 多少測試用例沒有被執行?
- 有多少缺陷被發現,這些缺陷的嚴重程度是什麼?
- 有多少失敗的測試用例是因為同一個缺陷等。
指標的生命週期:
➢ 分析:- 識別指標
- 定義指標
- 給利益相關者以及測試團隊解釋指標的需求
- 培訓測試團隊在處理測試指標時需要獲取的關鍵資料點
- 獲取、驗證資料
- 使用獲取的資料計算指標值
- 報告中衍生出有效結論
- 將報告發送給給利益相關者和各位代表
- 記錄利益相關者的反饋
手動測試指標的型別:
測試指標主要分為兩個類別: 計算指標和基礎指標。 **基礎指標**是由測試分析師在測試用例編寫和執行期間收集的資料派生來的指標。 這個資料將貫穿整個測試周期。 收集的資料譬如為, 某一專案所開發的測試案例的總數、 需要被執行的 測試用例的數量或是通過/失敗/受阻的測試用例計數。 **計算指標**是通過收集基礎指標資料而派生的。這些指標通常由測試經歷為測試報告等目的而進行追蹤。軟體測試指標的例子:
這裡來舉一個例子來計算各種各樣被用於軟體測試報告中的指標:
以下表格中的資料來自真正參與到測試中的測試分析師:
序號 | 測試指標 | 個數 |
---|---|---|
1 | 需求個數 | 5 |
2 | 平均每個需求所寫的測試用例個數 | 20 |
3 | 總的測試用例個數 | 100 |
4 | 被執行的用例的個數 | 65 |
5 | 通過的用例個數 | 30 |
6 | 失敗的用例個數 | 26 |
7 | 中斷的用例的個數 | 9 |
8 | 沒有被執行的用例個數 | 35 |
9 | 確定的缺陷數 | 30 |
10 | 嚴重的缺陷數 | 6 |
11 | 優先順序較高的缺陷數 | 10 |
12 | 一般的缺陷數 | 6 |
13 | 輕微的缺陷數 | 8 |
原文如下:
In software projects, it is most important to measure the quality, cost and effectiveness of the project and the processes. Without measuring these, project can’t be completed successfully.
In today’s article we will learn with examples and graphs – Software test metrics and measurementsand how to use these in software testing process.
There is a famous statement: “We can’t control things which we can’t measure”.
Here controlling the projects means, how a project manager/lead can identify the deviations from the test plan ASAP in order to react in the perfect time. Generation of test metrics based on the project needs is very much important to achieve the quality of the software being tested.
What are Software Testing Metrics?
A Metric is a quantitative measure of the degree to which a system, system component, or process possesses a given attribute.Metrics can be defined as “STANDARDS OF MEASUREMENT”.
Software Metrics are used to measure the quality of the project. Simply, Metric is a unit used for describing an attribute. Metric is a scale for measurement.
Suppose, in general, “Kilogram” is a metric for measuring the attribute “Weight”. Similarly, in software, “How many issues are found in thousand lines of code?”, here No. of issues is one measurement & No. of lines of code is another measurement. Metric is defined from these two measurements.
Test metrics example:
- How many defects are existed within the module?
- How many test cases are executed per person?
- What is the Test coverage?
- What is Software Test Measurement?
Measurement is the quantitative indication of extent, amount, dimension, capacity, or size of some attribute of a product or process.
Test measurement example: Total number of defects.
Please refer below diagram for clear understanding of the difference between Measurement & Metrics.
Why Test Metrics?
Generation of Software Test Metrics is the most important responsibility of the Software Test Lead/Manager. Test Metrics are used to take the decision for next phase of activities such as, estimate the cost & schedule of future projects.- Understand the kind of improvement required to success the project
- Take decision on process or technology to be modified etc.
Importance of Software Testing Metrics:
As explained above, Test Metrics are the most important to measure the quality of the software. Now, how can we measure the quality of the software by using Metrics? Suppose, if a project does not have any metrics, then how the quality of the work done by a Test analyst will be measured? **For Example:** A Test Analyst has to,- Design the test cases for 5 requirements
- Execute the designed test cases
- Log the defects & need to fail the related test cases
- After the defect is resolved, need to re-test the defect & re-execute the corresponding failed test case.
- How many test cases have been designed per requirement?
- How many test cases are yet to design?
- How many test cases are executed?
- How many test cases are passed/failed/blocked?
- How many test cases are not yet executed?
- How many defects are identified & what is the severity of those defects?
- How many test cases are failed due to one particular defect? etc.
Metrics Life Cycle:
➢ Analsis- Identifiation of the Metrics
- Define the identified Metrics
- Explain the need of metric to stakeholder and testing tean
- Educate the testing team about the data points need to be captured for processing the metric
- Capture & verify the data
- Calculating the metric(s) value using the data capyured
- Develop the report with effective conclsion
- Didtribute report to the stakeholder and respective representative
- Take feedback from stakeholder
Types of Manual Test Metrics:
Testing Metrics are mainly divided into 2 categories. Base Metrics Calculated MetricsBase Metrics:
Base Metrics are the Metrics which are derived from the data gathered by the Test Analyst during the test case development and execution. This data will be tracked throughout the Test Life cycle. I.e. collecting the data like, Total no. of test cases developed for a project (or) no. of test cases need to be executed (or) no. of test cases passed/failed/blocked etc.Calculated Metrics:
Calculated Metrics are derived from the data gathered in Base Metrics. These Metrics are generally tracked by the test lead/manager for Test Reporting purpose.Examples of Software Testing Metrics:
Let’s take an example to calculate various test metrics used in software test reports: Below is the table format for the data retrieved from the test analyst who is actually involved in testing:S.NO | Testng Metric | Data retrieved during test case development&execution |
---|---|---|
1 | NO.of Requirements | 5 |
2 | Avg.NO.of Test cases written per Requirement | 20 |
3 | Total no.of Test cases written for all requirements | 100 |
4 | Total no.of Test cases Executed | 65 |
5 | No.of Test cases Passed | 30 |
6 | No.of Test cases Failed | 26 |
7 | No.of Test cases Blocked | 9 |
8 | No.of Test cases unexecuted | 35 |
9 | No.of Test Defects identified | 30 |
10 | Critical Defects count | 6 |
11 | High Defects count | 10 |
12 | Medium Defects count | 6 |
13 | Low Defects count | 8 |