Test Coverage Symphony :
In my three years of experience in symphony , we have tracked automation coverage by comparing automated test cases against manual test cases. Whenever my manager asks the report, features based automated test count shared with him. The manager always informs us, Test coverage should be more than 80% percentage.
To achieve 80%, we have worked , since two to three years and automated nearly 10,000 test cases, which runs for four days in five to six VMS.
In over all, test coverage is not worked out to certify the quality of application. It never going to helpful to achieve bug free application, and still customer facing problem of production defects.
Test Coverage Huawei :
Huawei is process oriented company and they will follow customized agile (scrum & XP) .Huawei needs all kinds of test reports(Test Plan, Test cases, Test summary, RCA, Retrospective) to certify quality. All these test reports are maintained based on the iteration and release.
Test coverage perspective , we need to keep track of Integration tests should be more than 90%, and functional Test should not be less than 80%. Agile coach will audit all teams and give rating based on process and document. If functional & Integration tests are less than 80%, then coach will declare that team as low performing.
Its mostly forcing us to maintain all reports , irrespective of quality of application. In over all , they will measure quality of application based on process and test reports. I can see , still Huawei is struggling to get customer satisfaction ,to ensure the quality.
Test coverage in Thought-works.
Thoughtworks is agile oriented company and they will never believe on test reports. They will always choose , TDD or BDD type of development.
First write your test and write your code to pass the test, it will push all tests into lower level and keep minimal number of tests in higher level.
This way of working is totally new to me , they never write any manual test cases . There is no way of tracking test execution results , developer will automate all possible scenarios in Unit level and Integration level.
Responsibility of Tester , he will review all tests with respective to story and if there is any uncovered scenarios, then he will add into respective levels. With respective to test coverage , they will track Integration and unit Test coverage , but never compare automated test cases into manual tests case.
If clients are asking any kind of test reports , then they will prepare minimal report or either convince client and explain test approaches to client.
If you have time , please read this post ,
Martin flower : http://martinfowler.com/bliki/TestCoverage.html
Case study : http://www.amanking.com/measuring-progress/
No comments:
Post a Comment