Test Coverage

Test Coverage Symphony :

           In my three years of experience in symphony , we have tracked automation coverage by comparing automated test cases against manual test cases. Whenever my manager asks the report, features based automated test count shared with him. The manager always informs us, Test coverage should be more than 80% percentage.

    To achieve 80%, we have worked , since two to three years and automated nearly 10,000 test cases, which runs for four days  in five to six VMS.

In over all, test coverage is not worked out to certify the quality of application. It never going to helpful to achieve bug free application, and still customer facing problem of production defects.

Test Coverage Huawei :

         Huawei is process oriented company and they will follow customized agile (scrum & XP) .Huawei needs all kinds of test reports(Test Plan, Test cases, Test summary, RCA, Retrospective) to certify quality. All these test reports are maintained based on the iteration and release.

Test coverage perspective , we need to keep track of Integration tests should be more than 90%, and functional Test  should not be less than 80%. Agile coach will audit all teams and give rating based on process and document. If functional & Integration tests are less than 80%, then coach will declare that team as low performing.

           Its mostly forcing us to maintain all reports , irrespective of quality of application. In over all , they will measure quality of application based on process and test reports. I can see , still Huawei is struggling to get customer satisfaction ,to ensure the quality.

Test coverage in Thought-works.

       Thoughtworks is agile oriented company and they will never believe on test reports. They will always choose , TDD or BDD type of development.

First write your test and write your code to pass the test, it will push all tests into lower level and keep minimal number of tests in higher level.

       This way of working is totally new to me , they never write any manual test cases . There is no way of tracking test execution results , developer will automate all possible scenarios in Unit level and Integration level.

     

       Responsibility of Tester , he will review all tests with respective to story and if there is any uncovered scenarios, then he will add into respective levels.  With respective to test coverage , they will track Integration and unit Test coverage , but never compare automated test cases into manual tests case.

      If clients are asking any kind of test reports , then they will prepare minimal report or either convince client and explain test approaches to client.

  If you have time , please read this post ,

 Martin flower :  http://martinfowler.com/bliki/TestCoverage.html

 Case study :     http://www.amanking.com/measuring-progress/

   

Read More

Bug free Application

A lot of time I had asked the same questions to me, I couldn’t find the right answers.  I have collected metrics from my current projects ,what they are thinking in terms of bug free software application.

 Since we had done two RCA on production defects in the current year  and listed the outcomes.

There is no check in micro service for negative stock.

Missed scenario during development and testing

Boundaries are not Tested based on cost status (S, G)

Data pull job is not tested 

Not tested against real data especially for dc discount

Don't have functional and unit test for events.

 There are best practices one can engage to reduce the probability of bugs from appearing.

      In real world , Is there any application with 100% bug free? No, its impossible to certify 100% bug free application but we can deliver workable software to customers. How does google ensure quality of application to end-users? 

     We can see related information : http://googletesting.blogspot.in/2011/01/how-google-tests-software.html

  

From My experience :

If we meet the following expectation of customer ,then we are delivering the bug free application 

       The client should be happy with delivery,and it should meet their expectation.

       End users should not have difficulty to use the application.

       Enough help or guide should be provided to end users.

       They should not ask , any queries to understand the functionality.

       

Read More

How to identify better automation tool for your project ?

In-general ,before identifying any automation tool,first ensure the requirements or automation scope of project. Which will helpful to identify right tool in shorter period of time. I have listed ,what are general parameters which need to be consider to identify tool..

Web Application Testing

• To check the browser support (IE, Mozilla , chrome)
• Version of browser is important during selection of tool
•  Is there any available to intimate the page loading?
• Can I wait to image or page to load ?
• can i check the availability of objects like its enabled , visible, disabled  or contains…?
• Can I extract data from web page like all or within methods

Framework design support  

• Page Object Model or Object Mapping
• Test/Error recovery
• Support for Graphical User interface (Flex, Silverlight)
• Operating system support (Linux, Mac, Windows)..
• Third party components usage
• Support (licensed or open source). If its open source then check community support.
• Test Report and Parallel execution of script.

Database testing

        All application provide facility to keep data outside itself . This can be achieved through by introducing databases (oracle, db2 , postgres). All databases will use common query SQL  and use ODBC drivers to communicate databases. We need to check any available tool to execute or manipulate datas like store procedures.

Data

     1. How can we separate data from scripts  ?

     2. can you generate data automatically ?

     3. How can we keep seed or data dump updated always ?

     4. How can we randomize the access to data ?

Read More