Test Coverage

Test Coverage Symphony :

           In my three years of experience in symphony , we have tracked automation coverage by comparing automated test cases against manual test cases. Whenever my manager asks the report, features based automated test count shared with him. The manager always informs us, Test coverage should be more than 80% percentage.

    To achieve 80%, we have worked , since two to three years and automated nearly 10,000 test cases, which runs for four days  in five to six VMS.

In over all, test coverage is not worked out to certify the quality of application. It never going to helpful to achieve bug free application, and still customer facing problem of production defects.

Test Coverage Huawei :

         Huawei is process oriented company and they will follow customized agile (scrum & XP) .Huawei needs all kinds of test reports(Test Plan, Test cases, Test summary, RCA, Retrospective) to certify quality. All these test reports are maintained based on the iteration and release.

Test coverage perspective , we need to keep track of Integration tests should be more than 90%, and functional Test  should not be less than 80%. Agile coach will audit all teams and give rating based on process and document. If functional & Integration tests are less than 80%, then coach will declare that team as low performing.

           Its mostly forcing us to maintain all reports , irrespective of quality of application. In over all , they will measure quality of application based on process and test reports. I can see , still Huawei is struggling to get customer satisfaction ,to ensure the quality.

Test coverage in Thought-works.

       Thoughtworks is agile oriented company and they will never believe on test reports. They will always choose , TDD or BDD type of development.

First write your test and write your code to pass the test, it will push all tests into lower level and keep minimal number of tests in higher level.

       This way of working is totally new to me , they never write any manual test cases . There is no way of tracking test execution results , developer will automate all possible scenarios in Unit level and Integration level.

     

       Responsibility of Tester , he will review all tests with respective to story and if there is any uncovered scenarios, then he will add into respective levels.  With respective to test coverage , they will track Integration and unit Test coverage , but never compare automated test cases into manual tests case.

      If clients are asking any kind of test reports , then they will prepare minimal report or either convince client and explain test approaches to client.

  If you have time , please read this post ,

 Martin flower :  http://martinfowler.com/bliki/TestCoverage.html

 Case study :     http://www.amanking.com/measuring-progress/

   

Read More

Bug free Application

A lot of time I had asked the same questions to me, I couldn’t find the right answers.  I have collected metrics from my current projects ,what they are thinking in terms of bug free software application.

 Since we had done two RCA on production defects in the current year  and listed the outcomes.

There is no check in micro service for negative stock.

Missed scenario during development and testing

Boundaries are not Tested based on cost status (S, G)

Data pull job is not tested 

Not tested against real data especially for dc discount

Don't have functional and unit test for events.

 There are best practices one can engage to reduce the probability of bugs from appearing.

      In real world , Is there any application with 100% bug free? No, its impossible to certify 100% bug free application but we can deliver workable software to customers. How does google ensure quality of application to end-users? 

     We can see related information : http://googletesting.blogspot.in/2011/01/how-google-tests-software.html

  

From My experience :

If we meet the following expectation of customer ,then we are delivering the bug free application 

       The client should be happy with delivery,and it should meet their expectation.

       End users should not have difficulty to use the application.

       Enough help or guide should be provided to end users.

       They should not ask , any queries to understand the functionality.

       

Read More

How to identify better automation tool for your project ?

In-general ,before identifying any automation tool,first ensure the requirements or automation scope of project. Which will helpful to identify right tool in shorter period of time. I have listed ,what are general parameters which need to be consider to identify tool..

Web Application Testing

• To check the browser support (IE, Mozilla , chrome)
• Version of browser is important during selection of tool
•  Is there any available to intimate the page loading?
• Can I wait to image or page to load ?
• can i check the availability of objects like its enabled , visible, disabled  or contains…?
• Can I extract data from web page like all or within methods

Framework design support  

• Page Object Model or Object Mapping
• Test/Error recovery
• Support for Graphical User interface (Flex, Silverlight)
• Operating system support (Linux, Mac, Windows)..
• Third party components usage
• Support (licensed or open source). If its open source then check community support.
• Test Report and Parallel execution of script.

Database testing

        All application provide facility to keep data outside itself . This can be achieved through by introducing databases (oracle, db2 , postgres). All databases will use common query SQL  and use ODBC drivers to communicate databases. We need to check any available tool to execute or manipulate datas like store procedures.

Data

     1. How can we separate data from scripts  ?

     2. can you generate data automatically ?

     3. How can we keep seed or data dump updated always ?

     4. How can we randomize the access to data ?

Read More

How to install postgres in mac

Install postgres in mac by using brew

• brew install postgres

Want to view info about postgres

•    brew info postgresql

After installing postgres , better restart your host which will automatically start postgres or you can use following commands to start postgres..

•     pg_ctl -D /usr/local/var/postgres start
•     pg_ctl -D /usr/local/var/postgres stop -m fast

Modify config file to set password , if you don’t want to set password then leave it as trust . 

• file : /usr/local/var/postgres/pg_hba.conf

• host    all             all             127.0.0.1/32            trust

Just logging with psql and check it 

•   psql -U postgres

If its thrown error as “Postgres user doesn’t exist” then clean up postgres data by using following commands 

•   Navigate to file directory :   cd /usr/local/var/
•   clean up everything : rm -rf *
•   initdb -U postgres  -D postgres

Read More

Set TestData By Using Active Record

1. Create gem file(under test directory ) and add active record , Postgres and database cleaner gems, it will helpful to maintain consistent version.

2. Create database.rb file under support directory

Read More

What is Maven ?

Build tool
Standard project structure
Java project tasks simpler

• mvm clean compile 
• mvm clean package
• man clean install
• mvn db-migration:reset -P test
• mvn db-migration:migrate -P test

Download latest version of maven :

http://maven.apache.org/download.cgi

configure maven in five minutes :
http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html

For video tutorial : (This video will be helpful for beginners who want to know about JDK, MAVEN and IDE)
http://javafortesters.blogspot.co.uk/p/install.html

Good comparison between Maven and Ant were explained in detail in below post .
  http://stackoverflow.com/questions/603189/differences-between-ant-and-maven

Read More

Capybara methods

https://github.com/jnicklas/capybara

Read More

How to setup capybara with cucumber in 5 mins?

•  create test directory  (mkdir  test)
•  \curl -L https://get.rvm.io | bash -s stable —ruby 
•  gem install cucumber
•  gem install capybara
•  mkdir features\support
•  mkdir features\step_definitions

Capybara settings 

Create env.rb file under support folder with following content

require 'capybara/cucumber'

Capybara.default_driver = :selenium

Cucumber scenarios

 Create test.feature file under features folder 

Feature: Find the automationhints blog

  Scenario: Search for the blog

    Given I am on the Google homepage

    Then I will search for "automationhints.blogspot"

    Then I should see "Vijay Ragavan"

Step definitions for cucumber steps

Create my_steps.rb file under step_definitions directory.

Given(/^I am on the Google homepage$/) do

  visit 'http://www.google.co.in'

end

Then(/^I will search for "([^"]*)"$/) do |arg|

 fill_in 'gbqfq', :with => arg + "\n"

end

Then(/^I should see "([^"]*)"$/) do |text|

  page.should have_content(text)

end

RUN

Navigate to test folder in console and type command as "cucumber". it will run all scenarios which is part of feature files.

Read More

Server side Monitoring

Long back taken session on server side monitoring in Thought works . How can monitor server performance while executing  performance test cases against web application . I have installed Graphite server in one of ubuntu vms and then configure graphite client on agent machines (application server, database , networks engine.)

Graphite

   Setup graphite server on ubuntu:

   https://gist.github.com/jgeurts/3112065  

  Starting Graphite :

  sudo ./carbon-cache.py start

  Diamond

  sudo service diamond start

 Collectors:

  https://github.com/BrightcoveOS/Diamond/wiki/Collectors

 To edit agent info : sudo nano  /etc /diamond/diamond.conf

Linux server monitoring Tools and benefits :

• Monit :  Process monitoring     
• Nagios : Alerting system 
• Graphite : Tracking and Trending
• Windows :  PerfMom

 Presentation :
https://drive.google.com/file/d/0B_5gBErIQ4aAVTBQbjBXVjJqZW8/edit?usp=sharing

Read More

How Quality Engineer can update himself on daily basis?

Following http://stackoverflow.com/, https://github.com/trending, google tech talk videos, Tech Radar and participating on active forum .

Reading blogs & magazine:

•    Google :  http://googletesting.blogspot.in/ 
•    James Bach  : http://www.satisfice.com/blog/
•    Martin Fowler : http://martinfowler.com/
•    Reading Hacker News : http://thehackernews.com/
•    Reading magazine : http://www.automatedtestinginstitute.com/home/

Read More

Need to find three integers which can equal Pythagoras' Theorem (a^2+ b^2= C^2) and a+b+c=1000

Have tried with some math logics to solve this problem .
 a^2+ b^2= C^2
 a+b+c=1000

Finally with help of computer solved this problem..

for a in 1..500
  for b in 1..500
    for c in 1..500
      if (a+b+c) == 1000
        if (a*a) +(b*b) == (c*c)
          p a,b,c
          exit
        end
      end
    end
  end
end

Results : 200 ,375 ,425

Read More

Ruby : How to list all methods of element or object?

 A way to list methods of list in ruby

> list=[1,2,3,4,5]      # define one list
 => [1, 2, 3, 4, 5]
 > list.sort
 => [1, 2, 3, 4, 5]
 > list.methods.sort   -1.methods      # -1 exclude other object methods.
 => [:[]=, :all?, :any?, :assoc, :at, :bsearch, :chunk, :clear, :collect, :collect!, :collect_concat, :combination, :compact, :compact!, :concat, :count, :cycle, :delete, :delete_at, :delete_if, :detect, :drop, :drop_while, :each, :each_cons, :each_entry, :each_index, :each_slice, :each_with_index, :each_with_object, :empty?, :entries, :fetch, :fill, :find, :find_all, :find_index, :first, :flat_map, :flatten, :flatten!, :grep, :group_by, :include?, :index, :inject, :insert, :join, :keep_if, :last, :lazy, :length, :map, :map!, :max, :max_by, :member?, :min, :min_by, :minmax, :minmax_by, :none?, :one?, :pack, :partition, :permutation, :pop, :product, :push, :rassoc, :reduce, :reject, :reject!, :repeated_combination, :repeated_permutation, :replace, :reverse, :reverse!, :reverse_each, :rindex, :rotate, :rotate!, :sample, :select, :select!, :shift, :shuffle, :shuffle!, :slice, :slice!, :slice_before, :sort, :sort!, :sort_by, :sort_by!, :take, :take_while, :to_a, :to_ary, :transpose, :uniq, :uniq!, :unshift, :values_at, :zip] 

Read More

Store procedure to simulate dummy data

BEGIN
  FOR i IN 100000..300000 LOOP
  insert into table_name (column1,2,....)
  values(2013,234,i,5951,'01-JAN-2013',20.2,0,20.2,'31-DEC-2013') ;
  COMMIT ;
  END LOOP;
  END;

Read More