Thursday, December 4, 2014

Completed AST Test Design Course

Most challenging course in terms of the content to be skimmed, understood and put in practice.
Add some travelling and festivals in between and the challenges multiply.

Thanks to all the participants, the instructors, my course sponsor Ilari Henrik Aegerter and my family members, I could complete the course on time.

Leia Mais…

Friday, September 5, 2014

Software Testing, Some myths and the Path forward...

This blog post is about a topic close to my heart - Software Testing
It is going to be a long post highlighting few key points/myths and what I think about them.

Feel free to comment and we can discuss. Ok, let's start.

Case 1: Testing process
Scenario 1: 
You are supposed to test a feature. You receive a requirement document and there is a formal review process. You write test cases based on your understanding of the feature. Add a layer of review process. Start testing executing the test cases. File bugs. Measure how many test cases are executed per day per tester. Write a test case for every bug discovered, if they are found outside the test cases. Reject any new feature addition as it was not planned in your scope. You assure the quality of the product and tell when the product is ready for release.

Scenario 2:
You are suppose to test a feature. You figure out who is the decision maker, the stakeholders involved and try to understand your role. You seek help, remove traps to get access to the product, interact with the programmers and prepare a light-weight document (checklist/mindmap or on a simple sheet of paper ) which acts as a reference. Based on your interactions with people and products, you update your document. You test the product, file bugs, ask questions, seek help/tips from experts and inform the stakeholders about the status of the product and project.

I will let you pick the scenario you like.
If you are more familiar or in favor of Scenario 1, we will have lots to discuss.

Case 2: Manual Testing vs Automation Testing
Have you heard of the word 'Sapience'? Do you use just your hands while testing? Do you think? What happens in your brain when you test? Think for a minute. Yes, THINK.
I feel that the very notion of classifying testing as manual vs automated is wrong. Michael and James have come up with an excellent blog post here highlighting the difference between testing and checking.
Here is the diagram from their post.

 Also, check out this blog post by Michael -

These two posts talk about Testing and Checking:

What is testing then?
Before reading more on what testing is, do check out the slides from BBST Foundations which describe what a computer program is. The same folks who would define a computer program as a set of instructions would define testing as a process to find bugs or assure quality of the product and testers as the gatekeepers of quality.

Check out an excellent video on the talk between devil and angel of software testing:

Testing is an intellectual activity. If you can think well, you can test well. Every test is a question to the product, sometimes to the project stakeholder. Check out the lessons #16 and #19

Testing is not about test cases or documentation or tools. These were designed to help you test better, support testing. Instead, what do we see testers focusing on?

Learning new tools, writing better user stories, automating tasks and dreaming of days when testing would be fully automated!!!

Pick any resume and check the Skills section: 7 out of 10 would mention some or the other tool names.
Quick Learning is a skill. Tool is a tool. A fool with a tool is still a fool (maybe, a dangerous fool).
So, what am I proposing?
Focus on skills like Observation, Thinking (Critical, Lateral, Forward/Backward, Parallel, Technical), Reasoning, Preparation, Programming basics, Collaboration, Framing, Reporting, Bug Hunting, Social Skills, Knowledge about Psychology, understand and apply oracles and heuristics, quick learning and many more skills. I suggest two books - Lessons Learned in Software Testing (Cem Kaner, James Bach, Bret Pettichord) and Testing Computer Software (Cem Kaner, Jack Falk, Hung Q Nguyen)

Practice your testing skills at Weekend Testing
Read the blogs everyday from this feed - Ministry of Testing
Watch the videos from BBST courses - Videos
Build your test ideas repository -
Test Heuristics Cheat Sheet,
YANDY list
300 common software errors
Pick your topic of interest and become a specialist in that. The Ministry of Testing resources page can help you get started. Resources

Case 3: Automation is the future
If you have been hearing of new tools, new languages every week and wondering which one to learn, here is a simple tip:
Your job is safe if you have the skills. There is no single tool which can replace a working human brain. If tools can replace humans, we would have been jobless long back. If you think automation is the future, I am sorry. You are missing a very key element here - EMOTIONS. Check out the ppt file here - Emotions And Test Oracles by Michael Bolton.

Google for TDD and check how many links highlight why it does not work.
If you think that testers need to learn coding, I am not against it completely. My only question is: How often have we asked programmers to learn testing?

If you think that TDD is the future, remind yourself again that they are good for programmers and do not replace/negate the role of a tester. Read the first comment on this post:

Here is the bumper post: FDA highlighting exploratory approach. 
And if you still think that tools can test mobile applications and there will be no need for testers, I will assume that you have no idea about this excellent keynote by Jonathan Kohl.

And I leave you with this final post: Testers: Get out of Quality Assurance Business by Michael Bolton. 

Leia Mais…

Saturday, July 26, 2014

Software Testing World Cup and our experience!!!

Hope most of you are aware of the awesome software testing competition conducted by Matt Heusser, Maik Nogens and many volunteers across the world. The whole concept of competing for one spot per continent is awesome. You might have participated in contests within your company or a local gathering but can it get bigger than a continent? Software Testing World Cup lets you experience that moment where you compete with close to 250 teams. Yes, 250 and only 1 winner.

Before I begin highlighting our team's experience, I want to thank the lead organizers - Matt Heusser and Maik Nogens and all the volunteer judges, the sponsors and everyone who helped conduct this BIG contest at the highest level. Check out the website for more details: Software Testing World Cup website

The logo is quite good too :)
For the Asia preliminaries, we had started preparing from April. We were a team of four - Pranav, Satish, Sundar and myself. I created a folder on Google Drive and shared it with other teams representing Fiberlink, an IBM company. Two teams were representing us from the America continent. We had few documents, list of test websites, tutorials on the G Drive. We also scheduled for 30 minute testing sessions till 07th June. Due to our busy schedule, we could manage only 6-7 focused sessions. We tested websites, mobile apps and windows applications too.

Meanwhile, we were brushing up our knowledge on different bugs, patterns, quick tests and checklists. We not only practiced the testing sessions but we also practiced writing the final test reports. We did send a sample test report to few judges for feedback and this exercise helped.

On the day of contest - 07th July, we were all set to test and only after an hour of testing, we realized that many other teams were not able to access the SUT and the contest was called off.
We were worried about the future date as one of us had to fly and we were not sure on how we would manage testing across time zones. The date was announced - 25th July and we had the following conflicting events:
- The day after a hectic office trip where I enjoyed a lot and was tired by end of day
- Pranav had to be part of interview drive for developers
- Protest in Bangalore
- Sundar had to pack for his trip that night

We did not have any practice sessions after the contest was postponed. We were monitoring the other continents' contests closely. We planned to assemble at 8.30 am IST. As Pranav was busy with the drive, we asked Rahul to join us. One interesting observation from my experience is that people find it tough to clear traps till someone else clears similar trap. I reached at 8.30 am and logged in to my Agile Test manager account only to find that the password was incorrect. On attempting Forgot password, it asked for Security Answers which I had no clue of. Immediately, I pinged Michael from HP, Maik and Smita asking them to help me with my login.

Michael solved it immediately and I was able to login to discover that it displayed Europe instance instead of Asia. After confirming that it was an issue on their side, I was waiting for the organizers to announce the SUT. Sundar and Rahul joined me in the meeting room and Satish was stuck in traffic because of the protest.
Sundar tried connecting to mIRC on both the laptops but it kept retrying and connection was timed out. Then, Sundar did something brilliant - connected to VPN and tried connecting via US network and we were placed in the chat room immediately :) Many teams from our company backed out as they were not able to connect to mIRC chat.

I use and found it great to start with. The only disadvantage I face with it is few important emails - like the one from STWC announcing the SUT get moved directly to the Unroll folder. We were informed a day before the contest that there will be two websites and one windows application to choose from. I was asking Sundar on which application to choose - Website or Windows application. He replied and I asked him why NBA without realizing that the SUT was already announced.

Immediately, we started our task and we listed out our focus areas for the next 30 minutes. Satish was testing from the cab and sharing files over Skype. We had the Skype group chat going where each one would highlight the bug and send the screenshot. My role was to replicate, file the bug and do some testing when the bug flow was manageable. Sundar was focusing on the different tours, I took up the quick tests with different checkers and Rahul was testing on iPad. Satish was testing on Android smartphone.

We found quite a decent number of critical bugs and in no time, there was just 45 minutes left.
We covered most of the areas planned for testing and I started consolidating the report. Sundar was filing the bugs discovered by Satish and Rahul. He was also helping me with the mind map for additional testing scenarios. At 12.31pm, we were done with testing and finalizing the test report. We emailed it to the mentioned email address and quickly received a confirmation from organizers that they received the report.

Key Highlights:
- Previous experience of working under severe time pressure helps.
- Getting our test reports reviewed by the judges pointed us to key areas we had overlooked.
- Practice sessions helped us finalize on a convention within the team.
- Do not panic when you face a trouble - think through and take steps to solve it.

That was one contest well planned and executed :)
Now, waiting for the results.

Leia Mais…

Saturday, March 15, 2014

Testing Strategy and Test Planning - My mindmap

I do not subscribe to any testing related blog feeds. Instead, I pay close attention to Joris Meets' twitter feed and check Ministry of Testing feed. Both of them do a great job of consolidating important testing related blogs/articles. While browsing through James post on Test Jumper, I liked what he wrote. It matched with my preferred style of working - help people identify traps, test with them and create an environment conducive to learning and improving testing skills.

There have been instances in my previous and the current organization where I am called upon to help a project which is going nowhere. Most of the times, they face one of the situations:

  • Important bugs being identified late
  • Testers on unplanned leave
  • Unstable product or need for better coverage
  • Inexperienced project team
  • Important project
I like such challenges. They seem to get the best out of me. I visualize this video and feel good at the additional responsibility given to me.

I started thinking of what gives me the confidence of taking up the role of a Test Jumper. Can I pass any checklist for someone to use and build on it? I launched XMind and here is the output. 
My Project Preparation
This is again a heuristic and not a final plan. It depends on the project, answers to the different questions and other factors like project, team, stakeholders, risks and deadline :) 

Leia Mais…

Monday, March 10, 2014

Testing Timeline: What is your %

Do you test software? Do you test every day? Do you get paid for it?
If you answered yes to any of the two questions, I have one more question for you :)
What percentage of office time do you actually test (interact with the software)?
The answer to 'Testing includes...' might differ from one person to another. I don't want to get into that discussion right now. I am more concerned when people spend very little time interacting with the product and complain of bugs being found by the customer. Consider the following three scenarios:

Scenario 1:
A feature has been revamped and will be released to market soon. A tester who has never worked on the feature is called to test the revamp. The tester could:
  • Understand the existing feature
  • Understand the revamp
  • Analyze the XYZ specification
  • Write test cases
  • Test the feature and file bugs
  • Test the system
Scenario 2:
Your team reaches office at 9am and is available till 6pm.
The total time spent testing the product on an average is 3 hrs. Rest of the time is spent on the following:
  • Attending meetings
  • Updating KnowledgeBase pages
  • Plan for next release
  • Taking interviews
  • Document the learning
Scenario 3:
Here is a tester who refuses to follow the rules. She never attends any meeting unless her inputs are a must-have. If she wants to learn anything, she knows the right person who can help her. She is very good at networking and has lots of friends in the whole company. She has one bad habit though, as her colleagues mention. No - not the one about rules, when she is given a feature to test, she spends most of the time interacting with the feature. Her activities can be summed up as follows: 
  • Understand the mission quickly
  • Highlight the traps and risks
  • Test the feature, system
  • Use information from different sources as a heuristic
  • Get help from those who can help her
What is your take on 'How much should you INTERACT with the feature/product/system?'
If you are smart enough, I would expect your answer to start with
'It Depends...' and continue the discussion. At the same time, I am open for new answers.

Leia Mais…

Thursday, January 9, 2014

Submitting a paper?

There are usually many paper submissions for a conference. How easy is it for your paper to be selected?
Check out the following tips. Hope they are of use to you - especially if this is your first time.

Abstract / Description
- Let it be as simple and crisp as possible.
- Do not copy paste or explain your company or context or a technology in detail unless it is a new idea. On some of the abstracts, I have seen a lot of details about some well known technology which could have been googled.
- Please get it reviewed by someone who has already presented a paper or presented in a conference.

Key Takeaways
This is one section which the audience is more interested in, specifically when they need to choose between sessions. Some not-so-good examples of Key Takeaways:
- XYZ Technology or Some terms in ABC Process
- X% improvement in some method

When submitting a paper, please test it.
- Will you as a reviewer, like it?
- Is it really necessary to give you a speaking slot?
- Is it something which needs you or can be found on Internet?

These are three questions which when answered might give you a clear picture of whether your paper is good for a conference submission.

Best wishes.

Leia Mais…