Software Testing World Cup and our experience!!!

Hope most of you are aware of the awesome software testing competition conducted by Matt Heusser, Maik Nogens and many volunteers across the world. The whole concept of competing for one spot per continent is awesome. You might have participated in contests within your company or a local gathering but can it get bigger than a continent? Software Testing World Cup lets you experience that moment where you compete with close to 250 teams. Yes, 250 and only 1 winner.

Before I begin highlighting our team's experience, I want to thank the lead organizers - Matt Heusser and Maik Nogens and all the volunteer judges, the sponsors and everyone who helped conduct this BIG contest at the highest level. Check out the website for more details: Software Testing World Cup website

The logo is quite good too :)

For the Asia preliminaries, we had started preparing from April. We were a team of four - Pranav, Satish, Sundar and myself. I created a folder on Google Drive and shared it with other teams representing Fiberlink, an IBM company. Two teams were representing us from the America continent. We had few documents, list of test websites, tutorials on the G Drive. We also scheduled for 30 minute testing sessions till 07th June. Due to our busy schedule, we could manage only 6-7 focused sessions. We tested websites, mobile apps and windows applications too.

Meanwhile, we were brushing up our knowledge on different bugs, patterns, quick tests and checklists. We not only practiced the testing sessions but we also practiced writing the final test reports. We did send a sample test report to few judges for feedback and this exercise helped.

On the day of contest - 07th July, we were all set to test and only after an hour of testing, we realized that many other teams were not able to access the SUT and the contest was called off.

We were worried about the future date as one of us had to fly and we were not sure on how we would manage testing across time zones. The date was announced - 25th July and we had the following conflicting events:

- The day after a hectic office trip where I enjoyed a lot and was tired by end of day

- Pranav had to be part of interview drive for developers

- Protest in Bangalore

- Sundar had to pack for his trip that night

We did not have any practice sessions after the contest was postponed. We were monitoring the other continents' contests closely. We planned to assemble at 8.30 am IST. As Pranav was busy with the drive, we asked Rahul to join us. One interesting observation from my experience is that people find it tough to clear traps till someone else clears similar trap. I reached at 8.30 am and logged in to my Agile Test manager account only to find that the password was incorrect. On attempting Forgot password, it asked for Security Answers which I had no clue of. Immediately, I pinged Michael from HP, Maik and Smita asking them to help me with my login.

Michael solved it immediately and I was able to login to discover that it displayed Europe instance instead of Asia. After confirming that it was an issue on their side, I was waiting for the organizers to announce the SUT. Sundar and Rahul joined me in the meeting room and Satish was stuck in traffic because of the protest.

Sundar tried connecting to mIRC on both the laptops but it kept retrying and connection was timed out. Then, Sundar did something brilliant - connected to VPN and tried connecting via US network and we were placed in the chat room immediately :) Many teams from our company backed out as they were not able to connect to mIRC chat.

I use Unroll.me and found it great to start with. The only disadvantage I face with it is few important emails - like the one from STWC announcing the SUT get moved directly to the Unroll folder. We were informed a day before the contest that there will be two websites and one windows application to choose from. I was asking Sundar on which application to choose - Website or Windows application. He replied NBA.com and I asked him why NBA without realizing that the SUT was already announced.

Immediately, we started our task and we listed out our focus areas for the next 30 minutes. Satish was testing from the cab and sharing files over Skype. We had the Skype group chat going where each one would highlight the bug and send the screenshot. My role was to replicate, file the bug and do some testing when the bug flow was manageable. Sundar was focusing on the different tours, I took up the quick tests with different checkers and Rahul was testing on iPad. Satish was testing on Android smartphone.

We found quite a decent number of critical bugs and in no time, there was just 45 minutes left.

We covered most of the areas planned for testing and I started consolidating the report. Sundar was filing the bugs discovered by Satish and Rahul. He was also helping me with the mind map for additional testing scenarios. At 12.31pm, we were done with testing and finalizing the test report. We emailed it to the mentioned email address and quickly received a confirmation from organizers that they received the report.

Key Highlights:

- Previous experience of working under severe time pressure helps.

- Getting our test reports reviewed by the judges pointed us to key areas we had overlooked.

- Practice sessions helped us finalize on a convention within the team.

- Do not panic when you face a trouble - think through and take steps to solve it.

That was one contest well planned and executed :)

Now, waiting for the results.

Leia Mais…

Testing Mindset: Should we compete against each other?

                          

I participated in a recent competition conducted by www.99tests.com

There was a restriction of 40 testers and limited to 4 days contest. I saw few new testers join this contest. Whenever I join the contests, I do not jump in right from start. I wait for few hours and watch the bugs logged. I observe what areas are being tested, what testing oracles are being applied and so on. One of the advantages of starting late is that I get to know the thought process behind bug validation. What kind of bugs are termed as invalid and what bugs are accepted - these are useful information for me participating in the contest.

So, this time too I waited for sometime before starting my testing sessions. There were close to 20 bugs logged and some of them were validated too. The top three listed on the leaderboard were unknown to me. I love this challenge. New faces, new application and this application was a bit tough to understand. I felt the users had more of read-only access and no rights to create the data. It was different from the usual applications where the bugs danced right in front of you.

End of Day 01: Total bugs logged: ~200

Day 02: ~350

Day 03: ~400

Once the competition ended, a thought stuck me:

What made the testers log so many bugs even though the contest was a tough one in terms of easy bugs?

Is it the mindset of *find bugs at any cost* or *the five prizes* or *the competition mode* that brought out so many bugs out in the open?

In the daily projects, most of the times - there is a bug pattern: xx number of bugs. I have rarely seen so many bugs logged in such a less time. I do agree that the number of testers is different and might be the biggest factor in the number of bugs logged. There is obvious difference in having forty testers testing an application and say five testers testing an application.

It might not be feasible to add so many testers to every project. So, my question is: 

Should the testers test with a competition mindset once a while? 

It might be a good idea to engage a group of testers to test as if there were in a competition and there were prizes for the top testers. Maybe spend ten hours/week and see how it works...

I am going to try it out with my team in few days time :) Idea worth trying?

Image Credits: http://mindspower.com/mindspower-strategies-ii/out-%E2%80%93-innovating-the-competition/

Leia Mais…

Are you a 99tests tester?

After a routine day at office, I am in my cab to home. Relaxing at the last seat, chewing Center Fresh and listening to songs, I am lost in my thoughts... What makes me confident to take up any testing tasks without any fear? I am about to complete five official years of software testing and looking back, I need to credit my success to many people and many tasks I did again and again.

The long post is reserved for the fifth anniversary. Today I want to highlight how a single community has helped me grow and is helping many testers to grow. Few months ago, when I googled for 'Beta Testers', I hit the site www.99tests.com I registered as a tester and as the site was in Beta stage, there was nothing in store for a 'hungry-to-test' tester like me...

Few months passed, I had even forgot that I had registered for the site when I received the email about a new testing competition. Immediately I joined and found many colleagues from the testing community already in the hunt for the top tester for the competition. Healthy competition and true test of testing skills were on display. The best part about these competitions is the short duration. They are spread over few days (3-5 days) and lots of value to be gained compared to the few hours spent. One gets to know the thought process of many testers, learn different test ideas and earn some money too. I regularly participated in most of the 99tests competitions and had healthy contests with my co-testers.

What have I gained from 99tests?


Time management: Few contests taught me how to prioritize my testing tasks. Though the deadline was two days away, due to the high number of bugs, the competition would end 24 hrs in advance. Such experiences helped me learn the importance of time management and prioritizing the testing tasks.

Test Ideas: There were few competitions where I learnt a totally new test idea which I was unaware of before the competition. The new test idea got added to my list of test ideas.

Customer's Expectations: Though few bugs seemed very obvious, the customer would mark them as invalid. This happened to most of the testers in the competition. I learnt about what bugs actually mattered from the customer's point of view.

Healthy competition: Few competitions were so close that I felt I was playing international cricket. There was a healthy competition to be one among the top three testers.

Test of testing skills: After participating in 5-6 competitions, I tried testing my skills. I joined the competition late on purpose and tried if I could still win it. Sometimes I could and sometimes, the lead was too big to make a entry to the top three.

Variety of products: I must have participated in 7-10 competitions and that means 7 varied products. It is an amazing experience to learn the product, log bugs as per the customer's expectations and win the competition. You need to experience it to understand the feeling.

And I gained some money too...

I am proud to admit that 99tests has played a big role in my learning curve and continues to do so.

I am a 99tests tester. Are you one? 

Better late than never: join www.99tests.com and enjoy testing :)

Leia Mais…

VISTACON Bug Hunting Competition Winner :)

August seems to be the most busiest month for me. Last year, WeekendTesting was born on August 1st and WeekendTesting had its first public session on August 15th, 2009.

This month, I participated in two competitions.

LogiGear's 2010 Bug-Hunting competition

The competition was quite simple and straight forward. 

The tester had to register themselves, test the TestArchitect™ , submit the bug report in a template provided by the organizers. The judging criteria was also mentioned on the website - bug severity and quality of the bug report. I was happy after reading the conference schedule. Take a look at the impressive list of keynotes and track sessions. 

Registration:

As soon as I read the competition guidelines, I registered on July 19th. The reply was instant. I got my login credentials and the URL to the application.I downloaded the TestArchitect™ User Manual (.PDF)and the bug report template(.XLS). I immediately saved the documents in my dropbox so that I can access it from my mobile too.

Preparation:

In the first week of August, I took a hard copy of the User Manual. Everyday on the way to office, I jotted down my test ideas on the printout. August 15th was the deadline for uploading the bug report. August 16th was the deadline for the uTest Bug Battle. As it was a sunday, I got up quite early (9 a.m.) and started working on the TestArchitect™ application. 15 minutes passed and the television screen started flickering, the tubelight went off and I heard a loud sound. Some of the equipments on the electric pole on the street were burnt. I switched off the computer and hoped that someone would quickly repair the equipment.

1 p.m. : Power came but again low voltage prevented me from starting the computer.  :(

Execution: 

Finally after lots of excuses and concentrating on other tasks, it was 9 p.m. [3 hrs to the deadline according to IST]. I wanted to sleep and blame the power cut for not participating. But then, I realized that I would not forgive myself later. An opportunity was present right in front of me and I was hunting for excuses. Finally, I wanted to give it my best shot and leave the rest to the judges.

I started hunting for bugs and then realized that its better to prepare a model first and then hunt for risky bugs.

The complex application was slowly unfolding as individual features, the examples by other participants helped me learn quickly. I was aware of the *risk* that I was not the only one interacting with the application under test[AUT]. I kept all my test pages private [I did not want to share my test ideas in public ;)]

Half an hour into testing, I found an issue - High severity. 

'Unable to use three of the five options given to the user'. Awesome Ajay. I continued and was continuously taking screenshots and saving them in the dropbox. I was praying that there be no more power cuts.

I planned to stop at 10.30 p.m as my previous experience with the Test Crowd and other projects meant that bug-reports also took considerable time.

I started composing the bug report and was focussed on highlighting the important issues. 11.48 p.m and I had reported 13 issues. Yes, 13 seemed to be an unlucky number. I logged one more issue and finally ran the spell check, verified the attachments, severity and read the instructions for bug reports for one last time. 

So, before it struck 12, I had emailed the bug report and took a screenshot of the 'Report successfully uploaded.' message.

I was happy during this bug-hunting experience. Does emotions affect testing? 

Well, that's a different blog post on the way. 

Results:

The results were supposed to be announced on Aug 31st but to my surprise, I received an email on Aug 24th that read: "Congratulations! You are one of the winners of the BUG HUNTING CONTEST 2010"

Super, I won and that meant I can attend VISTACON. I can attend track sessions by Dr.Cem Kaner and other eminent personalities. 

Ho Chi Minh City: I'm coming... :)

And if you are wondering what was the second competition I participated in, a blog post is on the way. 

Stay tuned :)

Leia Mais…

Are you the BlogSTAR

I want to make this blog post simple and direct.

       Do you want to be part of       

EuroSTAR 2010?

Here is the opportunity. Grab it.

BlogSTAR 

"The winning blogger (known affectionately as BlogSTAR) will get an ALL EXPENSES paid trip to EuroSTAR 2010. Flights, luxury airport transfer, accommodation for 4 nights, ticket to the gala awards party and €500 for meals and champagne, as well as exclusive access to the EuroSTAR blog for the conference week, and celebrity access for the year ahead."

So, go ahead and register. Good luck to you and ;) wish me the same.
I'm also participating.

Leia Mais…

Software Testers: Can you take up this challenge?

Do you like fame?

Do you test software?

Do you like to be interviewed?

Do you like to be featured on the Test Republic Forum?

Do you like to be appreciated?

Do you like to win prizes?

Do you like to test yourself?

Do you like to compete with the best?

Do you like challenges?

Do you have the confidence that you are a good tester?

If you answered Yes to most of the above questions, then read no further. Just click on this link and start testing your testing skills.

Good Luck.

Last date is 12th July 2009. :)

Leia Mais…