VISTACON Bug Hunting Competition Winner :)

August seems to be the most busiest month for me. Last year, WeekendTesting was born on August 1st and WeekendTesting had its first public session on August 15th, 2009.

This month, I participated in two competitions.

LogiGear's 2010 Bug-Hunting competition

The competition was quite simple and straight forward. 

The tester had to register themselves, test the TestArchitect™ , submit the bug report in a template provided by the organizers. The judging criteria was also mentioned on the website - bug severity and quality of the bug report. I was happy after reading the conference schedule. Take a look at the impressive list of keynotes and track sessions. 

Registration:

As soon as I read the competition guidelines, I registered on July 19th. The reply was instant. I got my login credentials and the URL to the application.I downloaded the TestArchitect™ User Manual (.PDF)and the bug report template(.XLS). I immediately saved the documents in my dropbox so that I can access it from my mobile too.

Preparation:

In the first week of August, I took a hard copy of the User Manual. Everyday on the way to office, I jotted down my test ideas on the printout. August 15th was the deadline for uploading the bug report. August 16th was the deadline for the uTest Bug Battle. As it was a sunday, I got up quite early (9 a.m.) and started working on the TestArchitect™ application. 15 minutes passed and the television screen started flickering, the tubelight went off and I heard a loud sound. Some of the equipments on the electric pole on the street were burnt. I switched off the computer and hoped that someone would quickly repair the equipment.

1 p.m. : Power came but again low voltage prevented me from starting the computer.  :(

Execution: 

Finally after lots of excuses and concentrating on other tasks, it was 9 p.m. [3 hrs to the deadline according to IST]. I wanted to sleep and blame the power cut for not participating. But then, I realized that I would not forgive myself later. An opportunity was present right in front of me and I was hunting for excuses. Finally, I wanted to give it my best shot and leave the rest to the judges.

I started hunting for bugs and then realized that its better to prepare a model first and then hunt for risky bugs.

The complex application was slowly unfolding as individual features, the examples by other participants helped me learn quickly. I was aware of the *risk* that I was not the only one interacting with the application under test[AUT]. I kept all my test pages private [I did not want to share my test ideas in public ;)]

Half an hour into testing, I found an issue - High severity. 

'Unable to use three of the five options given to the user'. Awesome Ajay. I continued and was continuously taking screenshots and saving them in the dropbox. I was praying that there be no more power cuts.

I planned to stop at 10.30 p.m as my previous experience with the Test Crowd and other projects meant that bug-reports also took considerable time.

I started composing the bug report and was focussed on highlighting the important issues. 11.48 p.m and I had reported 13 issues. Yes, 13 seemed to be an unlucky number. I logged one more issue and finally ran the spell check, verified the attachments, severity and read the instructions for bug reports for one last time. 

So, before it struck 12, I had emailed the bug report and took a screenshot of the 'Report successfully uploaded.' message.

I was happy during this bug-hunting experience. Does emotions affect testing? 

Well, that's a different blog post on the way. 

Results:

The results were supposed to be announced on Aug 31st but to my surprise, I received an email on Aug 24th that read: "Congratulations! You are one of the winners of the BUG HUNTING CONTEST 2010"

Super, I won and that meant I can attend VISTACON. I can attend track sessions by Dr.Cem Kaner and other eminent personalities. 

Ho Chi Minh City: I'm coming... :)

And if you are wondering what was the second competition I participated in, a blog post is on the way. 

Stay tuned :)

Leia Mais…

Relationships Matter - Tester and Programmer

This post is close to my heart. I dedicate this post to all the programmers I have interacted with. My official testing career is just over four years old. In these four years, I'm very lucky to work on multiple products and interact with many programmers. Initially, I was worried that I never worked on any product for a full release cycle. After few months I realized that this continuous swapping between products meant increased interaction with different sets of programmers.

First product:
The programmer was my friend. Both of us joined the company on the same date. I always got the news about the changes in the build, which feature would be implemented, confidence level of the programming team and many more *secrets* before they were officially announced later. He used to tell me the bugs in others' code and I used to help him by testing his fix before it went into the build. Both were happy with this *adjustment* until the project was scrapped.

Second product:
I was the only tester in this project and there were four programmers. By this time, I knew that programmers are a good source of information. I also realized that they are ready to talk about their work. They too feel proud on doing a good job. My interactions with this team was more informal. We had more Coffee day meetings than formal meetings. I gave them my test suite and highlighted the different scenarios. They taught me ways to analyze log files and answered all my questions about the product.

I worked on many more products and I'm very happy to say that my programmers like to work with me. I'll share some of the tips which might help improve your relationship with programmers:
* Remember that they are human beings first and then programmers. Give respect and take respect.
* Programmers do not code to introduce bugs. If you think programming is easy, exchange your job responsibility for few hours.
* As Michael Bolton and James Bach highlight, our job is not to prove them wrong or make fun of them. Help them understand that you are helping them and not finding faults.
* Give them the information which would help them solve issues easily and quickly. Improve your bug investigation skills.
* Appreciate in public when they fix a very difficult bug, on their good work. People like to be encouraged and appreciated.
* Be patient, listen more, complain less, help more, fight less, talk more, argue less, discuss more, work together towards one mission.

And don't forget: 'Relationships matter' :)

Leia Mais…

Aware of the other side of your application ?

Logged in from eBuddy

For the past few months, I left office at sharp 6 p.m. I felt I should not invest more hours just because someone's estimate was wrong. So, I always took the 6 p.m. cab to home instead of the 8 p.m. or 10 p.m. cab.

To be connected to my colleagues and other friends who want to contact me on Google Talk, I enabled eBuddy on my mobile. Even when I was away from computer, I was able to respond to my friends and anyone who pinged me. I like to keep my status message simple such as a ':)' unless there is a particular link to be shared or I need information quickly.

Last evening, my colleague pinged me asking for some information.
I replied to it after two hours. This morning, I pinged him again and asked why he pinged me and did he get the answer to his question.

He replied that he got the message but the status message seems to repeat nearly 8-10 times. Wow, I never knew this. I had seen similar issues when the recipient was on a vacation or was online from mobile. I never paid much attention to this behavior.

Today, when my colleague highlighted this issue and also named the screenshot as 'SPAM', I understood that he was not happy with this behavior. I was totally unaware that my friends got such messages from my account.

I'm Sorry. I thought:

Are we aware of the side-effects of any application?
How many customers complain if they face a problem?
How many times we learn to live with the problem?
How can we train ourselves to see what others might see?
Do we keep silent if the value outweighs the problem?

Finally a thought from Jerry Weinberg's book - "Are your lights on?" struck me:

The trickiest part of certain problems is just recognizing their existence - Jerry Weinberg

Have you faced such problems - which you were not aware of unless someone else pointed it out?

Leia Mais…

No test is also a test ! Agree?

One of my colleagues - a tester logged an issue and the issue was assigned to a programmer. The issue was a CRASH and hence quite important. My colleague went on a vacation and the issue had to be verified. My team lead assigned the issue to me.
I was happy on receiving this task as it meant two things:
a. Escape from executing test cases.
b. A new challenge to apply my bug investigation skills.

On going through the details of the issue in the bug tracker, I was surprised. There were multiple comments and the issue had oscillated multiple times between the tester and the programmer. It reminded me of the BBST Bug Advocacy class where Dr.Cem highlighted that we should comment only if necessary as every comment meant an extra email to the concerned people.

The programmer had asked the tester to give additional details, verify on the latest build and so on. How many times have we faced this scenario - the bug report is sent back to the tester in need of additional details.

I asked my team lead if anyone else had worked on the issue. He replied that he was able to reproduce the issue but it was not consistent. I read the latest comment by the programmer - he was unable to reproduce the issue and wanted the tester to try on the latest build. I installed the latest build but on an incorrect operating system. The issue was reported on Windows operating system and I tried on Macintosh. I wanted to be sure that the issue was not reproducible on Macintosh.

First attempt: Unable to reproduce.
Second attempt: Unable to reproduce.
My friend who was sitting next to me teased me: Can't you see that the issue was logged on Windows? Why are you trying on Macintosh? I decided to verify on Windows machine.

When the installation on Windows was in progress, I wanted to try one more time on Macintosh. Once I was on the last step, my mobile rang. I received the phone call and after few seconds, I was surprised to see a crash on Macintosh. Hurray, I was able to reproduce the crash.

I started thinking - What did I do different?
Did I change the order of steps? No
Did I use a different instrument? No
Did I use a different file? No
Did I change the configuration? No
Did I use a different version? No

Then I realized that the one thing different in the third attempt from the previous two attempts was the phone call. I'd look stupid if I said phone call was the cause of the CRASH. In one way, phone call helped me identify the cause.

As I received the phone call and I was at the last step, I stopped moving the mouse and the window was open even though the task was supposedly complete. So, in the first two attempts, I had closed the window as soon as the task was complete. Hence, I missed the CRASH.

In this case as the window was open for more than a minute after the task was completed, I was able to notice the CRASH. When I highlighted this point to the stakeholders, many confirmed that they too had closed the window as soon as the task was completed.

I learnt that time delay between events plays an important role in discovering some issues.

No test is also a test ! Agree?

Leia Mais…

A valuable lesson - Save your data

I was in my cab after the weekly round of carrom. The only valuable task apart from sleeping in cab was to write a blogpost. I started writing the blogpost on my mobile. I almost finished the blogpost - which took almost forty minutes.

I was writing the last few lines and I received an email on my mobile. Out of sheer carelessness, I pressed the end key and replied to the email. Then I realized that on pressing the end key, all the contents was lost. I wish there was the autosave feature similar to the blogger on web.

How often do we forget what we were doing and press a key or click on some button. How often have we lost crucial screenshots or wasted a lot of time due to our carelessness. When I relate what happened now to testing, I'm reminded of policemen who investigate a crime scene. As the policemen are trained to not touch any objects or bodies with bare hands, are we trained enough not to perform any actions without thinking?

Think before you ink
Do we think before we click?

Are we skilled enough to be aware of our every action during testing? Is this what is known as 'Situational Awareness'
This particular experience will definitely make me think of 'Autosave'.
* Does the application support 'Autosave'
* Frequency of Autosave
* Overwrites existing information
* Other processes are inactive till Autosave is complete or the user can perform other actions
* File saved before; unsaved file

Its quite interesting that one carelessness led to a blogpost. Before I exit the browser or battery goes down, let me publish this blogpost :)

Leia Mais…

No testcases for the scenario. What do I do?

One of my seniors is a big fan of Silk, Selenium, Loadrunner, TestComplete and any other commercial tool available in the market. He is very good at automating few tests depending on the product. My team lead had no tasks for me. So, he asked me to learn Selenium from the senior. My senior demonstrated some basic commands of Selenium and asked me to play for few hours. Now the next question was: 'Which application to use?'

There were two choices presented to me:
a. An application yet to be released to the market and still being tested.
b. An application released to the market after being tested by the team reporting to the senior.

I chose the second application. My senior told me that I'd not find any bugs as it was tested by his team. I took this as a challenge and explored the application.
In twenty five minutes, I found two inconsistencies. The severity of the issues was high too. With a big smile on my face, I called my senior and showed him the two issues. He immediately called the tester who tested the feature.

On being asked how the issues were not found, the answer given by the tester shocked me. His answer was: 'We do not have test cases for those scenarios'

Few strange lessons I learnt:

1. Some testers fail to test beyond the testcases. Should I call that testing?
2. Testcases give a false sense of security to some management.
3. Forcing a tester to learn any automation tool might not be a good idea in the long run.
4. A new pair of eyes - A new test idea - might lead to discovery of a new issue.
5. Its not a good idea to criticize or interrogate your team member in public.
6. I took it as a challenge to find bugs. Did the challenge attitude help me find those issues or the challenge had no effect?

What would be your answers to the question:
'Why did you miss those issues?'

Did I tell you that the tester added the two test ideas as testcases later?

Leia Mais…

Interesting Virus and funny bug investigation

Cool, the last post was not that bad when compared to a blogpost from web. In terms of the format, the last paragraph had 2-3 additional line breaks. Hopefully, I'll not repeat the same mistake in this blogpost.

Today I reached office few minutes late. Two of my team members had already started working on the build released on friday. I heard one of the two team members talk about Virus with the systems guy.

Incident No.1

I heard one of the two team members talk about VIRUS. I thought she was talking about the VIRUS character from the '3 Idiots' movie. On seeing the systems guy, I understood that its the VIRUS - the one we worry about. The network cables were disconnected and the systems guy was busy checking the security updates, patches and other vital information.
Hmmm, just when I thought 'one resource down' for the day, the systems guy was laughing loudly and my colleague was smiling. I was wondering what happened and what was so funny? The systems guy left and I went to my colleague to know more about the incident.

# My colleague had called the systems guy. Her exact words were: 'A pop up says that there are 7 virus detected and the xxxxxx antivirus has not detected the virus. Please come fast. I've disconnected the cables.'
# How could this happen? Why did the antivirus not detect the virus? How did the virus breach the antivirus barrier? How risky was this virus? How many files and computers were affected?
# Simple reason was: It was NOT a VIRUS. It was one of those funny ads on the website which tries to distract the user and install some junk toolbar on the browser.

No wonder the systems guy was laughing so loudly.

Learning:
1. My colleague was so focussed on the application under test that she failed to look at the bigger picture. Is this an example of 'Inattentional Blindness' or 'Lack of DeFocus principle' ?
2. She could have investigated a bit more before calling the systems guy.
3. She could have called for help within the team.

Incident No.2:
Colleague next to me had to reproduce a customer issue to the programming team.
What is the scenario? Let me describe it. Our software is used to print a photo and a footer with details of the photo. Around 4-6 lines were printed as the footer. The first line was for the title of the photo.
What is the issue:
The first line of the footer was not printed completely in Japanese language. The programmer was not able to reproduce the issue and the customer had attached a screenshot of the problem as a pdf file. The pdf file clearly highlighted how the first line of the footer was not printed completely.

My colleague's approach:
As he was not familiar with Japanese language, he wanted to reproduce the issue in English. He printed the pdf and found that the first line of the footer was not printed completely. An email was sent to the programmer that the issue was reproducible.
Five minutes later, the programmer was in my colleague's cubicle and what came out of the small discussion was a bit funny.

My colleague had printed the pdf without the footer setting. As the pdf had the screenshot of the issue, my colleague thought that the footer was not printed correctly. :)

Learning:
1. Carelessness or Lack of focus?
2. Pressure to reproduce an issue
3. Importance of bug investigation skills.

Two incidents in one day... Lets see in next blogpost if there are any other interesting experiences.

Leia Mais…

Mobile Blogger :)

My new mobile:

I like to use my mobile more as a computer than a mobile. I always wanted to buy a mobile which would help me check emails, browse sites, watch videos, tweet, skype, chat and be connected to the online world. Having an unlimited internet connection on my mobile definitely helps. So, after losing my last mobile, I bought myself a Nokia E63.

Everyday learning:

So, with a new tool and loads of excitement, I started learning about the various features. I made quite a few mistakes, few factory resets, lots of installations and configurations. I started with installing Skype, Snaptu, ebuddy, Screensnap (an application to capture screenshot on mobile),,, the list just goes on.

Surprise Application:



Today when I was accessing a website, I observed an ad which read: Download OperaMini 4.1 (Customized for Vodafone services). I downloaded and on installation, I observed an icon which looked familiar - 'Blogger'.

And this is the first blogpost from my mobile. I hope to blog more frequently from my mobile. Meet you soon with yet another blogpost describing my experience.

Leia Mais…