Wednesday, September 17, 2008

My most embarrassing testing mistakes. (Part 2)

Premature Celebrations:

This is about a new product with six programmers and only tester (yours truly).

Something about the application:
This application had a login screen and as usual, unless one logged in with correct credentials, one could not use the application features.

I always have the habit of enabling the option 'Auto-hide the taskbar'. So even if a program is running, I would not know if its minimized unless I moved my mouse pointer near the bottom of the screen.

Now here is a classic example of me making a stupid mistake.

I launched the application by running the shortcut on the desktop.
I logged in with correct credentials and as my friend came to talk to me, I minimized all the programs including the application under test by pressing CTRL - D.

After talking to my friend, I saw the desktop.
Oh I forgot to test the particular scenario I had planned to test before my friend came to me.
Good that I remembered.

Let me launch the application. The moment I double clicked on the shortcut, I found something interesting.

The login window was on the foreground and on the background I had the application feature running.

For few seconds, I was a bit shocked with just one thought running in my mind.

"Have I cracked the login feature of this application. Have I bypassed the login screen? Have I found a way to access the application without entering any login credentials."

Cool! I had found a very critical bug.

When I moved the login screen aside and brought the focus to the application feature, I'm able to access the application without logging in.

As usual when I find a good bug, I called up my programmer and showed him this strange behavior.

He was confident that its impossible to see such a behavior.
Unfortunately for me, the programmer's manager was also around.

And why unfortunate? You'll soon know.

My application even if minimized would be restored to the full screen even if any other application is launched.

So what was a critical bug according to me turned out to be not at all critical.

The issue with the strange behavior had nothing to do with the login feature being broken.

It was just that when I double clicked on the shortcut, another instance had launched.

Again I had to face the sarcastic smile and remarks of the programmer.
I was so embarrassed that I felt like hiding somewhere far from these people.

A typical example of premature celebrations and lack of critical thinking on my part.

Lessons learnt:
> Generate new ideas to prove your bug as 'Not a bug'. The bugs which are not actual bugs would be filtered out.
> Learn to investigate any issue before terming it as an issue.
> Try to reproduce the issue and know its consistency before you talk to programmer.
Sometimes, its good to show any issue to your co tester in your team and have it criticized instead of making a fool of yourself in front of the programming team.
> Try to avoid disturbances if you are on a critical mission.
> Learn to focus and defocus and focus back immediately.

Learn to avoid raising false alarms like the boy in this story :
http://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf

10 comments:

Sachidananda Benegal said...

nice set of lessons. I think
1. you should have reproduced the defect again and then seen if there is a similar behavior...if so then report...
2.keep a watch on the number of processes etc that are running along with the application being tested...
3.Try to work as co-ordinators on product development with programmers, it more of getting a better product to the market than proving the programming team is superior, or the quality team.. its important to work as a team and create the best possible soln...
4. your purpose is the crack different ways to break the work flow designed by the software...just as an user...try to get into the mind of the end consumer and see what is it that he would anticipate more from the product and give the features as suggestions for improvement.

Anonymous said...

so i've got a question, whether your product is a multi instance application. if so what will happen if you give a different credentials to the login window of the second instance? hope you have thought these things also !!!

Ajay Balamurugadas said...

@anonymous,

Thanks for the question.
That would definitely help me when I'd test an application with multiple instance capability.

The application I tested was not supposed to support multiple instance scenario.

Hence I had found the bug 'Multiple instances of the application can be launched' while commiting this mistake.

-Ajay Balamurugadas

Anonymous said...

Ajay,

Apart from the lessons you mentioned,
-you also learned, these kind of tests can be performed where ever applicable.

Yes, you can show it to developers or people who gave good knowledge about the product.But still whether it can be a bug or not, proper judgment should be made by the tester.

I mean to say, it should be analyzed instead of agreeing to whatever is been reasoned. This is because, not that the person might be wrong but it might not appear as a problem to her but might be to others. We could find out the in what circumstances it might be possible from them.
Also telling them what all was done before you saw the bug(as much as we can remember)may help them to give possible cases where it might occur.
What say?

-Bhargavi M

Ajay Balamurugadas said...

@bhargavi,

>> Apart from the lessons you mentioned,
-you also learned, these kind of tests can be performed where ever applicable.

Surprised that you could figure out what lessons I learnt...

Giving steps leading to the bug definitely helps.

Thanks for the comment.

-Ajay Balamurugadas

Sharath Byregowda said...

Ajay,

Good write-up. I have been in a similar scenario before. I found out “how easy it is to loose the credential that you have built by working and delivering smartly for a long time”. From then on I always try to test my testing deliverables before I release it to the concerned.

-Sharath.B

Ajay Balamurugadas said...

Sharath,

Thanks for your encouragement.

Learning from mistakes should be our strength. In fact, recognizing a mistake in itself is a big strength.

-Ajay

Rajesh Babu said...

Ajay

Thanks for the lesson, In my PC i tried to minimize the screen by using Windows+D . How did you minimized using CTRL+D.

Thanks
Rajesh Maadireddy

Edward said...
This comment has been removed by the author.
Edward said...

Good article, no matter how simple it may seem, these login errors are not usually easy to solve.



________________________
I work in factored ia