My most embarrassing testing mistakes. (Part 2)

Premature Celebrations:

This is about a new product with six programmers and only tester (yours truly).

Something about the application:
This application had a login screen and as usual, unless one logged in with correct credentials, one could not use the application features.

I always have the habit of enabling the option 'Auto-hide the taskbar'. So even if a program is running, I would not know if its minimized unless I moved my mouse pointer near the bottom of the screen.

Now here is a classic example of me making a stupid mistake.

I launched the application by running the shortcut on the desktop.
I logged in with correct credentials and as my friend came to talk to me, I minimized all the programs including the application under test by pressing CTRL - D.

After talking to my friend, I saw the desktop.
Oh I forgot to test the particular scenario I had planned to test before my friend came to me.
Good that I remembered.

Let me launch the application. The moment I double clicked on the shortcut, I found something interesting.

The login window was on the foreground and on the background I had the application feature running.

For few seconds, I was a bit shocked with just one thought running in my mind.

"Have I cracked the login feature of this application. Have I bypassed the login screen? Have I found a way to access the application without entering any login credentials."

Cool! I had found a very critical bug.

When I moved the login screen aside and brought the focus to the application feature, I'm able to access the application without logging in.

As usual when I find a good bug, I called up my programmer and showed him this strange behavior.

He was confident that its impossible to see such a behavior.
Unfortunately for me, the programmer's manager was also around.

And why unfortunate? You'll soon know.

My application even if minimized would be restored to the full screen even if any other application is launched.

So what was a critical bug according to me turned out to be not at all critical.

The issue with the strange behavior had nothing to do with the login feature being broken.

It was just that when I double clicked on the shortcut, another instance had launched.

Again I had to face the sarcastic smile and remarks of the programmer.
I was so embarrassed that I felt like hiding somewhere far from these people.

A typical example of premature celebrations and lack of critical thinking on my part.

Lessons learnt:
> Generate new ideas to prove your bug as 'Not a bug'. The bugs which are not actual bugs would be filtered out.
> Learn to investigate any issue before terming it as an issue.
> Try to reproduce the issue and know its consistency before you talk to programmer.
Sometimes, its good to show any issue to your co tester in your team and have it criticized instead of making a fool of yourself in front of the programming team.
> Try to avoid disturbances if you are on a critical mission.
> Learn to focus and defocus and focus back immediately.

Learn to avoid raising false alarms like the boy in this story :
http://en.wikipedia.org/wiki/The_Boy_Who_Cried_Wolf

Leia Mais…

My most embarrassing testing mistakes. (Part 1)

Testing on the wrong build.

Every build was uploaded on the server on a particular day of the week. The release team would send an email once the build was uploaded on their side. It would take 4 to 5 hours for the build to be synched in to the local server. The testing team would check the date on the local server and copy the build from the local server and start their testing.

I was the one responsible to copy the build from the local server and start testing the build. After receiving the email from the release team, I waited for 4 hours and just copied the build folder.

I copied the folder, installed the build and started testing the last build.

I even logged a bug that the application about screen displayed the previous build version. So, the bug read ' about screen reads x.x.5 instead of x.x.6.

Continuing with my testing, I reopened two issues which were assigned to me as fixed. When the third bug fix also failed, I got suspicious and checked the build folder on the local server. It read the latest date and I thought the issues were not fixed on this build.

The programming team was also surprised and then checked the installed folders and found one of the .dll to be the old one.

The build folder was modified a week ago and the latest build had not yet been synched in when I copied the build. I did not care enough to check the folder modified date.

Wow!!! I was testing on the wrong build for the past 6 hours.

I had to uninstall the application, cut a sorry face in front of my manager and the programming team.

An experience I would never forget. Such mistakes often occur due to carelessness, casual outlook to testing and sometimes due to over confidence like in my case.

Lessons
From then on, I always check the build folder carefully and verify the one bug that clearly differentiates the two builds. I also have cultivated the practice of taking the screenshots of the folder contents of the current build and comparing it to the previous build screenshot and then start my testing once these checkpoints are cleared successfully.

Update:
You can ask me - What makes me enjoy such moments?

I enjoy my testing with the belief that such mistakes would not be repeated.
A waste of time/resource/cost to the company and to myself.

I'm happy that the mistake was not an expensive mistake.

Testers!!!
Beware of testing on the wrong build :)

Leia Mais…

My First Guest Lecture :)

It's been a while since my last post.

As I was busy preparing for my exams ('MS in Quality Management' from BITS, Pilani), I could not devote some time to my blog. The other reason being lack of any interesting topic in my testing life to blog on.

Finally, I got a reason to blog.

I delivered my First "GUEST LECTURE" to a batch of students undergoing a program titled

"Diploma in Software Testing"

The presentation is uploaded as a video.

If you want to use this video or its contents for commercial purposes, please send an email to ajay184f@gmail.com or you can provide a link to this post. Plagiarism is strictly prohibited and punishable under 'IT Act 2000'

Feel free to comment, agree, disagree, challenge, agree to disagree, question, appreciate, suggest on my testing approach and the post.

Update: I gave my guest lecture to the students of Edista's Finishing School.
It was a unique experience considering the fact that it was my first presentation on testing for students of testing.

There were quite a lot of drawbacks coupled with good things in my presentation which were well noted by Pradeep Soundararajan.

Every move I make in my software testing life, I learn a lot and I know that I need to unlearn a lot.

Improving day by day, practicing day by day, one day I'll become the 'Expert Tester' I dream of. :)

Leia Mais…

Interesting defect

This is about a defect I found in my application this morning.

The application was not tested for a month due to some requirements change. I was shifted to test some other products and finally I got my application back.
There were supposed to be lot of changes introduced as per client request.

My application has a login window as soon as it is launched.The login window has fields username & password and OK & Cancel buttons.Username: guest, password: guest

So, the first test is as follows:Enter guest in the username field and guadt in the password field.Click OK.Observed Behavior:A window with the message 'Incorrect username/password.' popped up.This window too had a OK button.
I clicked on OK and clicked again on OK button in the Login window.

As stated in www.satisfice.com/rst.pdf slides 108 and 109, I wanted to test 'Error Handling'.I kept on pressing 'Enter' key in the keyboard.

This happened for more than 20 times and I was reminded of M.Bolton's statement in one of the post : 'Do the right thing after you have tried the incorrect thing' (Not the same words)

So, I thought of entering the correct credentials as I had tried enough of incorrect credentials and the application is not allowing me to enter.

I entered 'guest' in both the username and password fields.

Bingo!!!

The same window : 'Incorrect username/password' popped up.

I deleted the password and entered the password 'guest' again carefully and slowly.
Again the same message.I found something interesting here.

I went to the developer and asked whether they had changed the password and they replied 'NO'.

I closed the instance and opened a new instance.Entered 'guest' in both the fields and the application opened without any problem.
So, I realized that entering incorrect credentials and then entering correct credentials might reproduce this defect.

I tried entering incorrect credentials 10 times and the correct credentials at the 11th time. Application worked.
15 times, it worked.
20 times, it did not work, message popped up.
19 times, it worked.
21 times, it did not work, message popped up.

So, finally I logged a defect that 'After 20 incorrect logins, login with correct credentials also fails.'

Test Ideas that helped me find this defect:
Error Handling technique.
M.Bolton's statement.
Ben Simo's 'Failure' mnemonic

Have you ever come across such a defect?
Have you come across defects more interesting and wierd than this?

Please share the test ideas too as it may help others.

Scripted Testers:Would this defect be found by the execution of test case?

Feel free to question, discuss and comment.
Regards,
Ajay
www.enjoytesting.blogspot.com
ajay184f@yahoo.com

Leia Mais…

Base state of a software application

This is a post on the importance of base state or initial state of a software application.
In the last post, I had mentioned about activating mobile office on my mobile.

I had not taken any note of the condition of my mobile connection settings. After I deactivated my mobile office, I was unable to access any websites on my mobile. After a lot of trial and error, setting up some other settings, I decided to approach the customer care and ask them to solve my problem. Finally I got it solved.

Observations:
> I was careless not to note the initial settings of the connection.
> I should have gathered more information about the new service(mobile office) as it did not support my access to wap enabled websites.

Connection with testing:
How many times have we come across a situation where we could not reproduce a defect or a behavior due to the change in the initial settings. We waste a lot of time reproducing the issue and finally realize that it was environment specific and had to ghost the system and start afresh. After this mobile experience, I realized the importance of noting the settings of my mobile. In testing, an environment of a test plays an equal role as the test itself.

Lessons learnt:
A tester has to know each of his test case/input. We never know which test may give the unwanted result/behavior.
How about running a test in our mind, imagining the probable consequences before running it in real. If testing was that simple, anyone could do testing.

Sometimes, we do need to probe a defect further to unearth even more critical defects. At the same time, we have to be careful to note the initial/base state of the application.

So, how do you deal with the ever changing state of a software?
Do you know each test you do?
How do you handle the issues you fail to reproduce at the first attempt?

Share your comments. And finally a question to you:
Is it necessary to know the initial/base state of the software under test?

Happy testing :)
-Ajay

Leia Mais…

Testing and Chess, Any Connection???

This is about a conversation I had with a Customer care executive.

I wanted to access internet on my desktop pc with my mobile as the modem. My mobile was connected to the pc through a bluetooth dongle plugged in the pc. I had to configure some settings in the pc. So, I called up customer care and the executive asked me to switch on the Loudspeaker mode and follow his guidelines.

I switched on the Loudspeaker mode and at the first instruction itself, we both are stuck.

He says: "Sir, go to Start button, click on it, you will find Settings option".

The option 'Settings' can be viewed only if the menu style selected for your system is 'Classic Start menu'. I had the first option 'Start menu' selected for my pc. So, I was not able to see the option: 'Start > Settings'.

Me: 'What is the option you are looking for?' CE: Sir, first get to Start > Settings. It'll be present in all the computers. Me: I know that, but now I'm not able to find the option.

Give me some time. ******** I change the option in Control Panel > Taskbar and Start Menu to 'Classic Start menu'. ******* CE: Go to Network Connections. As I already had a lot of network connections configured, placing the cursor on the option 'Network Connections' displayed all the available connections. I had to add a new connection. Finally, I could configure the network connection. This took around ten minutes.

Observations: > The executive is not ready to listen to me and stuck with the same sentence: You must find 'Settings' option on clicking 'Start' button. > I'm not informed about my mission (Setting up a new network connection). > My questions are not answered (What is the option you are looking for? Ans: Network Connections) > The executive is not ready to believe that the expected behavior is different from the observed behavior (reason: option selected in Taskbar and start menu display)

This one call with the customer care executive gave me valuable lessons to ponder on:

Lessons I learnt:
> Testing is like Chess. In chess, you think about different moves possible by your opponent and play the best move according to situation. Similarly, in testing it's better to keep in mind that for an input, there may be more than one output.
> As the executive was stuck with the process and not ready to move away from the process, he made me(the end user) unhappy. User is more happy if the end result is achieved and not the process.
> Be open for options.
> Be ready to help your customers(A good post: Educate your customers : http://testertested.blogspot.com/2008/02/educating-customers-on-testing.html ) Yesterday I understood the real meaning of educating the customers.

Feel free to comment. Share your experiences and views. God save customers from such executives...

Leia Mais…