One of my colleagues - a tester logged an issue and the issue was assigned to a programmer. The issue was a CRASH and hence quite important. My colleague went on a vacation and the issue had to be verified. My team lead assigned the issue to me.
I was happy on receiving this task as it meant two things:
a. Escape from executing test cases.
b. A new challenge to apply my bug investigation skills.
On going through the details of the issue in the bug tracker, I was surprised. There were multiple comments and the issue had oscillated multiple times between the tester and the programmer. It reminded me of the BBST Bug Advocacy class where Dr.Cem highlighted that we should comment only if necessary as every comment meant an extra email to the concerned people.
The programmer had asked the tester to give additional details, verify on the latest build and so on. How many times have we faced this scenario - the bug report is sent back to the tester in need of additional details.
I asked my team lead if anyone else had worked on the issue. He replied that he was able to reproduce the issue but it was not consistent. I read the latest comment by the programmer - he was unable to reproduce the issue and wanted the tester to try on the latest build. I installed the latest build but on an incorrect operating system. The issue was reported on Windows operating system and I tried on Macintosh. I wanted to be sure that the issue was not reproducible on Macintosh.
First attempt: Unable to reproduce.
Second attempt: Unable to reproduce.
My friend who was sitting next to me teased me: Can't you see that the issue was logged on Windows? Why are you trying on Macintosh? I decided to verify on Windows machine.
When the installation on Windows was in progress, I wanted to try one more time on Macintosh. Once I was on the last step, my mobile rang. I received the phone call and after few seconds, I was surprised to see a crash on Macintosh. Hurray, I was able to reproduce the crash.
I started thinking - What did I do different?
Did I change the order of steps? No
Did I use a different instrument? No
Did I use a different file? No
Did I change the configuration? No
Did I use a different version? No
Then I realized that the one thing different in the third attempt from the previous two attempts was the phone call. I'd look stupid if I said phone call was the cause of the CRASH. In one way, phone call helped me identify the cause.
As I received the phone call and I was at the last step, I stopped moving the mouse and the window was open even though the task was supposedly complete. So, in the first two attempts, I had closed the window as soon as the task was complete. Hence, I missed the CRASH.
In this case as the window was open for more than a minute after the task was completed, I was able to notice the CRASH. When I highlighted this point to the stakeholders, many confirmed that they too had closed the window as soon as the task was completed.
I learnt that time delay between events plays an important role in discovering some issues.
No test is also a test ! Agree?
Sunday, August 15, 2010
No test is also a test ! Agree?
Subscribe to:
Post Comments (Atom)
4 comments:
Hey Ajay,
Interesting story!
It might look like you weren't doing anything but you were actually giving yourself time (inadvertently) to observe the system.
It reminds me of certain help checklists we introduced into a load and performance test scenario once upon a time - after the traffic load had ramped down we'd observe and monitor the system for beyond the longest timer period known (or previously observed) to be running - this would allow us to check for unexpected behaviour.
So a moral of the story is: Always question the time delays you have in your observations. Maybe even include timings in bug reports.
Thanks Simon for the comment.
Yes, I agree: "Always question the time delays you have in your observations."
Regards,
Ajay
Although you did discover the problem by accident I will still give credit to your awareness and inference skills as a tester. Since you did recognize the problem in that scenario it was a test, but maybe not the test you anticipated.
I think this is what identifies you as a great tester. If you were untrained, even if you had observed the problem by accident you couldn't have identified it. Great work!
Thank you for sharing your wisdom with us. I would like to share information about the company. Choose Appconsultio for a seamless and scalable Flutter app development experience that propels your business to new heights. Our passion for technology, combined with our local expertise in Pune, makes us the ideal partner for turning your app ideas into reality. Elevate your mobile presence with Appconsultio– where innovation meets excellence.
Post a Comment