Testing on the wrong build.
Every build was uploaded on the server on a particular day of the week. The release team would send an email once the build was uploaded on their side. It would take 4 to 5 hours for the build to be synched in to the local server. The testing team would check the date on the local server and copy the build from the local server and start their testing.
I was the one responsible to copy the build from the local server and start testing the build. After receiving the email from the release team, I waited for 4 hours and just copied the build folder.
I copied the folder, installed the build and started testing the last build.
I even logged a bug that the application about screen displayed the previous build version. So, the bug read ' about screen reads x.x.5 instead of x.x.6.
Continuing with my testing, I reopened two issues which were assigned to me as fixed. When the third bug fix also failed, I got suspicious and checked the build folder on the local server. It read the latest date and I thought the issues were not fixed on this build.
The programming team was also surprised and then checked the installed folders and found one of the .dll to be the old one.
The build folder was modified a week ago and the latest build had not yet been synched in when I copied the build. I did not care enough to check the folder modified date.
Wow!!! I was testing on the wrong build for the past 6 hours.
I had to uninstall the application, cut a sorry face in front of my manager and the programming team.
An experience I would never forget. Such mistakes often occur due to carelessness, casual outlook to testing and sometimes due to over confidence like in my case.
Lessons
From then on, I always check the build folder carefully and verify the one bug that clearly differentiates the two builds. I also have cultivated the practice of taking the screenshots of the folder contents of the current build and comparing it to the previous build screenshot and then start my testing once these checkpoints are cleared successfully.
Update:
You can ask me - What makes me enjoy such moments?
I enjoy my testing with the belief that such mistakes would not be repeated.
A waste of time/resource/cost to the company and to myself.
I'm happy that the mistake was not an expensive mistake.
Testers!!!
Beware of testing on the wrong build :)
Tuesday, September 9, 2008
My most embarrassing testing mistakes. (Part 1)
Subscribe to:
Post Comments (Atom)
6 comments:
Thats why have a smoke test suite.
Which chks for the build version.
As soon as the build version fails,the build shud be rejected.
That what v were following i think :)
@glsandeep,
What if the build is the correct build and the version number is of the previous build.
The file which has the version number might be not replaced.
So, rejecting a build just for the incorrect version number does not help the testing team or the programming team.
It wastes a lot of time.
What do you think?
-Ajay Balamurugadas
Wow!! felt like revisiting my work at EFI,, and yes as sandeep said, the first thing v used to test was whether the version was correct or not,, whether all the installer folders were present and whether the date on the build is correct,, If at this stage it failed, v would not continue wit testing til these issues were fixed,, that would save us a lot of time and effort from testing the wrong build,, but i m sure ajay that u ll be extra cautious about the whole thing here onwards,, :-)
Why are we testers assuming that a software ( automated smoke test suite ) to not have bugs when we deal with software ( that full time programmers write ) and find bugs in it because it has a lot of them?
I don't automate my smoke test suite.
As I mentioned, I test for the folders and then proceed with my testing.
It is rightly said 'Test the tests first'.
-Ajay Balamurugadas
We learn from mistakes and thanks to them we improve a lot.
________________________
I work in factored ia
Post a Comment