After Part I and Part II , here is the blog post about the CAST2011 Tester competition.
On Day 1 - Aug 08th evening, there was a testing competition for four hours from 06:00 PM to 10:00 PM US time. After the awesome sessions the first day, it was time for some fun. And how do testers have fun? Testers have fun by competing in a testing competition.
As I belong to the Miagi-Do School, all the students of Miagi-Do formed one team. There were few other testers who were not part of the school also joining us. So, the team members were: Matthew Heusser, Markus Gaertner, Michael Larsen, Adam Yuret, Elena Houser, Justin Hunter, Simon Schrijver and myself. We were never in chance of winning the cash prize of $1401 as Matthew Heusser was one of the organizers. And still we wanted to participate in this contest. We wanted to test as the Miagi-Do school team!
06:00 PM - Setting up and Learning the Application
As we had more Mac machines than Windows and the application to test was a Windows app, we paired. Michael & Matt, Markus & Adam, Justin & Simon, Elena & myself formed different pairs. The first half an hour was very irritating. The app was close to 19 MB and the network speed was too slow compared to the usual speed. Close to 50-60 testers were accessing the same link from the same hall! I and Elena had USB and we got the app downloaded on them and passed to others in our team. By that time, Michael had already downloaded the app. Then we started to play around, learn what the whole app is about, read the document which came along with the installer. Some clicks here & there, there were few Access Violation errors displayed.
06:45 PM - Calm planning amidst chaos
Markus takes the lead and brings some calm amidst the chaos. Justin points that instead of everyone testing in their own way, we could have a plan of attack. Markus brings the flip charts to play. After our initial testing, we set ourselves a time limit of 20 minutes of testing sessions followed by 5 minutes of de-brief. The first few sessions were interesting - everyone took different features and going was smooth. The bugs were found and it was time to log them.
07:30 PM - A bit disconnected but focussed on testing
I can't speak for other pairs but our pair (Elena & myself) found some cool Access Violation errors. By this time, we found 3-4 errors. They were not easily reproducible but popped up now & then. After spending close to 40 minutes, we wanted to nail the errors. We also went to the developer who was also in the same room and asked him if he needed exact steps or just the error logs were sufficient. He was happy wit the error logs and the screenshots.
08:45 PM: Bug reporting in full flow
By this time, we had captured most of the bugs and we decided to log them. We had to upload the screenshots, video and the error logs. All these had to be zipped and uploaded. We tried uploading them to the specified location but it failed twice. Sometimes it displayed 90% uploading and then failed. Then I thought of uploading them to my most trusted friend - Dropbox. All the zip files were dropped onto my account and the link was shared. We felt happy that we overcame the link barrier.
09:30 PM: Test Report & Bug Hunting
Matt, Markus, Justin & Adam were busy with preparing the final test report. Matt asked me to report the bugs on the typewith.me document too from which he would consolidate all the bugs in the final report. We were busy logging bugs and did not update the document. In fact when the team was busy adding sections to the final report, Elena & I were still hunting bugs and reporting them.
What did I learn:
This team rocked! Everyone took a considerable amount of tasks out of the final list of tasks and contributed equally to the final test report. The testing coverage in terms of the different features model was also very good. During the de-brief, each pair picked up something new. Sometimes, if they felt that a particular area needed more testing, they continued. The approach was fantastic - 20 minutes of testing followed by 5 minutes of debrief was very effective. With such a big application to test, that was a very good approach to gain coverage and reduce redundant testing.
I liked the way Markus used flip charts to record during de-brief. The flip charts helped a great deal at the end as we were the only team with good flip charts highlighting test coverage. Those also helped in the preparation of the final test report. I liked how Matt organized the final test report. He gave us simple instructions - update the typewith.me and he would take over. I liked how Michael picked up the areas left by others for further investigation and filled the gaps. Markus was too good organizing the entire de-brief. Justin was cool with the mind map. Simon, Adam helped with the final test report. Elena was very helpful in paired testing especially the bug investigation for the Access Violation errors. She helped me use Jing (which I had never used before). Me & Elena were also talking with the developer asking if some of the bugs we found were useful for him.
The four hours were very exhausting. It was like an extended weekendtesting session. Being used to the weekendtesting sessions, I was kind of used of competing in these kinds of hectic testing environments. I learnt a lot about team co-ordination, organizing testing & de-brief sessions. I learnt how to prepare a test report for such a project quickly.
And finally, the next day - Matthew Heusser promoted Elena Houser, Michael Larsen and myself to the next level - Black Belt in the Miagi-Do School of Software Testing. That was a proud moment. I am waiting for the next testing sessions to demonstrate the learnings from this tester competition.
On Day 1 - Aug 08th evening, there was a testing competition for four hours from 06:00 PM to 10:00 PM US time. After the awesome sessions the first day, it was time for some fun. And how do testers have fun? Testers have fun by competing in a testing competition.
As I belong to the Miagi-Do School, all the students of Miagi-Do formed one team. There were few other testers who were not part of the school also joining us. So, the team members were: Matthew Heusser, Markus Gaertner, Michael Larsen, Adam Yuret, Elena Houser, Justin Hunter, Simon Schrijver and myself. We were never in chance of winning the cash prize of $1401 as Matthew Heusser was one of the organizers. And still we wanted to participate in this contest. We wanted to test as the Miagi-Do school team!
06:00 PM - Setting up and Learning the Application
As we had more Mac machines than Windows and the application to test was a Windows app, we paired. Michael & Matt, Markus & Adam, Justin & Simon, Elena & myself formed different pairs. The first half an hour was very irritating. The app was close to 19 MB and the network speed was too slow compared to the usual speed. Close to 50-60 testers were accessing the same link from the same hall! I and Elena had USB and we got the app downloaded on them and passed to others in our team. By that time, Michael had already downloaded the app. Then we started to play around, learn what the whole app is about, read the document which came along with the installer. Some clicks here & there, there were few Access Violation errors displayed.
06:45 PM - Calm planning amidst chaos
Markus takes the lead and brings some calm amidst the chaos. Justin points that instead of everyone testing in their own way, we could have a plan of attack. Markus brings the flip charts to play. After our initial testing, we set ourselves a time limit of 20 minutes of testing sessions followed by 5 minutes of de-brief. The first few sessions were interesting - everyone took different features and going was smooth. The bugs were found and it was time to log them.
07:30 PM - A bit disconnected but focussed on testing
I can't speak for other pairs but our pair (Elena & myself) found some cool Access Violation errors. By this time, we found 3-4 errors. They were not easily reproducible but popped up now & then. After spending close to 40 minutes, we wanted to nail the errors. We also went to the developer who was also in the same room and asked him if he needed exact steps or just the error logs were sufficient. He was happy wit the error logs and the screenshots.
08:45 PM: Bug reporting in full flow
By this time, we had captured most of the bugs and we decided to log them. We had to upload the screenshots, video and the error logs. All these had to be zipped and uploaded. We tried uploading them to the specified location but it failed twice. Sometimes it displayed 90% uploading and then failed. Then I thought of uploading them to my most trusted friend - Dropbox. All the zip files were dropped onto my account and the link was shared. We felt happy that we overcame the link barrier.
09:30 PM: Test Report & Bug Hunting
Matt, Markus, Justin & Adam were busy with preparing the final test report. Matt asked me to report the bugs on the typewith.me document too from which he would consolidate all the bugs in the final report. We were busy logging bugs and did not update the document. In fact when the team was busy adding sections to the final report, Elena & I were still hunting bugs and reporting them.
What did I learn:
This team rocked! Everyone took a considerable amount of tasks out of the final list of tasks and contributed equally to the final test report. The testing coverage in terms of the different features model was also very good. During the de-brief, each pair picked up something new. Sometimes, if they felt that a particular area needed more testing, they continued. The approach was fantastic - 20 minutes of testing followed by 5 minutes of debrief was very effective. With such a big application to test, that was a very good approach to gain coverage and reduce redundant testing.
I liked the way Markus used flip charts to record during de-brief. The flip charts helped a great deal at the end as we were the only team with good flip charts highlighting test coverage. Those also helped in the preparation of the final test report. I liked how Matt organized the final test report. He gave us simple instructions - update the typewith.me and he would take over. I liked how Michael picked up the areas left by others for further investigation and filled the gaps. Markus was too good organizing the entire de-brief. Justin was cool with the mind map. Simon, Adam helped with the final test report. Elena was very helpful in paired testing especially the bug investigation for the Access Violation errors. She helped me use Jing (which I had never used before). Me & Elena were also talking with the developer asking if some of the bugs we found were useful for him.
The four hours were very exhausting. It was like an extended weekendtesting session. Being used to the weekendtesting sessions, I was kind of used of competing in these kinds of hectic testing environments. I learnt a lot about team co-ordination, organizing testing & de-brief sessions. I learnt how to prepare a test report for such a project quickly.
And finally, the next day - Matthew Heusser promoted Elena Houser, Michael Larsen and myself to the next level - Black Belt in the Miagi-Do School of Software Testing. That was a proud moment. I am waiting for the next testing sessions to demonstrate the learnings from this tester competition.
3 comments:
Congrats !!!
Well it sounds like fun.
with so many efficient and established software testers, your learning curve would map exponentially, seems learning is fun.
cheers,
Deeps
sounds like fun! maybe more companies should have a bug-searching competition for every major version released and the prize should be a raise :)
Post a Comment