Challenge of dimensions of quality : BWT 8 : Experience Report

20th September 2009, 9pm -11pm IST would be etched in the minds of six testers who got together online to test the ‘Areca Backup’ application. This session marked the eighth session of BWT. It was exactly two months since the concept of ‘Bangalore Weekend Testers’ originated.

List of Participants: Ajay Balamurugadas, Bhargavi, Gunjan, Parimala Shankaraiah, Sudhakar, Vasupratha

Application: Areca Backup

In their own words…
“Areca-Backup is a file backup software that supports incremental, image and delta backup on local drives or FTP servers. Areca-Backup also allows you to browse your backups and navigate among different version of the files contained in your archives.”

All the testers were geared up for the testing session.
The application had been downloaded.

What next?
What about the MISSION?

The mission for this session was as special as the session.

Following mission was given to the testers:
Mission -: You have to choose one of the quality criteria out of the following –
Installability / Usability / Performance / Reliability / Compatibility /Testability.
Choose one quality criteria and stick to that quality criteria for the entire session to test the ‘Areca Backup’ application.

Special THANKS to Pradeep who suggested this mission.

Each tester was very enthusiastic on hearing the mission and started their journey of exploring the product in order to find valuable information.
The testing session started at 09.03pm and lasted till 10.03pm IST.
We had the Discussion session soon after the testing session.

Each tester was supposed to reveal the mission they chose, the approach followed during the testing session. The tester had to highlight any challenges they faced and any specific learning from this session. The individual tester’s experience of the BWT 8 session was the icing on the cake :)

Sudhakar, the first tester to send in his Test Report, started off the discussion session.
He was very clear in his mission:
“Find issues” and he chose Usability as the Quality Criteria.
One interesting thing about his whole approach to test this application was his expectation: “The application should guide the user”

With focus on data validations, Sudhakar was frustrated at the not-so-good validation implemented.
One major challenge apart from the poor validation was the time taken to understand the application. Lack of understanding the product fully prevented him from exploring the product to a greater degree. Finally, Sudhakar felt other than lack of time, the overall experience of participating in this session was good.

We moved on to Vasupratha’s experience.
Vasupratha echoed Sudhakar’s concern about the lack of time. Usability was the quality criteria once again. Vasupratha felt that additional time for testing would have helped in better exploration of the product.

Next turn was Parimala’s.
A different Quality Criteria: Installation was chosen.
The mission set by her was straight forward:
“To test the installability of Areca 7.1.5”
Following an Exploratory approach, Parimala gave a lot of valuable information in terms of bugs. As the number of installation steps was minimal, Parimala did not face a lot of challenges.
At the same time a particular intermittent bug was playing hide and seek with her. :)

Parimala learnt some new scenarios to test once she took up Installation as the criteria. The new learning (New Scenarios) helped her do a round of Scenario Testing.
With this being a good experience, she wanted to do some functional testing in the near future. :)

Gunjan was ready to share her experiences.
Her mission was decided more because of the circumstances than her choice.
Usability was her first choice. The application when launched was giving an error about a missing .dll file. So, Gunjan shifted her focus from Usability to Installability as she had to un-install and re-install the application.

With an exploratory approach to her rescue, Gunjan delved deep into issues in installation and un-installation. Some interesting issues greeted Gunjan even though System restore was also tried to get the application working.
The help file was one of the sources of information from which she tried out the scenarios. Her biggest learning was to ensure system is in correct condition before testing any application.
Gunjan being a first timer to BWT enjoyed herself and found it interesting to think “OUT OF THE BOX”. This was the first time she tested any software out of her office work.

Now, it was the turn of Bhargavi.
Bhargavi’s mission focused on finding problems with Performance as the Quality Criteria.
Following an Exploratory approach, Bhargavi could face many challenges easily.
The major challenge was the difficulty in understanding the features and knowing where to start and how to start modeling the application.
Some bugs pertaining to other quality criteria slowed down Bhargavi’s progress.

She had her share of learning too. As she took the “Performance” quality criteria which she hadn’t tested before, she learnt new ideas to test. This boosted her confidence. Bhargavi enjoyed testing the product with a different perspective – Focus on only one quality criteria.
Her tests forced 100% CPU usage as well as low disk space.

The mission taught Bhargavi to concentrate on particular quality criteria who habitually concentrated on all quality criteria. Parimala added a point as to how testers find interesting and intriguing issues when focus is on a small part of the application.

Finally, it was my turn.
I chose “Performance” as the quality criteria for the simple reason: Never tested for Performance alone before. I too followed an Exploratory approach with my toolkit which consisted of Process Explorer, Windows Task Manager, MS Excel, Notepad, WebEx Recorder and Date and Time Properties Window.

The biggest challenge for me was to learn the product quickly.
Help file helped me to some extent.
Once I understood how to backup, I started with 23GB folder and that was my biggest mistake of the day. :(

Expecting backup software to handle 23GB of data and backup within 15 minutes was very foolish on part of me. Thereby, I spent fifteen minutes watching the progress bar of the backup process.

On trying with a 4MB file, backup process completed within a matter of few seconds.
I glanced through the report which was generated after backup. A bug in the report took away my precious ten minutes.

Biggest learning I had out of this exercise was to prepare test data while the system was being modeled. Also having unrealistic goals(Read 23GB file) does not help the cause.

Later, I tried with 30MB, 60MB and 90MB folder to monitor the performance. But it was almost the end of testing session.
Bharath highlighted the value www.testersdesk.com added in testing the Performance criteria.
Experience was good as it marked the successful completion of two months of Weekend Testing.

Every BWT session gave me a different experience.
Right from BWT 1 where I and Parimala tested www.vischeck.com to BWT 8, every experience is a unique learning and thought provoking experience.

I’d like to thank all the testers, Pradeep and the BWT members: Manoj, Parimala and Sharath for their continuous support and hard work.

Looking forward to BWT 9: A new product, new testers and a new experience.
See you there. :)