What is the difference between Smoke test and Sanity test

As testers, we ask a lot of questions. Sometimes, we need to answer a lot of questions too. Few days ago, I was asked the question:

"What is smoke and sanity testing and how is it connected with high level test cases?"

I asked why he wanted to know the answer and pasted the link to blog post by Michael Bolton:
http://www.developsense.com/blog/2011/11/smoke-testing-vs-sanity-testing/

I told them that it is much more important to practice testing and improve one's testing skills than know answers to such questions. When I mentioned about BBST, I was asked if BBST meant Behavior Based Software Testing. I was disappointed and emailed the tester my first book - 'What If... A collection of tips on software testing'. To my surprise, the tester had already bought this book. I was even more disappointed.

The next day, I received this email from the tester:

Since I have read this book before and at that time, I have just read it like other testing guides and have joined people just to ask my testing queries those were not for testing but was for interview preparations.

But after your advice , I have started reading this book again. This time I am feeling like I have started learning the basics of testing  very first time, since the time you have just sent the book to read. You would be amazed sir, that since evening to this time I have just read only 6 pages of it and this time I have not skipped any links or any thing.

Now I am feeling like this book is the best gift given by anyone and once more thing initially i was using it like quick recap but today i feel that it is more like an encyclopedia of testing basics.

Sir, I would request you to just help me in getting to draw the Mindmap and also with basics of drawing these mindmaps (But at least this week i wouldn't get time for  this, not because i have a lot of work in office but I have your book to complete it with all its link with better understanding).

I am happy that he has found the book useful. Balaji from ZenQA, Hyderabad also finds the book useful. When I met him last time, he told me that he uses the book as a compulsory reading for the new joinees. 

Let us stop worrying about what Sanity or Smoke tests mean. Let us focus on our skills instead!!!

PS: I have titled this post as "What is the difference between Smoke test and Sanity test" to attract more hits. I hope more and more testers read this post and the post by Michael Bolton.

Leia Mais…

Release of my 4th book on software testing

I have released a book each on my mother's birthday followed by my sister's birthday and my birthday. Obviously, my father was feeling left out and I am releasing a book on his birthday. This is my fourth book on software testing.

Book 1: What If... A question every software tester must ask.

When I logged my first bug, I thought – ‘What if’ this bug was found after release? Years passed, many products were released, and I gained a lot of varied experiences.  I made a few embarrassing mistakes too. There were few instances where I wished that someone had warned me beforehand. So, I started preparing a book of tips targeted at software testers. Special care has been taken to keep each of the 22 chapters short and to the point. Emphasis is on ready-to-use tips which would give you instant results.

Book 2: What If... 50+ tips to win testing contests. 

This book is a collection of tips which might help any tester competing in a testing contest. Testers are under tremendous time pressure and the competition is tough. Skilled testers have a better chance of winning the contests. After participating in a number of testing contests, I realized that it is easy to win any contest if you dedicate some time and demonstrate the right skills. In this book, I have tried to highlight few points which will improve your chances of winning the testing contest.

Book 3: What If... 50+ tips to boost your productivity.

This book is a small collection of tips, tricks and list of tools to help boost your productivity. This is entirely based on my experiences in software testing as well as using computer. Internet is so powerful. A simple Google search will yield you so many search results. Google for “Screen Capture Tools” and you will find a minimum of ten tools in the first page itself. Which one do you choose? Do you have the time to try each one of them? What about Windows command prompts? There seems to be more than fifty commands. Which one is useful for us, especially for a software tester?

Book 4: What If... 50+ tips to improve tester-programmer relationship

This book brings into picture a very important person - the programmer & the programming team. Each one of us might have the experience of working with at least one tough programmer. Some programmers are very friendly and help us with finding bugs. Some of them are very strict with their deliverables and do not respond to any queries outside office hours. Some hardly talk to you unless you ask them a question. There are different types of programmers and bring in variety to our testing challenges. As I write this book, I have completed over six years of software testing and interacting with multiple programmers across different projects within and outside the company. With a rich experience of working with tough programmers, I write this book to help you.

My special thanks to my family members (for having a gap between the birthdays), my friends for accepting me as I am, my friends on twitter, facebook who keep encouraging my work, the programmers who keep challenging me, those who bought my first three books, those who provided me feedback and those who continue to believe in me :)
And of course, my love and thanks to my father who continues to encourage me in everything I do.

How to buy the books:
Download from bit.ly/booksaj

Leia Mais…

The Tools/Commands/Tips I use - Part 1

Thanks to Jon Bach for the inspiration behind this blog post.
This blog post is an attempt to tell you the different tools I use and how I learnt different tips & tricks useful for testing. Feel free to comment with your choice of tools/tips/tricks.

The first time I switched on a computer was the time I joined a computer course. The class was about DOS and MS-Office. That day I got to know what Ctrl + Alt + Delete did. It was like a magic combination. I did not know anything about what an operating system did. All I knew was that, you needed to press Ctrl + Alt + Delete to unlock the computer. Then I happily played Dave for the next one hour.

After few months, I understood the joke about a software engineer shouting "F1" when in danger. Well, he needed "Help" and the most common key associated with Help was F1. As I typed my college assignment in MS-Word, I found it difficult to edit long sentences. I pressed the left/right arrow keys till the cursor was at the right word and then I edited the word. My friend then showed me the power of Home and End keys.

During college days, me and my friends used to watch movies on friend's computer. Windows Media Player or VLC were the most commonly used tools. I was surprised when Alt + Enter changed the mode to Full Screen. Then I got to know what PrntScr did. I was always hesitant to use those keys - specially the F1 - F12 and the Insert, Pg Up, Pg Dn, Home, End. I did not want something bad to happen as most of the savings went into buying a computer.

Then I joined as a software tester in a company. The first day in training, the application was not responding. It was happening only on my machine. The programmer was called for by my trainer. He came and pressed some keys and Windows Task Manager popped up. I asked him the combination. It was Ctrl + Shift + Esc. That key combination was the first combination I learnt as an employee. Then, he right clicked on the Application name, Go to Process and clicked 'End Process'. This was how he force closed the application.

I spent more time observing the programmers. I did not know how to check the version of any application other than using the About/Help menu. The programmer taught me to check using the Add/Remove Programs window to note any application version. When I made a note of the flow to get to Add/Remove Programs from Start > Control Panel, he immediately showed me an easier way. He asked me to try 'Run > appwiz.cpl'. When my eyes asked the question "What?", he told me that appwiz = Application Wizard and cpl = Control Panel. I smiled and thanked him. My other friend taught me what Run > temp and Run > %temp% did.

Then I started using more tools. I used CCleaner. I was surprised by how much space it cleared for me by deleting the temp files and fixing some registry related files. The amount of free space increased. This tool was specially useful to me who never cleaned the temporary files. It cleans up your computer in minutes. As a tester, I have to take many screenshots. My friend suggested Greenshot and it was quite good. I used it for some time before I found Jing. I think it was Elena Houser who introduced me to Jing at CAST 2011. Yes, it was Elena. We used it for the testing competition. I continue to use it.

As I tested more and more applications, I found many application crashes. Then my programmer friend taught me how to use DrWatson. He was happy that I attached the logs and I was happy that I learnt something new. In one of the applications I tested, I found that the logo appeared to be a bit different from the one on the company website. When I showed that to the programmer, he attributed that to my machine display settings. I was not convinced. I searched for a tool which will let you know the RGB values of the color and also the hex code. I used ColorCop and proved to the programmer that there was indeed a difference.

After attending Rapid Software Testing (RST) workshop, I started using Perlclip a lot more. It was easy to generate lots and lots of text very quickly. Some of the programmers were impressed and the testers surprised. :) I started attaching video files to my bug reports. WebEx Recorder proved to be a very good tool for this task. The only drawback was that you could play the recording only using WebEx Player as the output file format was .wrf . So, I started using Jing to record the videos too.

I started registering on many social networks. The number of open applications too increased. While I knew that Alt + Tab switched between applications, I faced a lot of difficulty when the number of tabs in my browser increased. A browser is a single application and Alt + Tab did not help. Accidentally, I discovered that Ctrl + Tab switched tabs on a single browser instance. It was all good until Ctrl + x opened the xth tab where x was a number. Example: If you had ten tabs open, Ctrl + 2 opened the second tab. Ctrl + 9 always opened the last open tab.  Well, it doesn't open the respective tab, it just brings the focus onto the tab.

There were instances where I used Ctrl + to zoom in and Ctrl - to zoom out on a browser window. But I did not know how to view the original size through any shortcut. Ctrl + 0 seemed to be the shortcut.
I received a lot of emails with broken hyperlinks and I had to manually copy the entire link text and paste on the browser. Then, my friend showed me that using "" and typing the text between the quotes and pressing enter would turn the entire text into a hyperlink. This helps avoid broken links specially for network paths. Try it on Outlook.

There was an informal group in my company started by me and my friend. We called ourselves the Mission group. There was no hierarchy and we shared useful information about testing via emails. As we shared lot of article links, we used bit.ly a lot to shorten the url. I used to take a printout of the article and write just the 6 characters of the bit.ly link at the top of the page. We used typewith.me a lot to take simultaneous testing notes sitting at different computers.

Sometimes, the visible screen on the browser was not enough to highlight the problem. I used F11 to view full screen and then take screenshot. I have also used the website PDFmyURL to convert the entire page to a pdf file.

And for mind map lovers:

Don't forget to comment with your choice of tools/tips/tricks. Thanks.

To be continued...

Leia Mais…

Release of my ebook: "What If... : 50+ tips to win testing contests"

October 16th, I released my first ebook titled - 'What If...'. It is a collection of tips on software testing. Thanks to everyone for your wishes, more than fifty copies were sold.

This sunday - Nov 20th, I am releasing my second ebook titled -

"What If... : 50+ tips to win testing contests"

Contents page

What is this book about?
This book is a collection of tips which might help any tester competing in a testing contest. Testers are under tremendous time pressure and the competition is tough. Skilled testers have a better chance of winning the contests. After participating in a number of testing contests, I realized that it is easy to win any contest if you dedicate some time and demonstrate the right skills. In this book, I have tried to highlight few points which will improve your chances of winning the testing contest.

As seen in the image above, initial chapters focus on test environment, test planning. Then the focus shifts to bugs - finding important bugs, bug reporting and finally learning from the testing contests.

Just like my first book - "What If...", the tips are short and to the point. Focus is on ready-to-use tips. Hope you enjoy reading the book and win many more contests.

Download from bit.ly/booksaj

Special Thanks to
Praveen Singh - Director of www.99tests.com for founding 99tests. It is an excellent platform to participate in testing contests and learn from other testers. I learn from every contest at 99tests.

Friends who helped me review the book:
It is a tough task to review any book. Alek, Bernice and Elena helped me by reviewing the entire book. Thanks to them, I am able to release the book on time.

Hope this book helps you win at 99tests and many other testing contests.

Leia Mais…

Dropbox: I love you

Being a tester, I like to test many applications. This exercise also helps me in choosing a product for the weekendtesting sessions. Over the last two years, I might have installed more than a hundred applications. I would download, install, test, retain it if I like it or uninstall after few days.

There are some applications/tools which have taken a permanent place in my toolkit. Today's post is about one such application and it is Dropbox.

What is Dropbox and how is it useful?
It is a free web based file hosting service. Once you install Dropbox on any computer and save files in the Dropbox folders, they automatically are backed up on Dropbox website. If you want to access the same files on some other machine, install Dropbox on that machine. If the Dropbox service is running on both the machines, any files you modify on one computer gets automatically synced and saved on both the computers and on the website too. You no longer have to copy folders/files from one computer to another. Let Dropbox do its job and you concentrate on the files.

Usefulness:

• Backup feature: Every file you edit is saved on the Dropbox website too. 
• Online availability: Forget your test machine, personal machine and all the different machines you use to test. Be assured that your files are always available online.
• Freedom from emails, USBs, CDs: Are you in the habit of emailing yourself copies of the document every time you edit and switch to a different computer? Do you carry a device with the data everywhere you go? Dropbox solves it. 
• Sharing feature: Do you share files with your friends a lot? You don't want to spend on the upload/download charges and make it a simple process to share? According to me, Dropbox solves it in a matter of few clicks.
• Accidental Deletion: Did you delete the files by mistake and work on them from scratch? Dropbox saves even the deleted files and you can RECOVER them easily!
• Free upto 8GB space! When you install, you get 2GB free space and when you refer friends (share link) and they install, you get 250MB/referral. 

What are you waiting for? 

Click on the link http://db.tt/xcAYzF1 , install Dropbox and enjoy the benefits! 

Post your questions about Dropbox as comments to this post and I will answer them for you. 

If you are someone like me who can't live without Dropbox, feel free to share this post or comment on how Dropbox helps you. 

Image Credits: https://www.dropbox.com/tour#4

Leia Mais…

WeekendTesting & its benefits to a fresher

Q: I am a fresher. I want to improve my testing skills. How will Weekend Testing help me?
A: Welcome to Weekend Testing (WT). I am happy for you for two reasons:
You are aware of WT & WT is one of the good platforms to improve your knowledge.

Let me highlight some of the ways WT can benefit you:


1. An excellent platform to test your skills.
WT has this brilliant idea of coming with a unique mission every session. Though there are chapters in India, Europe, Australia-New Zealand, Americas and European WeekNight Testing as of today, every chapter makes sure that no mission-product combination is repeated. Every session is a totally different experience and what's cool is you are free to participate in any of the sessions - No chapter restrictions.

1. Varied experience
If you have no experience of working in any organization, I cannot think of any other easy way to face such a diverse set of products & missions. Just imagine, after every weekend you would have tested 2-3 products and at least one would interest you to explore more. Last week, I learnt about Rapid Reporter, Text2MindMap and Process Monitor. Worst case, you might learn about 1 tool every weekend. Isn't that cool? Different tools, different experience added to your ammunition list.

1. Excellent repository of experience reports
If you are not satisfied by the sessions and want more, the WT website - www.weekendtesting.com must act as a huge repository of excellent experience reports. We are open for scrutiny. You can go through each report, chat transcript at your own pace and ask us questions. We are ready to help you. You might have an idea which no one of us thought of. We are waiting to hear that idea.
WT is all about Test, Learn and Contribute.

1. Cost Vs Value
We ask for two hours of your time and nothing else. Two hours of pure learning, interactions with other testers, new products, new friends, new test ideas. Think about the value you get for just two hours of investment. Cost Vs Value. WT is aware that you are dedicating your valuable time and hence tries its best to give you a learning platform. Make use of it.

1. Be part of the community
Join us on twitter, create your own blog, add us on Gmail, Skype. Lets talk. So many WT sessions are discussed even after the sessions on different forums, blogs and on twitter. Take part in those discussions. You might learn something new. Build your reputation, demonstrate your testing skills and be part of the testing community. Your next organization's QA lead might be someone whom you paired with in a weekendtesting session!

Looking forward to your participation in weekendtesting sessions.

Leia Mais…

No testcases for the scenario. What do I do?

One of my seniors is a big fan of Silk, Selenium, Loadrunner, TestComplete and any other commercial tool available in the market. He is very good at automating few tests depending on the product. My team lead had no tasks for me. So, he asked me to learn Selenium from the senior. My senior demonstrated some basic commands of Selenium and asked me to play for few hours. Now the next question was: 'Which application to use?'

There were two choices presented to me:
a. An application yet to be released to the market and still being tested.
b. An application released to the market after being tested by the team reporting to the senior.

I chose the second application. My senior told me that I'd not find any bugs as it was tested by his team. I took this as a challenge and explored the application.
In twenty five minutes, I found two inconsistencies. The severity of the issues was high too. With a big smile on my face, I called my senior and showed him the two issues. He immediately called the tester who tested the feature.

On being asked how the issues were not found, the answer given by the tester shocked me. His answer was: 'We do not have test cases for those scenarios'

Few strange lessons I learnt:

1. Some testers fail to test beyond the testcases. Should I call that testing?
2. Testcases give a false sense of security to some management.
3. Forcing a tester to learn any automation tool might not be a good idea in the long run.
4. A new pair of eyes - A new test idea - might lead to discovery of a new issue.
5. Its not a good idea to criticize or interrogate your team member in public.
6. I took it as a challenge to find bugs. Did the challenge attitude help me find those issues or the challenge had no effect?

What would be your answers to the question:
'Why did you miss those issues?'

Did I tell you that the tester added the two test ideas as testcases later?

Leia Mais…

Three Years: Worth the wait?

Five Lessons Learnt from my last Testing project

1. Every project teaches you something new if you are ready to learn.
I tested a web based application for the first time. Introduced to terms like IISRESET, HOSTS, PROXY SERVER, RAD GRID, AJAX, HTTPS, DOMAIN NAME, DATABASE SERVER.

2. A tester's role is to meet the mission.
I found 73 bugs in 6 hours and had to close 69 bugs because they were not at all important from the customer's perspective.

Previous Project: Found 710 issues and 685 were fixed. Everyone appreciated me.
This project: Found 73 issues and 4 were fixed. Everyone asked me to concentrate on features which the customer would use.

3. Test on the expected environment.
Tested for two weeks on an environment which was not the environment at the customer end. Now I feel, why did we waste those two weeks?

4. Test the environment first.
Believed a technical person for setting up the environment and the environment was wrong. First build to customer and it failed miserably. Tested the environment and found that a simple mistake meant that we tested on the wrong environment for two months.

5. Work as one team towards one goal.
It is good to interact with programmers, product managers, tech support, sales manager, QA head, Development Head. Everyone has something new(read different) to contribute.

WOW, I'm very happy that after three years of testing in office, this is my first product release to market.

I'd say worth the wait for three years.

So, when's the next release BOSS? I'm waiting :)

Leia Mais…

A five minute conversation with a programmer

This is a post highlighting the conversation I had with one of my programmers.


Programmer (P): Hi Ajay! I need your help in reproducing the defect #abcdef

Defect #abcdef:
Step 1: ...
Step 2: Enter a value 50 in the text field.
Step3: ...

Ajay (A): Sure, How can I help you?

P: I'm unable to reproduce the defect.

A: Which OS are you trying it on? I had logged it on Win 2003.
P: Yes, I know that. It would be easier if I could reproduce that on Win XP.
A: (he he smiles) OK, I'll reproduce the defect first on Win 2003 and then we could try on Win XP also.
P: OK, great.

A: Step1, Step2, Step3 and here it is, REPRODUCIBLE!!!
P: Ok, let me try it. Step1, Step2 and Step3 and ??? Where's the defect?
A: Oh!!! Let me try again. Step1, Step2 and Step3 and again REPRODUCIBLE!!!
P: (Smiles) Step1, Step2 and Step3 : NOT REPRODUCIBLE

Silence for few seconds.

A: (thinking what could be different) Hmmm, maybe the speed with which I execute the steps is different from your speed of execution.
P: Maybe. Very little chance of that happening.
A: OK, let me try consecutive times. Step1, 2, 3: REPRODUCIBLE. Step1, 2, 3: REPRODUCIBLE.
P: Oh!!! How do YOU reproduce that Man!!! See, Step1, 2, 3: NOT REPRODUCIBLE.

P: Maybe you are pressing 'Enter' after entering the number.
A: Hmmm, I'm not pressing Enter key.
P: Then maybe single click causes this problem. Or maybe double click to select the field before entering the number.

A: (thinking WOW, so many factors!!! Let him go on)
P: Or this might happen if you select the text by 'Click and Drag' using mouse
P: OK, I'll look into this issue. Thanks for your time.
---------- END OF CONVERSATION ---------

This particular conversation refreshed my memory of how many different factors affect a single entry in a text field.

> Operating System
> Response time
> If the focus is on the field or not
> After entering the number, did the user press Enter or Tab?
> Did the user double click on the field to select the default text?
> Did the user delete the text before typing in the new text?
> Did he press 'Delete' or 'Backspace'

I'm sure there are many more factors related to the single entry in the text field.

The point I want to highlight here is "How useful is it to have a conversation with a programmer about the product?"

In my case, it was useful. Have you experienced such an interaction?
Do let me know.

Please feel free to share such experiences(Good & Bad).

Leia Mais…

Simple Things Do Matter :)

I had to check a fix provided by a programmer to solve an issue.

I had to replace two files- One an .exe and the other .ini file.

Once I replaced the files, changed the contents of the files to suit my test and went through the steps to reproduce the issue, the issue was still reproducible.

So, I sent an email to the programmer that the issue is not resolved and the fix does not solve the issue. He wanted to have a look at my machine and try the scenario once.

Before handing over the control to the programmer, I copied the two files which were part of the test. This way I can compare the files after the programmer has completed his test.

The programmer was in Germany and was accessing the machine. I could not see what actions he was carrying out on the machine.

After ten minutes, I got a reply to that email that the issue is solved. Please re-test to confirm.

I was surprised and then along with my manager wanted to carry out the test. Before conducting the test, I wanted to compare the current file(after programmer conducted the test) and the file used by me to conduct the initial test.

And surprisingly, the contents were slightly different.

The initial file had 'Timer: 5:00:00 PM' and the new file had 'Time: 5:00:00 PM'.

Also, the programmer asked me to include an additional step so that the issue is resolved.

The additional step was to exit the service and run the .exe so that it may re-read the contents of the modified .ini file. I had not run the .exe after modifying the .ini file.

I replied back to him that the contents were changed and also the additional step was missed. These were the two reasons why I was able to reproduce the issue and he wasn't.

Learnings:

1. It is always safe to have a backup of files before and after conducting the test. Again it depends on what value this step adds to the overall mission. In my case, If I had not compared with the original file, there was very little chance that this issue would have been fixed before the customer found it.

2. Programmers have a natural tendency to fix issues on the fly. It is by nature that they fix issues and they do not consider it important to inform the tester about the changes. In my case, it turned out to be 100% true. The programmer never informed me about the change of text to 'Time' from 'Timer'.

3. I must have paid a bit more attention and come to the conclusion that the .exe must have been run to re-read from the .ini file. This taught me to relate the interaction between files & application to the tests I conduct using those files.

Simple things like having a backup of important files helped me find an important issue before it knocked the customer's door :)

Leia Mais…

Base state of a software application

This is a post on the importance of base state or initial state of a software application.
In the last post, I had mentioned about activating mobile office on my mobile.

I had not taken any note of the condition of my mobile connection settings. After I deactivated my mobile office, I was unable to access any websites on my mobile. After a lot of trial and error, setting up some other settings, I decided to approach the customer care and ask them to solve my problem. Finally I got it solved.

Observations:
> I was careless not to note the initial settings of the connection.
> I should have gathered more information about the new service(mobile office) as it did not support my access to wap enabled websites.

Connection with testing:
How many times have we come across a situation where we could not reproduce a defect or a behavior due to the change in the initial settings. We waste a lot of time reproducing the issue and finally realize that it was environment specific and had to ghost the system and start afresh. After this mobile experience, I realized the importance of noting the settings of my mobile. In testing, an environment of a test plays an equal role as the test itself.

Lessons learnt:
A tester has to know each of his test case/input. We never know which test may give the unwanted result/behavior.
How about running a test in our mind, imagining the probable consequences before running it in real. If testing was that simple, anyone could do testing.

Sometimes, we do need to probe a defect further to unearth even more critical defects. At the same time, we have to be careful to note the initial/base state of the application.

So, how do you deal with the ever changing state of a software?
Do you know each test you do?
How do you handle the issues you fail to reproduce at the first attempt?

Share your comments. And finally a question to you:
Is it necessary to know the initial/base state of the software under test?

Happy testing :)
-Ajay

Leia Mais…