Friday, December 11, 2009

Do you TRUST the fix?

Something interesting happened when I was testing with my team.

One of the team members had logged a critical issue.
It was a bit complicated too. It involved a lot of parameters to consider.

After a week, the issue was fixed and the team member who logged the issue was verifying the fix.

After quite a bit of regression testing, I heard an interesting comment:

"Amazing!!! I'm surprised how could the programmer fix this issue without any side-effects"

Hmmm, it so happens that we expect the defects to be fixed.
And when they are fixed, we start questioning.

It is good to question but questioning everything.... being so skeptical... I don't know.

Maybe it is good some times but not always.

Do you TRUST your programmer and his fix?

Let me know.


Mark Waite said...

I wonder if the governing factor is not the complication of the problem, or the complication of the bug report, but rather complication of the fix. If the bug report is complicated, but the fix is simple, I might be more likely to believe there were no unintended consequences. If the fix is complicated, whether or not the bug report is simple, I would expect unintended consequences to be more likely.

A fix to a complicated problem without side effects seems possible but unlikely. I can certainly imagine cases where a complicated problem had a simple root cause and a simple fix, but those have not been the cases I've seen.

More often (for me) it went the other direction, a "simple fix" had unintended consequences and complications. Complicated fixes are even more likely to have unintended consequences.

Anonymous said...

I totally agree. In this case, the problem was complicated. Many factors had been changed. something which the programmer had termed as 'correct' behavior, had unintentionally been fixed by this fix.
This made me a bit apprehensive about the correctness of the fix.

Ajay Balamurugadas said...

Thanks for your comment.
If the fix is complicated, whether or not the bug report is simple, I would expect unintended consequences to be more likely.

Well, Expecting unintended consequences is good but not trusting even after the regression tests is done, is not that good.

Sometimes, it is good to respect good fixes and skills of the programmer.

sunjeet81 said...
This comment has been removed by the author.
sunjeet81 said...

I would not trust assume anything about any fix...BEFORE i have run my tests

Would let my tests prove/refute the claim made by the developer.

If it passes the tests then there should be a level of confidence
but if the feeling of confidence is still not present , then maybe the tests are not reassuring/focused/extensive/correct enough ?

another aspect to consider is...
If a fix fails/side effect goes undetected... but my tests had passed it then the question needs to be asked WHY , did this happen ?

was it a shortcoming of the tests ?
did the fix not regression tested completely/properly ?
was the focus of testing elsewhere ?
so on...