The Plague of Amnesia
There are two types of amnesia that plague software testers. We have team amnesia that causes us to forget our prior projects, our prior bugs, tests, failures and so forth. It’s time we developed a collective memory that will help us to stop repeating our mistakes. Every project is not a fresh start, it's only a new target for what is now a more experienced team. The star ship Enterprise keeps a ship's log. A diary that documents its crews’ adventures and that can be consulted for details that might help them out of some current jam. I'm not advocating a diary for test teams, but I do want a mechanism for knowledge retention. The point is that as a team we build on our collective knowledge and success. The longer the Enterprise crews' memory, the better their execution.
Quick, tell me about the last major failure of your teams’ product? Does your team have a collective memory of common bugs? Do you share good tests? If one person writes a test that verifies some functionality does everyone else know this so they can spend their time testing elsewhere? Do problems that break automation get documented so the painstaking analysis to fix them doesn’t have to be repeated? Does the team know what each other is doing so that their testing overlaps as little as possible? Is this accomplished organically with dashboards and ongoing communication or are the only sync points work-stopping, time-wasting meetings? Answer honestly. The first step to recovery is to admit you have a problem.
The second type of memory problem is industry amnesia. When I mentioned Boris Beizer and the pesticide paradox in my last post, how many of you had to look it up? And those who did know about it, how's your AJAX skills? Be honest...there are folks with both a historical perspective and modern skills but they are far too rare. Our knowledge, it seems, isn’t collective, it’s situational. Those who remember Beizer's insights worked in a world where AJAX didn’t exist, those who webspeak without trouble missed a lot of foundational thinking and wisdom. Remembering only what is in front of us is not really memory.
Industry amnesia is a real problem. Think about it this way: That testing problem in front of you right now (insert a problem that you are working on here) has been solved before. Are you testing an operating system? Someone has, many of them. Web app? Yep, that’s been done. AJAX? Client-server? Cloud service? Yes, yes and yes. Chances are that what you are currently doing has been done before. There are some new testing problems but chances are the one in front of you now isn’t one of them. Too bad the collective memory of the industry is so bad or it would be easy to reach out for help.
Let me end this column by pointing my finger inward: How will we (Google) test the newly announced Chrome operating system? How much collective knowledge have we developed from Chrome and Android? How much of what we learned testing Android will help? How much of it will be reusable? How easily will the Chrome and Andriod test teams adapt to this new challenge? Certainly many test problems are ones we've faced before.
Will we remember?
I test Web Applications for a living and the thing that bothers me the most is how hard it is to automate browser testing. Firefox requires you to inject script through a plugin. Safari (on Windows) requires you to special html tags. IE requires you to use the awful COM library. Plus, you have to use 2 different tools to test the Browser components vs. the Web Application. It would be nice to have a browser automation standard.
ReplyDeleteAs far as Chrome is concerned, I would add these to the Chrome questions: How will Google use what it's learned from chrome testing to help others that need to test their web application in Chrome? How will Google make it's Chrome testing reusable by others? How will Google share tidbits gained from testing Chrome to others.
Nice Article.
ReplyDeleteThe ease at which many of us forget is quite scary. Whether it's forgetting or refusing to learn from previous mistakes is another thing all together.
You may find that people are not suffering Amnesia, but due to time / monetary / and staffing constraints things get left out meaning that teams get stung with the same issues over and over again.
Some easy to use tips that we use here some that hopefully others will find useful is WIKI's these are great for jotting down easy to remember information and they are usually opensource so zero cost to the organisation.
BrownBags to share knowledge over lunch hours is another, Again breaking those knowlesge silo's means that everyone benefits even the person hold the BrownBag by comments and questions they never thought of.
I also have a few more over at my site under the Tag of Productivity.
Some of these amnesia affects shouldn't happen.
ReplyDeleteWhatever method or process you use in testing, reporting is vital.
This is important for many reasons, one of the biggest is for the test team to consolidate and distill their findings and recommendations.
It's important that the team can give a good account of what they've done/achieved, what they didn't accomplish and why, issues found with any recommended actions and an overall summary with points to bear in mind for the future and any desirable further action.
It's this group summary that's a very important input to the next team that works with the product/feature. Previous reports alongside new requirements for the next iteration.
The documentation/report can be as lightweight as needed - it's the story that it tells (for whoever does the next iteration or flavour of the product) that's important.
If you're not documenting your work as a test team then it's not complete and you're asking for trouble - or amnesia ;-)
In answer to the final question I can't remember always, hence the need to write and document...
Amnesia is normal.
ReplyDeleteWhen you do a project you may have to solve 1000 hard problems. future projects may have 1 or 2 problems of the same type.
Yes it whould be good to share and information about how to solve those 1000 problems. but it is just to much information.
Solving the same problem in differnt ways until one single solution is part of the programing culture, this is the way, i don't think there is another way arround it.
Completely agree with dfsd. Amnesia is rare illness. When we speak of testing, we should think of one of greatest gifts of human being - ability to forget. It's great in other words that we do not remember exactly how to we tested this or that previous time. Yes, we spend some more on making up new plan for this feature. But it will guarantee that the little train of our testing process do not miss (I hope) this time the station, it missed before. Each project should be something new, fresh, something that inspires you to do your best and not to copy-paste only.
ReplyDeleteSo just be positive :)
Inspiring piece of text I agree with totally.
ReplyDeleteI think the business of software testing only has a relatively short term memory at the moment.
Just go around and ask test consultants what they think of work published by William Hetzel, David Gelperin, Barry Boehm, Thomas McCabe, Boris Beizer, Glenford Myers, Gerald Weinberg, William Elmendorf, Michael Fagan, Edward Miller or William Howden and you will find out soon enough how well the collective memory is functioning.
It was your post, by the way, that inspired a thought process that eventually led me to creating a history of software testing. See my website Testing References for that.