New Year's Resolutions
Tuesday, January 04, 2011
By James Whittaker
I know many people who laugh at the concept of resolutions, easily made and easily broken. All true. However, I am a runner now because of a resolution I made about a decade ago and my personality has undergone a successful renovation or two over the years as well. When they stick, resolutions can become habits and the emergence of the occasional butterfly makes them a worthwhile exercise. With the optimism of a new year, I present my Google Testing resolutions for 2011 which I hereby declare the Year of the User.
1. I will listen to users more and developers less.
Developers, by definition, are engineers lost in the details of implementation. When it comes to testing concerns, such implementation details clog a tester's neural pathways with issues that simply should not be relevant. I resolve to take the high road as often as possible and consider user scenarios, integration issues and end-to-end uses of the system above all other concerns. And, yes, that will mean telling developers "sorry, dude, your broken build simply is not my concern."
2. I will push all implementation testing issues to developers.
My first resolution may lead readers to believe that testing implementation details isn't important. Let me be clear. Testing implementation details is important. When they go untested they create enough noise that user-oriented testing is compromised by the constant emergence of silly bugs. Silly bugs mask important ones. Find them at the source: ensure that proper unit testing and automated smoke tests are present and owned by the people most qualified to write and maintain them: developers. I resolve not to be sidetracked by silly bugs but to push back hard on the developers who are happy to write the bug but neglect to write the test for it.
3. I will endeavor to tie together all user-oriented testing.
In the run up to releasing Chrome OS for pilot last year it was clear that many of the bugs found during dogfood (internal testing), crowd-sourced and out-sourced testing had already been found by my test team. Not only is there is a lot of repetitive and wasteful testing being performed, my team isn't getting enough credit for finding these important issues early. I resolve to introduce technology that will allow all testers to share testing tactics and see each other's work, ultimately erasing the boundaries between these common phases and allowing testers who join a project late to build upon the work of those who've been there for a while.
Finally, I resolve to expose more information about how Google tests internally. I am going to return to the conference circuit this year and talk frankly about what we are doing, the good, the bad and the downright embarrassing in the hopes that other testers at other companies do the same. I am also going to push more Google testers to post their experiences on this blog and join me at industry events to discuss these things with anyone struggling with the same issues.
Happy New Year! May it be one that sees a higher level of quality from every corner of our industry.
I know many people who laugh at the concept of resolutions, easily made and easily broken. All true. However, I am a runner now because of a resolution I made about a decade ago and my personality has undergone a successful renovation or two over the years as well. When they stick, resolutions can become habits and the emergence of the occasional butterfly makes them a worthwhile exercise. With the optimism of a new year, I present my Google Testing resolutions for 2011 which I hereby declare the Year of the User.
1. I will listen to users more and developers less.
Developers, by definition, are engineers lost in the details of implementation. When it comes to testing concerns, such implementation details clog a tester's neural pathways with issues that simply should not be relevant. I resolve to take the high road as often as possible and consider user scenarios, integration issues and end-to-end uses of the system above all other concerns. And, yes, that will mean telling developers "sorry, dude, your broken build simply is not my concern."
2. I will push all implementation testing issues to developers.
My first resolution may lead readers to believe that testing implementation details isn't important. Let me be clear. Testing implementation details is important. When they go untested they create enough noise that user-oriented testing is compromised by the constant emergence of silly bugs. Silly bugs mask important ones. Find them at the source: ensure that proper unit testing and automated smoke tests are present and owned by the people most qualified to write and maintain them: developers. I resolve not to be sidetracked by silly bugs but to push back hard on the developers who are happy to write the bug but neglect to write the test for it.
3. I will endeavor to tie together all user-oriented testing.
In the run up to releasing Chrome OS for pilot last year it was clear that many of the bugs found during dogfood (internal testing), crowd-sourced and out-sourced testing had already been found by my test team. Not only is there is a lot of repetitive and wasteful testing being performed, my team isn't getting enough credit for finding these important issues early. I resolve to introduce technology that will allow all testers to share testing tactics and see each other's work, ultimately erasing the boundaries between these common phases and allowing testers who join a project late to build upon the work of those who've been there for a while.
Finally, I resolve to expose more information about how Google tests internally. I am going to return to the conference circuit this year and talk frankly about what we are doing, the good, the bad and the downright embarrassing in the hopes that other testers at other companies do the same. I am also going to push more Google testers to post their experiences on this blog and join me at industry events to discuss these things with anyone struggling with the same issues.
Happy New Year! May it be one that sees a higher level of quality from every corner of our industry.
I didn't find an email, so I will post it here. Did you write "you're" on purpose on the first resolution?
ReplyDeleteI loved this post James and have also forwarded it to my team, you are one of the most outright person.
ReplyDeleteThis year I want to learn more and more, its always like that but this time I will follow a more structured regime. Majority of the times I do follow the 1 and 2 resolutions as mentioned by you but now I want to focus more on the 3 pointer. Please share your methods and approaches as well that you will be taking for minimizing the repetitive and wasteful testing being performed.
I think Google do a better job than most large technology firms when it comes to publicising known issues, although I guess there's always room for improvement.
ReplyDeleteI've not looked at ChromeOS but I've delved a little into the Chrome browser bugs and appreciated the input from the Chrome dev team.
Nice to see your user-focussed approach. I've always considered myself to be a user-centric tester (been doing mostly UAT-related testing throughout the last ten years). User experience can easily get pushed down the priority list when it should be right at the forefront
Getting testers to blog their experiences and collaborate on issues is an excellent resolution.
ReplyDeleteCarl
http://kungfutesting.blogspot.com/
I am really looking forward to read more about your testing practices this year. Great resolution!
ReplyDeleteAlthough you sound a bit like you want to emphasize some sort of disconnect between testers and developers. I am convinced that in healthy teams both roles are equally responsible to deliver quality. So, they want to work closely together, listen to each other, and have a sense for the problems the other role deals with.
I don't want to sound all touchy feely, my experience is just that some us vs. them does not help the user at all.
And yes, drive testing from the user's perspective! I could not agree stronger!
Looks great!
ReplyDeleteI am always wondering why companies like Google, Microsoft try to create the hybrid between tester and dev (SDET).
Why not a splitting the positions, at least in 3: dev that helps only testers, SDET and testers. Of course everyone should have the same skill but in different percentage.
I am a tester that focus on ET a lot so I don't know the implications on managing a high number of test engineers. And of course I don't know why and what is needed. It is my opinion from my perspective.
This is good article
ReplyDelete