WOPR15

I recently attended WOPR15 (WOPR = Workshop on Performance and Reliability, http://performance-workshop.org). The theme of this workshop was “Building Performance Test Scenarios with a Context-Driven Approach.” It’s amazing what you learn when a couple dozen or so bright people get together to share experiences and vigorously discuss and debate ideas. This blog will fall woefully short of portraying the experience effectively, much like looking at a picture of the Grand Canyon, but here goes...

I have to applaud the content owner, Michael Bonar, and the WOPR organizers on selecting a line-up of presenters that came at this challenging theme from a variety of creative, sometimes non-intuitive, angles. This made it even more interesting to see the emergence of patterns, or sub-themes, across these disparate stories.

From my perspective, the most compelling of these sub-themes was “Testing in Production.” Companies like FaceBook, eBay, Google and Microsoft support operations of such massive scale that it is not feasible, or cost effective, to support production-scale test environments. Instead, they invest in processes and tools that allow them to use their production systems for testing.

Another prevalent sub-theme was “Just get started.” There seems to be a consensus among these leading practitioners that waiting for all questions to be answered, or all forms to be filled out, is a trap (frequently self-imposed) that hand-cuffs testers and puts them under extreme pressure later. Start exercising the system as soon as you can. Maximize your ability to plan, design, test and report concurrently.

Yet another significant theme touched upon by several of the presenters was “Have a plan, but adjust aggressively.” Frequently something happens in the midst of a testing effort that will lead you away from the plan if you pursue it. Good! That’s why we test! We are finding potential problems – those problems don’t always respect your plans.

Some other nuggets:

• Luck and intuition may play a role in the success of your testing effort, but an exploratory approach, executed by skilled testers and test managers, positions you to harvest that luck and intuition. (My take on Jon Bach’s experience report (ER)) 
• Great testing can become a selling point for your product. (from Paul Holland’s ER) 
• What could you do if you were not afraid? (A brave attitude toward development and deployment - from Goranka Bjedov’s ER) 
• Focus on wait times and queue lengths when looking for problems, not so much on CPU and memory utilization. (also for Goranka’s ER) 

Like I said before, I cannot do this workshop justice in a short blog entry. There were several other presenters and many more interesting points of discussion. I continue to derive tremendous value from this format. This is focused experience sharing, discussion and debate of our practice at its finest. The next WOPR, which is WOPR16, will be hosted by my employer, Progressive, and I am the content owner. The theme is “The Intersection of Performance and Functional Testing.” A call for proposals will go out in January. The conference dates are April 28 – 30, 2011.