What Skills Performance Testers Need and How to Get Them? By Andre Vondran

From time to time I see questions on different forums asking what skills are necessary for performance testers.
There were pretty interesting discussions. Looks like most experts agree that performance testing requires more skills than just knowledge about how to create a script for a particular load testing tool.
While it is still possible to imagine a performance tester in a large corporation with deep specialization who only creates scripts and mechanically runs them while other performance experts monitor the system and analyze results, I don't see many perspectives neither for this person, nor for the approach.
Systems become so complicated now that the sum of specialized expert views doesn't give the whole performance picture.

Thinking about skills necessary for performance testing I'd specify the following areas in addition to load testing per se:

• What is going on with the system? Monitoring and Performance Analysis.
• You got results. What if …? Modeling and Capacity Planning.
• We see the bottleneck. What to do? Tuning and System Performance Engineering.
• And write, present, communicate, organize all the time.

You probably need to know about tuning and have understanding of how applications should be designed to perform well (Software Performance Engineering). You don't need to be an expert, for example, in database tuning – most companies have DBAs for that – but you need to speak to a DBA in his language to coordinate efforts effectively. Or raise concerns about performance consequences of the current application design. Unfortunately it is not easy – you need to know enough to understand what is going on and communicate effectively.

The question is how to get such skills. Constant self-learning and gradually getting experience? Yes, of course, but that takes a lot of time. Moreover, many areas are pretty hard to jump into from a scratch.
You need to get some basic understanding before you will be comfortable enough to learn further yourself. Go to a class? Definitely for a performance testing class and your main tool. But for many other different products you are working with? Usually there are several week-long performance-related classes for each product. They are developed for specialists making a living tuning this particular product. You don't have time to go to all classes and usually you don't need to dive so deep. Talk to an expert? Sure, if you find one around. Performance experts are sparse and busy, so you'd better have well-prepared question(s) – hard to do if you know little about the subject.

When you get well enough along the road you get into another trap. You already know enough that basic training won't be beneficial. Still there are almost no advanced classes at all for performance testers – when you go beyond basics, details of environments, tools, systems, application, etc. become so different that it makes no sense to make a class for specific combinations.
You know areas where you need more information, you need to verify your approaches and practices against other experts, you need more advanced tips and tricks, and you need to find somebody you may discuss your problems with.

I believe that a good conference is a solution in both cases. Somebody digests information and presents it back to you. Not that it is an absolute ideal – quality of presentation and presenters vary – but probably the most effective way when you need to jump into many different topics. At the moment I am aware about one open and practical conference covering all performance-related aspects: Computer Measurement Group (CMG). With over 150 sessions related to performance, it allows everybody to find something interesting.

Let's look at the coming CMG conference from a performance tester point of view. For Monitoring and Performance Analysis, you have five half-day classes under the CMG-T umbrella - real classes starting from the beginning and giving what you actually need to know. From leading experts in the field: one generic class, one class per major platform (Windows, Unix/Linux, mainframe), and one for network. Another class reviews what you need to know from statistics to be comfortable in the performance field:

• Statistics for Performance Analysis & Capacity Planning by Ray Wicks, IBM

Plus there are many presentation addressing specific environments or issues. A large variety of topics is covered from:

• Java Performance Analysis 401 by Peter Johnson, Unisys

• Windows Server 2008 Performance, Scalability and Tools by Bradley M. Waters, Microsoft

to very specific topics like Capacity Planning is a core area of CMG so there are many presentations investigating the subject in detail. Starting with the first part of the keynote presentation:

• Is Capacity Planning Still Relevant? by Jerred Ruble, TeamQuest Corporation

A caution here is that many modeling and capacity planning presentations are pretty advanced for an unprepared listener. While I strongly believe that a performance tester must understand the basic concepts of modeling and capacity planning (to be able to do "back of the envelope" calculations), serious mathematical modeling may be pretty complicated and is rather optional for performance testers.

Another CMG focus track is Software Performance Engineering (SPE). There is a great tutorial to start from:

• Software Performance Engineering: A Tutorial Introduction by Dr. Lloyd G. Williams, PerfX and Connie U. Smith, Performance Engineering Services

The following presentations are directly related to performance testing:

• How to Select Significant Workloads in Performance Models by Paolo Cremonesi, Politecnico di Milano

• Multiple Dimensions of Performance Requirements by Alexander Podelko, Oracle

One more great type of presentation involves overviews of different technologies for "performance" professionals: some insights that you need to know and probably won't find in marketing materials. There are so many new buzzwords that you can not go to training for each of them (and shouldn't - usually at least 95% of what is included in training is irrelevant to performance testing). One hour presentation discussing what this new thing may mean for you can give you a pretty good overview, good enough for what you are doing.

André Vondran
SQA Engineer
www.aqasol.com.ar