The Performance and Tuning Doctor by Sturnus

"If I had an hour to solve a problem I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions."

Design Thinking

Those of you who have any exposure to Design Thinking will be aware that before we even look for solutions, we need to make sure that we are solving the 'correct' problem.

To help us do this, we go through an process called re-framing. The principle behind this is that rather than just launching ourselves into the first problem we can think of - we make sure we have indeed analysed the problem fully.

The way we go about it is as follows:

• For any issue, ask yourself the following question: If I solved this problem, what would it do for me? How would the problem be shifted elsewhere? What becomes the new problem I'm trying to solve?
• When you have an answer, ask the question again. What would it do for you if you solved this new problem? Where has the problem now shifted?
• Repeat this cycle until you run out of problems to solve.

How does this help?

By focusing in on a specific or low-level problem, you are limiting the options that you have to solve it. As you move up the chain of problems, the range of possible solutions increases. By expanding the problem out, you have also expanded the solution space.

Let's work through an example:

• Imagine I start with the problem of how to get hold of a ladder.
• If I solved this problem, what would it do for me? Well, it turns out that I need the ladder to climb a tree. So if I solved the problem of how to get hold of a ladder, it would allow me to climb the tree.
• Repeating the process: if I solved the problem of how to climb the tree, what would it now do for me? In this example it would allow me to reach an apple in the tree.
• Reaching the apple would allow me to eat it...
• Which would satisfy my hunger.

By going through this process, we've completely re-framed our problem from "how do I get hold of a ladder?", to "how do I satisfy my hunger?" In the process, we've also expanded the range of possible solutions. We might have started with just two possible alternatives: borrow a ladder from a neighbour, or go to the local hardware shop and buy one. Following the re-framing, we have now have more options: eat some left-overs from the fridge, prepare a meal with ingredients in our kitchen, go to a restaurant, buy a sandwich from a local deli etc., (as well as, of course, our original proposed solution, which was to climb the tree and get an apple). Now that we have increased the number of possible solutions, it should be easier to find one that works for us.

Before I bring this back around to software performance, some of you may be asking why on earth we would need to perform this re-framing in the first place? How could anyone be so silly as to try and solve the wrong problem?

In fact it happens all the time. Often it's because the person who has decided on the problem is not the same person trying to solve it, so they may have already decided on a course of action, and assumed most of the solution. The person solving the problem only gets the last problem to solve, and so has to backtrack the problem back to the actual one.

Other times it may be simply because it is not obvious that separate issues are part of the same problem. We may have to artificially adjust our thinking to spot this.

Performance Analysis

OK, so what does this all have to do with software performance?

Well, it turns out that the same exercise of re-framing can help significantly when it comes to the analysis and improvement of application performance.

Interestingly, before I discovered Design Thinking, I used to advocate a similar technique - calling it "taking a holistic view" or "stepping back and taking a broader perspective". But let's face it, both of these sound a bit rubbish compared to "re-framing" - so let's just stick with that.

How does it work?

Let's ease into performance by using a real-world example. Imagine a factory that produces fizzy drinks - such as bottled cola. There will be various steps in the process: sterilising bottles, mixing up the drinks, filling the bottles, putting the tops on, putting the bottles in boxes etc. If we look at each of theses steps as separate problems to analyse, we will have limited success. But if we now re-frame the problem to "how do I produce 50,000 boxes of bottled fizzy drinks per day?", we now have a greater range of possibilities. In fact, this gives us pipelining : every step in the bottling process is performed in turn, but once (e.g.) the bottles have been filled, they are passed on to the next step, whilst another set of bottles is filled.

Pipelining is a form of parallelism, because at any moment in time each step is happening at the same time, but on a different items. Note that whilst we haven't increased the time taken to produce a box of bottles, we have improved the throughput significantly - the number of boxes that we can produce in a set time.

This may seem like a trivial example, but what may not be quite so obvious is that the CPU running inside your computer or mobile device right now is almost certainly using the same technique of pipelining to improve its performance. The instructions that processors execute also require multiple steps to complete. Just like a factory, a particular step for one instruction can be performed at the same time as a different step for an adjacent instruction. Decades ago someone re-framed (although they might not have called it that at the time) the problem of how to speed up the execution of a single instruction, and changed the problem to speeding up the execution of multiple consecutive instructions instead.

Database Loading

Imagine we are trying to solve the problem of how to load a record into a database faster. We might be able to adjust the indexes to ensure that (a) the correct position can be found quickly, and (b) there aren't too many other indexes that need to be maintained following the insert, but other than some tuning possibilities, this doesn't really give us much to work with. It also means that unless we've done something bad (like missed indexes), we are unlikely to see any dramatic performance improvements.

How can we re-frame this? If we ask ourselves what we would get if we solved this particular problem, we might discover that it would make the loading of the (e.g.) 20,000 records that we are in fact loading, much faster.

But rather than give us the same problem 20,000 times over, we have now opened up our problem space to this: how do I load 20,000 records faster. This now gives us batching : sending multiple queries in the same string. My favourite example of the efficacy of batching is a database load process that ran about 36 times faster after batch sizes of 500 were used.

I should point out that the reason this is my favourite example is because (a) it makes me feel really clever, and (b) it helps when justifying my day rate.

HTTP APIs

Another example is that of mobile apps making requests to, for instance, APIs on a server. If we limit the scope of the problem to simply how do we speed up the processing on the app, we are limiting both the options we have, and the improvements we can make. If we re-frame the problem and look at how we improve the efficiency of the whole user experience - including both the mobile app requests and the server APIs, we have more options; we can potentially make dramatic improvements.

In systems where the client requests and the server APIs have been coordinated and streamlined, the user experience in terms of speed is almost always better than those where the server team has created a beautifully pure set of APIs, but which are cumbersome and slow to use.

Summary

The Design Thinking technique of re-framing is a proven way of altering problems to make them easier to solve. It works in everyday life, and it works in performance analysis. There are many situations where this re-framing exercise has led to dramatic performance improvements.

You may have to adjust your questions slightly to help your re-framing, but when you're analysing the performance of any process or system, try asking yourself the following:

• If I solve this problem, what will that do for me? What's the next problem I'll need to solve? And the next?
• What task did I perform before this one? What will I need to do next? Can I combine all these into one problem scope?
• What tasks am I likely to perform in the next few seconds, minutes, or hours? Can I consider these to be part of the same problem?

Give re-framing a try, and see what it can do for you.

Back to top