Using Metrics to Turn Technical Decisions into Business Decisions

If you knew how a change to your application was going to affect users before you made it you would be making perfect decisions all day long. That level is not quite attainable, but by capturing important metrics you are in a better place to make informed decisions.

On a reactive application I’m currently working on we have asynchronous processes and external services that give rise to emergent behaviour. We also have specific business policies that we must adhere to in the face of this emergent behaviour.

By using metrics to understand some of the emergent properties we are able to help the business achieve greater profits through getting the product right quicker — costing less to develop whilst being available to users sooner.

What first appeared to be technical decisions soon turned into significant business ones once we introduced metrics.Unpredictable Suppliers vs. Mandatory Pricing Strategy

Cheapest offers win at all costs — my client pride themselves on being able to offer users the cheapest deals available. They do this by having partnerships with suppliers. Therefore, when you hit their website you absolutely must see the cheapest available offers based on the search criteria you entered.

Technically this is easy. When a user makes a search request our system could go off to all the suppliers, get the results and send the cheapest ones to the browser. Unfortunately, some suppliers, for some types of search can take upto 20 seconds.

Everyone knows performance is a feature, including the client who aren’t happy with the current wait-and-aggregate solution — especially when competitors have real-time results streaming.

And here is the problem — we just don’t know which results will be cheapest until all suppliers have responded; A process that could take upto 20 seconds — yet will have results available within less than 0.5.

The technical challenges are to get users results as fast as possible, whilst they must see the cheapest results possible — and oh, there’s over a 1000 results sometimes so don’t kill mobile devices, please.This is a Business Decision, Let’s Talk to Them

As a tech team we sat down and listed a number of possible solutions; many of which affected which results users would see, how many results users would see and also how fast users would see results. As you can see these are big trade-offs that can significantly affect how many sales will be made and how much profit the client will make.

So we should talk to them…..

“How important is it that users always see the cheapest results”, “What if results dissapear off the screen?”, “What if cheaper products are available after 10 seconds?”, “What if we introduce a hybrid that always waits for 3 seconds?”.

During these discussions with the client it was often a case of “how many cheaper results?”, “for what percentage of searches would there be cheaper results 20 seconds later?”, “which suppliers usually have the cheapes offers for each type of query?” and similar questions.

Now we’re making progress… starting to identify variables of a formula that will allow the business to calculate a sensible starting point and prioritise the importance of their business rules to a highly-granular level.How to Capture the Data?

Once we identified the variables it was quite easy to work backwards to determine what to measure such as the number of cheaper offers from each subsequent supplier and the offset in response time. For our application we’re using Coda Hale’s metric library but there are a number of tools out there that could also be used depending on your platform.

In our case we are able to take request logs from the existing system and apply them to the prototype of the system we are building before it goes live. More generally speaking we just wanted to get feedback and data as fast possible to achieve most profit for the company, with least expenditure on rework or loss of revenue from not deploying sooner.Moral of the Story

A lot of technical decisions are camouflaged business decisions…. A lot of business decisions require data to provide insights about how best to please customers and make money…. The better the business decision the less re-work, expenditure and loss of opportunity needed to reach optimal profitability.

Therefore, you write code but you have a lot of opportunity to make a difference. Look for ways to provide insights to the business by probing them about what they could do if you started to measure x, y or z. Then measure it.