Security Metrics – Measure the right thing

Every business needs to have a way of making sure that the money it spends on things is justified by the value those things have to the business. Entire industries have grown up around developing ways to measure how much items cost to produce, how much it costs to get them to the market and, therefore, how much they need to be sold at to make a profit. For better or worse, metrics drive business.

Unfortunately, security metrics have a long way to go before they reach maturity of the rest of the business measurements. This leads to lots of problems and, frequently, results in businesses making poor decisions about security. Without good metrics, it is difficult to know what security controls are providing value for your business and which ones are simply draining resources.

Developing good metrics not only improves your overall security but directly saves your business the costs of unnecessary security controls.

The difficulty most businesses face is the often scary hurdle of developing good metrics in the first place. As security measurements are still immature, there is no single common approach to take. This is compounded by every business facing different security risks, and therefore needing to measure different things, which effectively forces every business to develop their metrics from the start.

Frequently this leads to businesses resorting to generic metrics or, worse, ones provided by particular security vendors. While this may, initially, appear like a quick and easy way to develop security metrics, there are some fundamental flaws:

Generic Security Metrics. Using generic metrics means you are measuring security controls that someone else thinks are important. Frequently these resort to simply measuring things you can easily measure (such as how quickly software patches are applied, when Anti Virus data bases were last updated etc) rather than providing you with useful business intelligence to drive decisions about security controls.

Vendor Security Metrics. Most security products come with some way of measuring how they are used, and frequently these are promoted as providing you with business security metrics. However this is fundamentally flawed as inevitably this will be biased towards making the product look good and it leaves you no way to compare across products and brands. It is also frequently impossible to compare metrics from one security area to another, meaning you cant easily re-prioritise. This is always a poor choice for metrics.

How to develop security metrics

As mentioned above, there is no one-size-fits-all solution. Security metrics must always be developed with your businesses goals, culture, risks and environment in mind. While it may take effort, it is still worthwhile building your own metrics.

It is crucial that engagement begins at the highest level within your organisation and gets full management support. Establishing metrics means you may have to make difficult decisions over what parts of your business are important, and this should never be subjected to office politics.

Step 1: Look over your business and see how you can group important assets – common groupings are information (customer data, business intelligence data, market research, new product development plans etc), physical assets (stock, premises, factories, land etc), people (employees, customers, visitors); but make sure you group them in a way that makes sense for your business.

Step 2: Create an inventory of your assets within each group, and if possible give approximate values – this will help in the future when you need to assess the cost:benefit of protecting each asset.

Step 3: Review what things can go wrong – stock can be stolen, customers &/or employees can be violently assaulted, intellectual property can be copied. This is the longest step and it is vitally important that a lot of effort is spent getting this right. Initially it makes sense to map the threats to the groups you put together in the first step, but you should also try to be specific enough that at least the high value assets in each group are covered.

Step 4: Determine an appropriate, and meaningful, way to gauge success in each   control area. This the step where general advice is the least useful, as it very much has to depend on your individual circumstances. Don’t fall into the common trap of simply measuring what is easy to measure and, when you are developing your metrics, don’t be afraid to come up with metrics that you aren’t currently able to measure – you might come up with some way to measure it in the future.

Some things to consider are how much stock you have lost to “wastage”, how many fraudulent transactions your credit card processor rejects, how frequently visitors to your hospital are violent etc.

Step 5: Measure and use the output to determine where security controls should be enhanced and where they can be relaxed. Remember, though, that good metrics take time to collect so make sure this is all part of your long term business plans.

Problems with Security Metrics

Metrics are good, and important, but they should also be treated with some caution. Some points to consider are:

  • Measuring the wrong things leads to a false sense of your organisations security posture, which invariably leads to security breaches and losses. For example, an organisation may spend a lot of time and effort monitoring how frequently software patches are applied but ignore how frequently the firewall is breached.
  • Metrics can be misleading and this is especially true with security metrics when you might be looking at very infrequent events. This can lead to thinking that a control is working, when it isn’t, or that a control isn’t needed when it is. To combat this, you should always make sure that you collect as much data, for as long as possible so you can be reasonably confident that you are comparing like for like.
  • Human nature means that people will strive to improve the reported metric, which is not always the same as improving the security. For example, if your metric is how many portable devices are reported lost or stolen, globally, each month then there is the very real risk that people will just stop reporting. This is the worst possible outcome: both your metrics, and any follow up action (especially if disciplinary action is involved) should always keep this in mind. Where possible, metric collection should be automated rather than reliant on people.
  • Only measure things that matter, don’t get caught in the trap of measuring things simply for the sake of gathering data. Every metric should be developed with a view to how you will act on the information it provides. If you don’t intend to do anything with it, don’t bother collecting it.

Security Metrics

Developing good security metrics is worthwhile and will always save your business money over the long term, whatever the initial effort it requires. Having good metrics leaves you better placed for facing future risks, dealing with existing threats and making sure that your resources are being properly allocated.

Metrics should always be developed specifically for your organisation’s circumstances and should be tailored to reflect what is important to you. Once you have developed your metrics, you should also put together a plan for how you will act on the information they give you.

If you want more advice on how to build reliable, effective, security metrics for your business, or would like to discuss any aspects of how security can improve and enhance your organisation, then get in touch with Halkyn Security Consultants and we’ll be happy to talk to you.

Taz Wake - Halkyn Security

Certified Information Systems Security Professional with over 19 years experience providing in-depth security risk management advice to government and private sector organisations. Experienced in assessing risks, and producing mitigation plans, worldwide in both peaceful areas and war zones. Additionally, direct experience carrying out investigations into security lapses, producing evidential standard reports and conducting detailed interviews to ascertain the details of the incident. Has a detailed understanding of the Security Policy Framework (SPF) and JSP440, as well as in depth expertise in producing cost-effective solutions in accordance with legislative and regulatory guidelines. Experienced in accrediting establishments and networks as well as project managing the development of secure, compliant, workable business processes.

This Post Has One Comment

Comments are closed.