The Point of a Risk Assessment

Left of Bang Briefing #82

Welcome back to The CP Journal, where we break down what it takes to get left of bang.

What makes a risk assessment good?

I realize that might not be the most engaging way to open an article hitting your inbox on a Sunday morning, but I’ve been spending a lot of time over the past few weeks thinking about, talking about, and writing about risk beyond what you see on this site. Somewhere along the way, I’ve become a little fixated on the subject.

At least in theory, every major emergency management, public safety, and corporate security decision begins with an assessment of risk.

For these agencies and teams—again, at least in theory—most major investments eventually get justified by the risk they help address. Regardless of whether it is a piece of equipment that gets purchased, a plan that gets developed, an exercise conducted, a staff member that gets hired, or a funding request approved, it is often tied to one or more ways organizations reduce risk.

This comes in three forms:

• Reducing the probability of an event occurring and making it less likely.

• Reducing the consequences or damage or impact if it happens.

• Becoming more ready to respond to, recover from, or continue operating through it.

If you want to learn more about how organizations assess risk, read What Are You Preparing For? where I break down a practical approach to deciding which threats and hazards deserve your attention.

In every case, someone is making a judgment about what could get in the way of the organization doing what it is supposed to do, so we naturally spend a lot of time trying to understand that risk.

But I’m increasingly convinced that the way we assess risk and the way we actually use those assessments don’t line up very well.

---

---

When people talk about developing a risk assessment, the emphasis is usually on the quality of the analysis. Is it comprehensive? Is it data-driven? Did we consider enough scenarios? Did we miss anything important? And I understand why. If you are the person responsible for the assessment, leaving something out feels risky.

But once all of that work is finished, it gets handed to someone who has to make a decision, which is at the core of what I’ve been focused on.

A risk assessment approached as a deliverable gets developed differently than one designed—from the beginning—to support decisions.

When the assessment itself is the deliverable, there is a natural incentive to keep adding to it. More research makes the work feel more defensible, and before long the document is carrying more detail, data, citations, and supporting material because completeness starts to feel like quality.

Yet, even though all of that research is necessary at times, the problem emerges when the complexity required to develop the assessment becomes complexity for the person trying to use it.

You can end up with an analyst who has spent weeks or months buried in the information and who has developed a document so long that few of the people making decisions will ever use it.

But the executive doesn’t have that same benefit. They don’t have the same understanding of the assumptions, the tradeoffs, or the weak spots in the data used to develop that report.

Unless the analyst is attached to the executive’s hip—sitting in the meetings and present on the calls where decisions are actually made—that deeper understanding doesn’t automatically translate into better decisions for the organization.

This is because executives already have another way of assessing risk. They have their intuition.

Many experienced leaders maintain a natural understanding of their operating environment and the risks they face. Their job is to know where their organization is strong, where it is exposed, what keeps going wrong, what has changed recently, and what concerns them.

That isn’t to say their intuition reflects the “best” information available. Their understanding may be incomplete, biased by personal experience or beliefs, or overly influenced by recent events. But it has one enormous advantage over a formal risk assessment: it is available immediately and all the time.

So any documented assessment has to compete with it. If finding the assessment takes too long, if interpreting it requires outside help, or if the executive questions whether the information is still current, then there is a good chance they bypass a review of the formal assessment and just make the decision using the understanding already in their head.

---

That puts a different standard on the assessment. If its purpose is to help people make better risk-informed decisions, then I think a few things have to be true.

It needs to be accessible.

When someone is making a risk-oriented decision, they should be able to quickly check the assessment and understand how the issue they are considering fits within the organization’s larger risk picture.

That means they need to know the assessment exists, where to find it, and get to the relevant information without needing an analyst to walk them through it. The more friction involved in finding the information, the less likely it is to be used when decisions are made.

It needs to be current.

Even if an executive can find the information quickly, they still have to trust that what they are looking at reflects today’s reality.

An assessment that accurately describes an organization’s risk two years ago may still contain useful information, but risk changes all of the time. New information and research becomes available, threats and hazards evolve, the operating environment shifts, and organizations gain and lose capabilities every time a person joins or leaves the organization.

So updating the assessment has to be easy too. Adding new information and data, and recalculating its effect on risk can’t require restarting the entire process. The goal is to provide a living understanding of risk that can change as the environment changes.

It needs to be fast.

There is also a problem of how long it takes to create a risk assessment. If you think about a hazard mitigation planning process, the goal is to determine which mitigation projects will have the greatest impact and where a community can invest their limited funding and resources.

Yet before getting to those decisions, they might spend months assessing or reassessing their risk. Some of that work to collect good information, validate it, and apply it will take time, but every week spent getting to a usable understanding of risk is another week before the organization can begin deciding what to do about it.

And sometimes you don’t have months. If a leader asks how a newly identified threat compares to existing priorities, whether a proposed investment addresses a significant risk, or what has changed since the last assessment, the answer can not take months to produce. It has to be answered while the decision is being made.

Finally, it needs to be relevant to the decision being made.

This might be the most important test and probably the question I’d start with. Before beginning a risk assessment, we should be able to answer a fairly simple question: What decisions are we trying to make better?

The answer should shape the assessment itself—what gets included, how deep and far you go with the analysis, and what the decision-maker actually needs to see presented.

There will always be more that could be researched. There is always another data source, another scenario or angle to explore, another vulnerability, and another layer of analysis that could be added. But at some point, that process needs to stop and the information has to reach the person making the decision in a form they can actually use.

The judgment required to know when to stop can only come from knowing what decision is being made.

---

That brings me back to the question I started with: What makes a risk assessment good?

Comprehensiveness, accuracy, methodology, and the quality of the underlying research are all important, but those elements are secondary to making better decisions. The outputs of this process should show up in the quality and intentionality of where an organization invests its time and money.

A good risk assessment should reduce the friction and time required to develop an assessment because that friction and time carry a cost. Every additional week spent researching, analyzing, documenting, or updating a previous risk assessment delays the moment when an organization or a leader can recognize a priority, choose between competing investments, identify where additional capability is needed, or decide where limited time and resources should go.

The less friction there is between understanding the risk and making those decisions, the more valuable the assessment becomes and the sooner an organization can get to the work of actually doing something about the risk it identified.

---

As we close out this essay, I’m curious where the most amount of friction shows up in your organization’s risk assessment process—developing it, keeping it current, or actually using it. If you’re open to it, hit reply and let me know.