Can anybody do risk analysis?

Barry Ezell, PhD

Cyn: Sometimes I sing and dance around the house in my underwear. Doesn't make me Madonna. Never will – Working Girl, 1988

What can go wrong - how likely is it - what are the consequences?  Seems easy, right?  Perhaps not.  Yet we need to tackle these questions in order to responsibly lead our respective organizations, be it in business or government.  But how many times have you seen a very smart person put in a position to lead a large risk assessment and neither that leader nor his/her key staff members have a clue?  In the past ten years, we have seen a multitude of large-scale security risk studies conducted yet key members of the team including the leadership and senior management did not have fundamental education or training in risk analysis.

Risk analysis is not an amateur sport.  It is a serious discipline with its own theory, literature, professional society, rich history, and notable giants in the field.  It lives at the intersection of many disciplines such as systems analysis, operations research, decision analysis, social sciences, to name just a few.

It is also fair to say that there are some largely un-resolvable matters of discourse in the discipline that deal with differing worldviews on modeling adversaries, combining subjective data with frequency data, aggregating natural hazard verses manmade hazards.

So for what it’s worth, I have some suggestions for the person in charge of the next big risk assessment[1]:

  1. Nosce te ipsum (Know Thyself) – Let’s face it.  If you are not trained or educated to be a risk analyst it will be clear to everyone.  Build a team of real experts with exceptional credentials.  Credentials come in the form of education – not just the school but also the courses, the advisors, the thesis if masters, or the dissertation if a Ph.D.
  2. If your team is filled with junior people with the wrong education you have a serious problem and your management must be made aware. (If you find yourself in this position, quickly print this article out and show it to your boss).
  3. It’s holistic, systemic, systematic, repeatable, transparent, informed by stakeholders, guided by senior advisors (with rock star analytic credentials), governed by an authoritative directive, and technically led by a professional risk or decision analyst.
  4. That’s like aiming one’s life directly at happiness.  It’s too high level and we know that there are things under happiness that are necessary to accomplish to achieve the goal of happiness.  This has created a host of problems - impossible to access effectiveness of assessments, weak linkages to customers, ambiguous results not tied to any particular decision issues, and work that gets done that just wastes time and money.
  5. Write a study plan – The study plan critical.  It lays out the issues, essential elements of analysis, analytic approach, data requirements, and becomes the contract with the sponsor. 
Risk Analysis.png

I’ve seen the results when risk study teams fail to adequately align their model output with the key decision issues of the sponsor of the risk assessment.   Shown below are the most common mistakes that doom a study.

  • The study team does not identify all the critical stakeholders in detail, to include name, supporting agency, and what their needs are.
  • The study team does not achieve a common understanding of study purpose, objectives, and decision issues. 
  • The study team either fails to establish key constraints, limitations, and assumptions; or fails to get approval by sponsor for the key constraints, limitations, and assumptions. 
  • For large studies that impact multiple agencies, the sponsor does not publish a directive or other authoritative documents that: directs the study to occur, assigns a study director, and establishes other resourcing instructions. 

Can anyone do risk analysis? Yes.  Go for it!  Nota Bene - the last ten years have shown us that risk analysis is wicked hard requiring the best people, disciplined processes, and clear executive guidance and buy-in.  Again:  it is not an amateur sport.  Don’t get me wrong - Flag football is fun. But like Dennis Buede once said to a young scientist in 2005, “Just because you purchased Crystal Ball doesn’t mean you can do risk analysis”.

Some links:

The Society for Risk Analysis (SRA):  http://sra.org

SRA publishes a journal: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1539-6924

[1] Smaller efforts should follow the intent of this process are obviously much more tightly bound by resource constraints.