Risk Evaluation in the Final Stages of the Pandemic
By David Brentlinger
Small Talk, August 2022
The direct and indirect effects of the pandemic have impacted many actuaries’ work since the spring of 2020. This article addresses three considerations actuaries should account for when using risk evaluation models which are focused on projecting remote, highly unlikely losses. In this context, “using” can refer to providing services such as setting or reviewing assumptions or the model output. The three areas of consideration are:
- Model effectiveness
- Asset risk
- Insurance (mortality and mortality) risk
Risk evaluation models which are focused on projecting remote, highly unlikely losses are used to support the Own Risk Solvency Assessment (ORSA) document, an annual requirement for many insurance companies, economic capital models, and capital adequacy assessments.
My career as an actuary has provided many experiences in setting and reviewing assumptions and model output for various types of models. I always found the models which were intended for evaluating risk under remote, highly unlikely conditions to be the most challenging. I undoubtedly found myself waffling between “are the projected losses too high?” and “are the projected losses not high enough?”
A healthy degree of skepticism is a positive characteristic of a good actuary, particularly when it comes to using models which will be used as the basis for decision-making. Skepticism goes with the territory for these types of models; none of us can be sure whether we have ever experienced a 1-in-200 or 1-in-300 year event, so, intuitively, there is a struggle in “substituting facts for appearances and demonstrations for impressions”[1] in these instances.
But skepticism can become unhealthy when it gets in the way of delivering a high-quality work product on time.
Model Effectiveness
The pandemic provides an opportunity to ask questions about the effectiveness of existing risk evaluation models, meaning how did the results of the model, which was in effect going into 2020, hold up to the actual results which transpired since the spring of 2020? Questions actuaries may find helpful in determining the effectiveness of risk evaluation models include:
- Was the output of the model found effective in supporting management during the pandemic; for example, either by assuring management that “we expect things to be OK” or used to support actions taken by management in response to experience that began emerging during the pandemic?
- Did the pandemic reveal any perceived biases in the model? Do the users of the model now feel that the projected losses associated unlikely events (e.g., a 1-in-100 year or 1-in-200 year event) are materially under or over stated?
- Were any methodologies or assumptions found to be not well understood or misunderstood by stakeholders?
- Were any aspects of the model found to be too simplified?
- Were any key assumptions found to be inconsistent with each other?
- Does the current model adequately reflect that future remote, unlikely events may materially differ from past remote, unlikely events?
If not performed on a regular basis already, I would strongly recommend establishing a cadence of performing a deep model review for each risk/sub-risk. The deep dive would include reviewing the appropriateness of the model methodology and assumptions and challenging each to the scrutiny of internal and external critiques. An annual exercise like this provides a check and balance against complacency that develops in a model, particularly one measuring highly unlikely events. The mindset for those involved in the deep dive should be “how should we change the model to better capture the inherent risk” as opposed to “how can we continue to support the model.” The cadence could be to perform the deep dive of each risk or sub-risk every three years.
Another healthy practice would be to scrutinize the maturity of models used for projecting losses for risks identified as top emerging risks (based, for example, on the Society of Actuaries’ Annual Survey of Emerging Risks).
Asset Risk
Asset risk refers to risks associated with the insurer’s investments, including the possibility of investments defaulting or loss of market value. The pandemic, as well as the prolonged low interest rate environment which preceded the pandemic, drove many insurers into alternative investment classes which were expected to provide additional yield as well as improve the diversification of returns. These investment classes have different liquidity profiles and different risk factors compared to investment classes the life insurance industry has traditionally invested.
Questions which the actuary should consider include the following:
- How well does the risk evaluation model reflect the new emerging investment classes used by the insurer?
- Actuarial judgement has its place in setting assumptions for these types of models. However, is the actuary meeting the guidance provided in the Actuarial Standards of Practice when it comes to assumptions for the new emerging investment classes for projecting asset losses and the classes’ contribution to the diversification benefit?
- In quantifying the diversification benefit, has the actuary considered the changing degree of diversification at various stress levels?
- Is the appropriate reliance being made on the amount of historical data used in setting assumptions?
- Do the assumptions reflect the risk factors underlying the state of the asset market as of the effective date of the analysis?
- Can the appropriateness of the assumptions be supported?
- Are the management actions assumed for these new investment types appropriate and represent actions which would be performed in the event of a crisis?
It is likely the actuary will be working with investment experts when modeling asset risk, in many cases relying on these experts. In these situations, the actuary should still be the healthy skeptic in asking questions similar to the ones provided above. The objective is not that there is the one correct result from the model, but rather to produce a model which meets the intended purpose and captures the range of possible outcomes.
Insurance Risk
Insurance risk refers to risks associated with future claims differing from expected. Is the risk evaluation model prudently capturing the effects of the insurance risk in the insurer’s portfolio? The answer to this question, in light of the pandemic, is the $64,000 question. I am used to measuring insurance risk in terms of claims volatility, mis-estimation of the expected claims level (or trend risk), and catastrophic risk. The pandemic’s impact on these risk factors continue to emerge and will not be known for years. Complicating the assumption setting is the uncertainty around the expected claims level, a typical standard process in most years, but not this close to the pandemic.
Questions which the actuary should consider include the following:
- Has the change in the company’s in-force been appropriately reflected in the risk factors?
- How does the catastrophe model calibrate, in terms of likelihood and severity, to historical pandemics over the last two hundred years?
- Does the actuary feel the catastrophic model has been overfitted to the 1918 flu or the COVID-19 pandemic, or another catastrophic event?
- Did the actuary consider adding margin to an assumption or a parameter used to determine an assumption to reflect the lack of reliable data (e.g., future mortality and morbidity expectations given the emerging impacts of the pandemic)?
- How is the level of parsimony (achieving a desired level of goodness of fit using as few explanatory variables as possible) for projecting catastrophic events justified?
- Has the pandemic caused the company to review any of its pricing assumptions related to its current underwriting processes and expected mortality and morbidity rates?
Where Do We Go From Here?
Complicated questions usually demand complicated answers. Actuaries have the training and the experience to address the questions like the ones posed in this article. Below are suggestions I have found effective in using models focused on projecting remote, highly unlikely losses:
- Assumption setting for these types of models is a challenge. Always start with the Actuarial Standards of Practice (in particular, ASOP 46 “Risk Evaluation in Enterprise Risk Management,” ASOP 56 “Modeling,” and the draft of ASOP “Setting Assumptions”).
- Be transparent. I have found sharing the challenges of setting certain critical assumptions with key stakeholders to be very helpful. This transparency creates a feedback mechanism, as well as creates credibility and model buy-in.
- Leverage the expertise of others. Partner with your reinsurers, your investment advisors, external experts (both in and outside of the insurance industry), and others in your professional network.
- Follow emerging experience. Leverage resources from organizations like the Society of Actuaries and the CDC.
- Follow up on lessons learned. Companies that created a list of lessons learned from the pandemic should revisit the list and not “repeat the mistakes of the past.” Modeling lessons learned from one type of risk are usually transferrable to other risks.
- Supplement stochastic model output with scenario-based output.
- One technique I have found helpful in contributing to the support of output from risk evaluation models is to compare the results of a model to historical events (appropriately adjusted for differences in the size and makeup of the business, market conditions, etc.). This technique can provide a basis for establishing the plausibility of a loss at a stated likelihood level (e.g., 1-in-200 event); for example, compare projected losses at a given likelihood to losses the company sustained in 2008–2009, the pandemic, etc. Care must be taken in applying this technique:
- The future can, and will, unfold differently than the past, even from historical extreme events.
- Be careful not to be fixated on calendar year timeframes; expanding the analysis beyond a typical calendar year view can be instructive.
- Improving a model should not lead to creating an overly complicated model, one borne with a high degree of model risk.
- As mentioned earlier in this article, perform a scheduled periodic deep dive of all risk models.
Not to be too cliché, but risk is opportunity. These challenging times provide actuaries an opportunity to shine by demonstrating their skills in assisting companies with understanding their risk-taking practices.
Statements of fact and opinions expressed herein are those of the individual authors and are not necessarily those of the Society of Actuaries, the editors, or the respective authors’ employers.
David Brentlinger, FSA, CERA, MAAA, is president of Brentlinger Consulting LLC. He can be reached at david@brentlingerconsulting.com.