Metamodels & SHAP analysis: Unearthing data interactions
Date:
This study demonstrates how SHAP analysis can be applied to a neural network-based metamodel to perform sensitivity analyses and elucidate interactivity between inputs.
Abstract: Neural network-based metamodels have been shown to accurately reproduce the results of complex health economic models. SHapley Additive exPlanations (SHAP) is a technique that can be used to explain the output of a neural network. This study demonstrates the use of SHAP analysis to interpret the outputs of neural network-based metamodels to ascertain and elucidate relationships and drivers within the underlying health economic models, facilitating the quantification of uncertainty within outcomes such as cost-effectiveness.
A neural network-based metamodel was fitted to a dynamic prevalence model of chronic kidney disease (CKD) between 2024 and 2060, based on a system of ordinary differential equations (ODEs). Latin hypercube sampling was used to generate training and test sets of 20,000 and 10,000 model scenarios, respectively. SHAP analysis was then carried out on the neural network to quantify the influence of the inputs on the model outcomes, including total excess deaths and total end-stage kindey disease-related deaths. The SHAP analysis indicated that each outcome was influenced by different drivers with a shared commonality of increased sensitivity to changes in parameters related to older age groups.
SHAP analysis is a useful technique to elucidate sensitivities within neural networks and can serve to ascertain and quantify sensitivities within health economic models via neural network-based metamodels. There is a potential to use this approach to complement conventional deterministic and probabilistic sensitivity analyses, especially when distributional knowledge of inputs is missing or where interactivity between parameters should be characterised.
Recommended citation: Vodyanov, A., Padgett, T., Mitchell, C., McEwan, P. (2025) Metamodels & SHAP analysis: Unearthing data interactions. ISPOR EU 2025, 11 November 2025; Poster.