Methods
The MHCOVID framework consists of three main streams of work, outlined in detail below.
We follow the standard systematic review approach, adapted to the case of prevalence studies. The protocol has been registered with PROSPERO and full details can be found there.
We include studies described as cohort studies, cross-sectional studies, surveys and prevalence studies that are undertaken in the general population irrespective of age. Studies undertaken exclusively in subgroups of the general population (e.g. people with diabetes, pregnant women etc.) are also included. Participants should be affected by the COVID-19 epidemic/pandemic; hence the study should have taken place after December 2019. We include only studies that used validated questionnaires or interviews about the symptoms and diagnoses of the conditions of interest. We exclude studies exclusively reporting on health care personnel, SARS-CoV-2 cases (suspected or confirmed) and COVID19 patients.
We worked together with librarians from ISPM to develop a search string for the identification of studies in the ISPM COVID-19 living evidence database. This is a curated data set that includes COVID-19 relevant studies and its construction is described here, along with the full search string. In January 2021, the search strategy has identified more than 34,000 potentially relevant papers.
We extract data on mental health conditions that include but are not limited to depression, anxiety, psychosis, post-traumatic stress etc, but also alcohol and drug abuse, aggression and violence. As the main outcome is the change in symptoms and diagnosis, before and after the pandemic, we extract any relevant information presented in the paper. We also search for the most recent prevalence study undertaken in the same region or country before the pandemic to boost the informative value of the data.
As there is no standard and widely accepted tool for assessing the risk of bias for prevalence studies, we are developing our own risk of bias assessment tool tailored to our own needs, containing items derived from a variety of relevant instruments for assessment of risk of bias in prevalence studies.
Each study identified and included in our review was undertaken at location \(l\) at timepoint(s) \(t\). We use the following definitions of exposure.
To measure the intensity of the pandemic, in a period of 5 to 7 days before the survey, we identify for location \(l\) the:
- Number of daily new confirmed cases per 1000 people
- Number of daily COVID-19 deaths per 1000 people
Similarly, to measure of the intensity of the containment measures we consider:
- Days in containment measures at time point \(t\)
- Level of movement restrictions in three groups:
- No-restrictions (e.g. Sweden)
- Shops closed and some restrictions (e.g. Switzerland in April)
- Complete lockdown with legal penalties (e.g. Italy and Greece in April).
Our main source of information are the Worldometer statistics for the pandemic measures and the Oxford COVID-19 Government Response Tracker (OxCGRT) for the containment measures. The OxCGRT collects information on 17 indicators of government responses to the pandemic. They provide a strigency index per country and timepoint using information on containment and closure policies, economic policies and health system policies.
When enough data for a condition is available, we plot each study’s prevalence (proportional to the study sample size) against the four exposure measures identified in Work Stream 2. We synthesise data on the association between exposures and change in mental health status using a Bayesian meta-regression model. Below we present the model for a dichotomous prevalence. For a change in symptoms (continuous measure) the same model holds with modifications that relate to the likelihood function (normal instead of binomial).
Consider a study \(i\) that reports the prevalence of a condition \(C\), \(p_{lti}^C\) in location \(l\) at timepoint \(t\) during the pandemic. The condition \(C\) can be any mental health condition (anxiety, depression, anorexia, drug dependence, aggression etc.). Consider also the prevalence of the same condition \(p_{l0i}^C\) before the pandemic.
The data in study \(i\) are the number of people with the condition \(C\), \(r_{lti}^C\), out of the total number examined \(n_{lti}^C\). The binomial likelihood of the data is \(r_{lti}^{C} \sim Bin(p_{lti}^{C}, n_{lti}^{C})\).
Then, we model the logit-prevalences as $$logit(p_{lti}^{C})=logit(p_{l0i}^{C}+\beta^{C}\times \mathit{EXP}_{i} + \gamma^{C} \times X_{i}) \tag{1}$$ where \(\mathit{EXP}_i\) is one of the four exposure measures in Stream of Work 2 \(\mathit{EXP}_1\) to \(\mathit{EXP}_4\), \(X_i\) is a matrix with patient and setting characteristics such as average participants' age, percentage of women included, study quality etc. The coefficient \(C\) measures the impact of the intensity of the pandemic or containment measure on condition \(C\) and the vector \(C\) measures the impact of the characteristics \(X_i\).
If study \(i\) reports data suitable to estimate the pre-pandemic prevalence \(p_{l0i}^{C}\) needed in Equation 1 we use them and we add information from studies prior to the pandemic by assigning a prior beta distribution to \(p_{l0i}^{C}\).
Because synthesis is taking place within each condition, it is likely that only a few studies are available and are not enough to estimate the parameters in model of Equation 1. We take advantage of the Bayesian setting to assume exchangeability across conditions \(C\), by assuming all \(\beta^C\) follow a common distribution within all "typical" mental health conditions such as depression, anxiety, or eating disorders (but not alcohol/substance abuse or violence): \(\beta^{C} \sim N(B^{MH}, \sigma^{2})\).
Similarly, the impact of population characteristics can be assumed exchangeable \(\gamma^{C} \sim N(\Gamma^{MH}, Τ^{2})\). Note that coefficients that refer to alcohol/substance abuse or violence are independent.
We develop our models using the rjags package in R software.