Example: Social Anxiety

This vignette describes the analysis of 101 trials comparing 41 first-line treatments in 17 classes for social anxiety disorder in adults (Mayo-Wilson et al. 2014). The data are available in this package as social_anxiety:

head(social_anxiety)
#> # A tibble: 6 × 8
#>   studyn studyc          trtn      y     se trtc                            classn classc           
#>    <dbl> <chr>          <dbl>  <dbl>  <dbl> <chr>                            <dbl> <chr>            
#> 1      1 BOWLER2012         1 NA     NA     Waitlist                             1 Waitlist         
#> 2      1 BOWLER2012         6 -0.916  0.318 Self-help internet no support        5 Self-help no sup…
#> 3      2 ABRAMOWITZ2009     1 NA     NA     Waitlist                             1 Waitlist         
#> 4      2 ABRAMOWITZ2009     7 -0.956  0.390 Self-help book with support          6 Self-help with s…
#> 5      3 ANDERSSON2012      1 NA     NA     Waitlist                             1 Waitlist         
#> 6      3 ANDERSSON2012      8 -0.768  0.125 Self-help internet with support      6 Self-help with s…

Mayo-Wilson et al. (2014) analysed this dataset using a NMA model with class effects, which we recreate here. When fitting class effects models there are multiple modeling options available.

We demonstrate the model selection strategy proposed by Perren et al. (in preparation) to determine the most suitable class effects model.

Setting up the network

We follow Mayo-Wilson et al. by analysing results as standardised mean differences (y) with standard errors (se) therefore we use the function set_agd_contrast to set up the network. We set treatment classes with trt_class = classc and set Waitlist as the network reference treatment.

sa_net <- set_agd_contrast(social_anxiety,
                           study = studyc, 
                           trt = trtc,
                           y = y, 
                           sample_size = 1,
                           se = se,
                           trt_class = classc,
                           trt_ref = "Waitlist")

sa_net
#> A network with 101 AgD studies (contrast-based).
#> 
#> -------------------------------------------------- AgD studies (contrast-based) ---- 
#>  Study           Treatment arms                                
#>  ABRAMOWITZ2009  2: Waitlist | Self-help book with support     
#>  ALDEN2011       2: Waitlist | CBT group                       
#>  ALLGULANDER1999 2: Paroxetine | Pill placebo                  
#>  ALLGULANDER2004 3: Paroxetine | Pill placebo | Venlafaxine    
#>  ANDERSSON2006   2: Waitlist | Exposure in vivo                
#>  ANDERSSON2012   2: Waitlist | Self-help internet with support 
#>  ANDREWS2011     2: CBT group | Self-help internet with support
#>  ASAKURA2007     2: Fluvoxamine | Pill placebo                 
#>  BALDWIN1999     2: Paroxetine | Pill placebo                  
#>  BERGER2009      2: Waitlist | Self-help internet with support 
#>  ... plus 91 more studies
#> 
#>  Outcome type: continuous
#> ------------------------------------------------------------------------------------
#> Total number of treatments: 41, in 17 classes
#> Total number of studies: 101
#> Reference treatment is: Waitlist
#> Network is connected

We create a plot at the class level by setting level = "class".

plot(sa_net, level = "class", weight_nodes = TRUE) + 
  theme(legend.position = "bottom", legend.box = "vertical")

Model selection strategy

We follow the model selection strategy proposed by Perren et al. (in preparation)

START Step 1: Heterogeneity Assessment - FitFE and RE models, select preferred model Step 2: Consistency Assessment - Fitthe UME model for the chosen FE/RE model Is the consistencymodel preferred? Inconsistency present - Checkfor data issues, investigatepotential causes of inconsistency.If these can be resolved, restartthe model selection process. Step 3: Class Assessment - Fit an exchangeable classmodel with corresponding fixedor random treatment effects. Is the exchangeableclass model preferred? Avoid class assumptions - Optfor the no class effect model.If disconnected, seek alternativenetwork-connecting methods. STOP Step 4: Class Model Assessment - Fita common class model with correspond-ing fixed or random treatment effects. Does the model use ran-dom treatment effects? Choose preferred model:Exchangeable class FE,Common class FE.STOP Fit exchangeable class FE model.Select the preferred model:Exchangeable class RE,Exchangeable class FE,Common class RE.STOP No Yes No Yes No Yes

STEP 1: Heterogeniety assessement

First, we assess heterogeneity by fitting fixed effects (FE) and random effects (RE) models using nma() and stating trt_effects = "random" for the RE model and trt_effects = "fixed" for the FE model. We use uninformative priors on the treatment effects with prior_trt = normal(0, 100) and heterogeneity with prior_het = half_normal(5).

set.seed(951)
sa_fit_FE <- nma(sa_net,
                 trt_effects = "fixed",
                 prior_trt = normal(0, 100),
                 prior_het = half_normal(5),
)

sa_fit_RE <- nma(sa_net,
                 trt_effects = "random",
                 prior_trt = normal(0, 100),
                 prior_het = half_normal(5),
)

The model fit under the FE and RE models can be checked using the dic() function.

(sa_dic_FE <- dic(sa_fit_FE))
#> Residual deviance: 288.3 (on 147 data points)
#>                pD: 40.1
#>               DIC: 328.5
(sa_dic_RE <- dic(sa_fit_RE))
#> Residual deviance: 162.1 (on 147 data points)
#>                pD: 94.5
#>               DIC: 256.6

The DIC for the random effects model (256.6) is much lower than that of the fixed effects model (328.5) due to the large decrease in residual deviance showing a much better fit to the data. Therefore, our preferred model is the RE model, which we use in subsequent steps.

STEP 2: Consistency assessment

The next step is to assess inconsistency using an unrelated mean effects model (UME), comparing model fit statistics for the UME model and the NMA (consistency) model, both with RE. To fit a UME model we specify consistency = "ume".

sa_UME_RE <- nma(sa_net,
                 trt_effects = "random",
                 consistency = "ume",
                 prior_trt = normal(0, 100),
                 prior_het = half_normal(5))

We compare model fit from the NMA (consistency) RE model and the UME RE model using the dic() function and \(\tau\).

(sa_dic_RE <- dic(sa_fit_RE))
#> Residual deviance: 162.1 (on 147 data points)
#>                pD: 94.5
#>               DIC: 256.6
(sa_dic_ume_RE <- dic(sa_UME_RE))
#> Residual deviance: 161.5 (on 147 data points)
#>                pD: 109.1
#>               DIC: 270.5

summary(sa_UME_RE, pars = "tau")
#>     mean   sd 2.5% 25%  50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau 0.22 0.03 0.16 0.2 0.22 0.24  0.29     1059     1906    1
summary(sa_fit_RE, pars = "tau")
#>     mean   sd 2.5%  25%  50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau 0.21 0.03 0.15 0.19 0.21 0.23  0.27     1137     2163    1

The residual deviance is not meaningfully different between the two models, however, the DIC is lower in the NMA model (256.6) compared to the UME model (270.5). There is no evidence of inconsistency at the global level.

To asses local inconsistency, we use the plot() function that produces a “dev-dev” plot of the residual deviance contributions for the UME model plotted against the NMA consistency model.

plot(sa_dic_RE, sa_dic_ume_RE, show_uncertainty = FALSE) +
  xlab("Residual deviance - No Class model") +
  ylab("Residual deviance - UME model")

The dev-dev plot comparing residual deviance contributions for the UME model and no-class model shows that most data points lie close to the line of equality.

as.data.frame(sa_dic_RE) %>%
  arrange(desc(resdev)) %>%
  head(5)
#>          .study n_contrast   resdev  leverage      dic df
#> 1 EMMELKAMP2006          2 6.731005 0.9721711 7.703176  2
#> 2     ALDEN2011          1 6.610904 0.4682298 7.079134  1
#> 3  STANGIER2003          2 5.315711 1.0146986 6.330410  2
#> 4  VERSIANI1992          2 4.572550 1.1223400 5.694890  2
#> 5  HEIMBERG1998          3 4.251600 1.9468477 6.198447  3

We can see from investigating the pointwise contributions to the residual deviance that the two outlier studies are ALDEN2011 and EMMELKAMP2006 with their large resdev values in the no class model.

We can explore the consistency of specific evidence loops using node-splitting by setting consistency = "nodesplit". However, given the size of the network, running node-splitting on all comparisons would be computationally intensive. We instead run the node-splitting analysis specifically for the loops that include the studies identified in the dev-dev plots (ALDEN2011 and EMMELKAMP2006). We first define the specific treatment comparisons of interest by creating two data frames that specify the desired treatment comparisons. These are then passed to the nodesplit argument within the nma() function.

EMMELKAMP2006 <- data.frame(
  Treatment_1 = c("CBT individual", "Waitlist", "Waitlist"),
  Treatment_2 = c("Psychodynamic psychotherapy", "Psychodynamic psychotherapy", "CBT individual")
)

ALDEN2011 <- data.frame(
  Treatment_1 = c("Waitlist"),
  Treatment_2 = c("CBT group")
)

sa_fit_RE_nodesplit_EMMELKAMP <- nma(sa_net,
                           consistency = "nodesplit",
                           nodesplit = EMMELKAMP2006,
                           trt_effects = "random",
                           prior_trt = normal(0, 100),
                           prior_het = half_normal(5),
)
#> Fitting model 1 of 3, node-split: Psychodynamic psychotherapy vs. CBT individual
#> Fitting model 2 of 3, node-split: Psychodynamic psychotherapy vs. Waitlist
#> Fitting model 3 of 3, node-split: CBT individual vs. Waitlist

sa_fit_RE_nodesplit_ALDEN <- nma(sa_net,
                             consistency = "nodesplit",
                             nodesplit = ALDEN2011,
                             trt_effects = "random",
                             prior_trt = normal(0, 100),
                             prior_het = half_normal(5),
)
#> Fitting model 1 of 1, node-split: CBT group vs. Waitlist
summary(sa_fit_RE_nodesplit_ALDEN)
#> Node-splitting model fitted for 1 comparison: CBT group vs. Waitlist.
#> 
#>        mean   sd  2.5%   25%   50%   75% 97.5% Bulk_ESS Tail_ESS Rhat
#> d_dir -0.89 0.13 -1.16 -0.98 -0.89 -0.80 -0.63     4045     3309    1
#> d_ind -0.73 0.14 -1.00 -0.82 -0.73 -0.63 -0.43     1242     1896    1
#> omega -0.17 0.19 -0.53 -0.29 -0.17 -0.04  0.19     2045     2157    1
#> tau    0.21 0.03  0.15  0.18  0.21  0.23  0.27     1288     2380    1
#> 
#> Residual deviance: 162.8 (on 147 data points)
#>                pD: 95.3
#>               DIC: 258.1
#> 
#> Bayesian p-value: 0.37
summary(sa_fit_RE_nodesplit_EMMELKAMP)
#> Node-splitting models fitted for 3 comparisons.
#> 
#> --------------------- Node-split Psychodynamic psychotherapy vs. CBT individual ---- 
#> 
#>       mean   sd  2.5%   25%  50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> d_dir 0.71 0.33  0.06  0.49 0.71 0.93  1.37     4935     3566    1
#> d_ind 0.45 0.26 -0.07  0.27 0.45 0.62  0.97     3593     3100    1
#> omega 0.26 0.39 -0.48 -0.01 0.26 0.51  1.06     4076     3187    1
#> tau   0.21 0.03  0.15  0.19 0.21 0.23  0.27     1264     2040    1
#> 
#> Residual deviance: 161 (on 147 data points)
#>                pD: 95.3
#>               DIC: 256.3
#> 
#> Bayesian p-value: 0.51
#> 
#> --------------------------- Node-split Psychodynamic psychotherapy vs. Waitlist ---- 
#> 
#>        mean   sd  2.5%   25%   50%   75% 97.5% Bulk_ESS Tail_ESS Rhat
#> d_dir -0.60 0.19 -0.96 -0.73 -0.59 -0.47 -0.23     6521     3097    1
#> d_ind -0.69 0.41 -1.50 -0.96 -0.68 -0.41  0.14     4026     2534    1
#> omega  0.09 0.45 -0.80 -0.21  0.09  0.39  1.00     4470     2522    1
#> tau    0.21 0.03  0.15  0.19  0.21  0.23  0.28     1225     1829    1
#> 
#> Residual deviance: 162.5 (on 147 data points)
#>                pD: 95.8
#>               DIC: 258.3
#> 
#> Bayesian p-value: 0.85
#> 
#> ---------------------------------------- Node-split CBT individual vs. Waitlist ---- 
#> 
#>        mean   sd  2.5%   25%   50%   75% 97.5% Bulk_ESS Tail_ESS Rhat
#> d_dir -0.84 0.24 -1.31 -1.00 -0.84 -0.67 -0.35     5024     3598    1
#> d_ind -1.46 0.24 -1.93 -1.62 -1.46 -1.30 -1.00     1288     1950    1
#> omega  0.62 0.34 -0.03  0.39  0.62  0.84  1.29     2256     2578    1
#> tau    0.20 0.03  0.15  0.18  0.20  0.22  0.26     1040     1541    1
#> 
#> Residual deviance: 162.1 (on 147 data points)
#>                pD: 94.3
#>               DIC: 256.4
#> 
#> Bayesian p-value: 0.064

Our node‐splitting analysis shows that the CBT individual and Waitlist comparison has a p-value of 0.065 and a stronger effect for the indirect estimate compared to the direct estimate, however the CrIs do overlap and both estimates are in the same direction and do not include 0. This indicates no evidence of inconsistency in the network meta‐analysis. However, studies should be reviewed and sensitivity analyses are advised.

STEP 3: Class effects assessment

In this step, we assess whether a class assumption is suitable for the data by comparing our NMA RE model with an exchangeable class model with a RE treatment model. In this model, we specify class_effects = "exchangeable", meaning that treatments within each class are considered to be from a common distribution. The prior_class_sd sets a prior on the standard deviation between treatment effects within each class. Because it is difficult to estimate the class standard deviations with only a small number of treatments in each class, here we specify an informative \(\mathrm{N}(0.33,0.1^2)\) prior distribution that is chosen to keep class variability within a clinically-plausible range (Perren et al., in preparation). We also define class_sd as a list of character vectors indicating which classes share a common SD, again aiding estimation of these parameters. For the class effects means, prior_class_mean is set as \(\mathrm{N}(0, 10^2)\).

sa_fit_EXclass_RE <- nma(sa_net,
                         trt_effects = "random",
                         prior_trt = normal(0, 100),
                         prior_het = half_normal(5),
                         class_effects = "exchangeable",
                         prior_class_mean = normal(0, 10),
                         prior_class_sd = normal(0.33,0.1),
                         class_sd = 
                           list(`Exercise and SH no support` = 
                                  c("Exercise promotion", "Self-help no support"),
                                `SSRIs and NSSA` = 
                                  c("SSRI/SNRI", "NSSA"),
                                `Psychodynamic & Other psychological therapies` =
                                  c("Psychodynamic psychotherapy", "Other psychological therapies"))
                         )

We compare model fit from the Exchangeable class RE model and the no class RE model using the dic() function and \(\tau\).

(sa_dic_EXclass_RE <- dic(sa_fit_EXclass_RE))
#> Residual deviance: 162.6 (on 147 data points)
#>                pD: 87.4
#>               DIC: 249.9
(sa_dic_RE <- dic(sa_fit_RE))
#> Residual deviance: 162.1 (on 147 data points)
#>                pD: 94.5
#>               DIC: 256.6

summary(sa_fit_RE, pars = "tau")
#>     mean   sd 2.5%  25%  50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau 0.21 0.03 0.15 0.19 0.21 0.23  0.27     1137     2163    1
summary(sa_fit_EXclass_RE, pars = "tau")
#>     mean   sd 2.5%  25% 50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau  0.2 0.03 0.14 0.18 0.2 0.22  0.26     1371     2427    1

We use the plot() function that produces a “dev-dev” plot of the residual deviance contributions for the Exchangeable class model plotted against the no class model.

plot(sa_dic_EXclass_RE, sa_dic_RE, show_uncertainty = FALSE) +
  xlab("Residual deviance - Exchangeable Class model") +
  ylab("Residual deviance - No Class model")

Although both models yield similar residual deviance (162.6 vs. 162.1) and between‐study heterogeneity, the exchangeable class model reduces model complexity—reflected by a lower effective number of parameters ( vs. 94.5) and a lower DIC (249.9 vs. 256.6). A deviance–deviance plot confirms that all data points fit equally well. Overall, these findings support using the class model for the social anxiety data.

STEP 4: Class Model Assessment

After we have determined that a class effects model is appropriate, it is time to finalise which combination of common or exchangeable class effects and fixed or random treatment effects provides the most suitable model fit. As we are using a random effects model, we now fit 2 other models to our data; Exchangeable class FE and Common class RE. This gives us a total of 3 models to compare. Exchangeable class FE, Exchangeable class RE and Common class RE.

sa_fit_COclass_RE <- nma(sa_net,
                         trt_effects = "random",
                         prior_trt = normal(0, 100),
                         prior_het = half_normal(5),
                         class_effects = "common")

sa_fit_EXclass_FE <- nma(sa_net,
                         trt_effects = "fixed",
                         prior_trt = normal(0, 100),
                         prior_het = half_normal(5),
                         class_effects = "exchangeable",
                         prior_class_mean = normal(0, 10),
                         prior_class_sd = normal(0.33,0.1),
                         class_sd = 
                           list(`Exercise and SH no support` = 
                                  c("Exercise promotion", "Self-help no support"),
                                `SSRIs and NSSA` = 
                                  c("SSRI/SNRI", "NSSA"),
                                `Psychodynamic & Other psychological therapies` = 
                                  c("Psychodynamic psychotherapy", "Other psychological therapies"))
                         )
#> Warning: Tail Effective Samples Size (ESS) is too low, indicating posterior variances and tail quantiles may be unreliable.
#> Running the chains for more iterations may help. See
#> https://mc-stan.org/misc/warnings.html#tail-ess

We compare model fit from all three models using the dic() function and evaluate \(\tau\) in the common class RE model and the exchangeable class RE model.

(sa_dic_COclass_RE <- dic(sa_fit_COclass_RE))
#> Residual deviance: 159.1 (on 147 data points)
#>                pD: 93.8
#>               DIC: 252.9
(sa_dic_EXclass_FE <- dic(sa_fit_EXclass_FE))
#> Residual deviance: 284.7 (on 147 data points)
#>                pD: 34.2
#>               DIC: 318.9
(sa_dic_EXclass_RE <- dic(sa_fit_EXclass_RE))
#> Residual deviance: 162.6 (on 147 data points)
#>                pD: 87.4
#>               DIC: 249.9

summary(sa_fit_COclass_RE, pars = "tau")
#>     mean   sd 2.5%  25%  50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau 0.25 0.03  0.2 0.23 0.25 0.27  0.31     1117     1994    1
summary(sa_fit_EXclass_RE, pars = "tau")
#>     mean   sd 2.5%  25% 50%  75% 97.5% Bulk_ESS Tail_ESS Rhat
#> tau  0.2 0.03 0.14 0.18 0.2 0.22  0.26     1371     2427    1

Both random‐effects models showed substantially better fit than the fixed‐effect model (posterior mean residual deviance of 159.1 and 162.6 versus 284.7, and DIC values of 252.9 and 249.9 versus 318.9). Although the common class model achieved a slightly better absolute fit, it had higher between‐study heterogeneity (0.25 vs. 0.20) than the exchangeable class model. We use the plot() function to produce a “dev-dev” plot of the residual deviance contributions for the Exchangeable class model plotted against the no class model.

plot(sa_dic_COclass_RE, sa_dic_EXclass_RE, show_uncertainty = FALSE) +
  xlab("Residual deviance - Common Class model") +
  ylab("Residual deviance - Exchangeable Class model")

The dev–dev plot indicates that while some data points favor one model over the other, overall the fit is similar. Therefore, given nearly identical DIC scores, the choice between the two random‐effects models should be guided by clinical considerations: use the exchangeable class model if reporting distinct treatment‐level effects is important, or the common class model if the focus is on class-level effects and not concerned with potential variability of treatment effects within classes.

Results for final chosen model

Relative treatment effects

Relative treatment effects are produced with the relative_effects() function, and can be plotted using plot().

plot(relative_effects(sa_fit_EXclass_RE), ref_line = 0)

Relative class effects

We can produce a plot showing relative class effects against Waitlist using plot() with pars = "class_mean" to select these parameters.

plot(sa_fit_EXclass_RE,
     pars = "class_mean",
     ref_line = 0)

We can combine the treatment and class effects into a single plot as follows:

# Relative treatment effects
trt_eff <- as_tibble(relative_effects(sa_fit_EXclass_RE)) %>% 
  # Add in class details
  mutate(Class = sa_net$classes[as.numeric(.trtb)],
         level = "treatment")

# Class effects
class_eff <- as_tibble(summary(sa_fit_EXclass_RE, pars = "class_mean")) %>% 
  # Extract class details
  mutate(Class = factor(gsub(".*\\[(.+)\\]", "\\1", parameter), levels = levels(sa_net$classes)),
         level = "class",
         .trtb = factor("Class Mean", levels = c(levels(sa_net$classes), "Class Mean")))

# Combine and plot
bind_rows(trt_eff, class_eff) %>% 
  ggplot(aes(y = .trtb, 
             x = mean, xmin = `2.5%`, xmax = `97.5%`,
             colour = level, shape = level)) +
  geom_vline(xintercept = 0, colour = "grey60") +
  geom_pointrange() +
  facet_grid(rows = "Class", scales = "free", space = "free", labeller = label_wrap_gen(22)) +
  scale_shape_manual(values = c(15, 16), guide = guide_none()) +
  scale_colour_manual(values = c("#113259", "#55A480"), guide = guide_none()) +
  xlab("SMD") + ylab("") +
  theme_multinma() +
  theme(strip.text.y = element_text(angle = 0))

We aim to visualize the class ranks and rank probabilities by summarizing their distributions. To achieve this, we extract the posterior draws for class means, add a reference treatment, and compute the rank of each treatment at every iteration. We then summarize these ranks by calculating the median and 95% credible intervals to generate a point-range plot and derive the rank probability distributions for each treatment class, which we then plot.

# Class means
EXclass_mean <- as.matrix(sa_fit_EXclass_RE, pars = "class_mean")

EXclass_mean <- cbind(`d[Reference]` = 0, EXclass_mean)

# Take ranks at each iteration
EXranks <- t(apply(EXclass_mean, 1, rank))

# Get median rank and 95% credible interval
EXresults <- t(apply(EXranks, 2, quantile, probs = c(0.025, 0.5, 0.975)))

# Convert to data frame
EXresults_df <- as.data.frame(EXresults)
EXresults_df$class <- rownames(EXresults_df)

EXresults_df$class <- factor(EXresults_df$class, 
                             levels = sort(unique(EXresults_df$class), decreasing = TRUE))

ggplot(EXresults_df, aes(x=class, y=`50%`, ymin=`2.5%`, ymax=`97.5%`)) +
geom_pointrange(size = 0.5) +
coord_flip() +
xlab("Class") + ylab("Posterior Ranks") +
theme_multinma()

Treatment classes with higher ranks have median values closer to 1, positioned toward the left of the x-axis. The Combined treatment class has the highest median rank, followed by CBT individual. In contrast, Exercise promotion appears to have the lowest rank among the treatment classes, aside from the reference.

We now calculate and plot the class rank probabilities.

EXranks_df <- as.data.frame(EXranks)

# Rank probabilities for class
rank_probs_EX <- apply(EXranks_df, 2, function(x) table(factor(x, levels = 1:ncol(EXranks_df))) / nrow(EXranks_df))

# Convert to data frame
rank_probs_df_EX <- as.data.frame(rank_probs_EX)

# Convert the data frame to a long format
rank_probs_long_EX <- rank_probs_df_EX %>%
  mutate(Rank = row_number()) %>%
  pivot_longer(
    cols = -Rank,
    names_to = "Class",
    values_to = "Probability"
  )

# Plot density
ggplot(rank_probs_long_EX, aes(x = Rank, y = Probability)) +
  geom_line() +
  facet_wrap(~ Class) +
  theme_multinma() +
  labs(x = "Rank",
       y = "Probability")

Higher-ranked classes have a greater density on the left side of the x-axis, indicating a higher probability of achieving a top rank across posterior samples. The Combined class shows the highest density on the left, suggesting it is most likely to be ranked among the top treatments, followed by CBT individual. In contrast, most other classes have relatively flat distributions, reflecting greater uncertainty in their rankings. Exercise promotion, Other psychological therapies, and the reference class have densities skewed toward the right, indicating a higher probability of lower rankings.

References

Mayo-Wilson, Evan, Sofia Dias, Ifigeneia Mavranezouli, Kayleigh Kew, David M Clark, AE Ades, and Stephen Pilling. 2014. “Psychological and Pharmacological Interventions for Social Anxiety Disorder in Adults: A Systematic Review and Network Meta-Analysis.” The Lancet Psychiatry 1 (5): 368–76.
Perren, S. J., H. Pedder, N. J. Welton, and D. M. Phillippo. In preparation. “Network Meta-Analysis with Class Effects: A Practical Guide and Model Selection Algorithm.”