If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Synthesis of medical literature to determine the best treatment for a given problem is challenging, particularly when multiple options exist. Network meta-analysis (NMA) allows the comparison of different treatment approaches in a single, systematic review including treatments that have never been compared head-to-head. A key to understanding NMA is to focus on the network geometry showing the number of included studies and their relationships: different treatment options are illustrated as nodes. Lines between nodes represent direct comparisons. For nodes not directly compared, indirect effects may be determined by use of the property of transitivity. Limitations of NMA include heterogeneity, where variability among included studies biases pairwise comparisons, and consistency, if direct and indirect comparisons between treatments do not agree. In the end, NMA allows numeric ranking of the estimated effects of each treatment from most to least effective. A disadvantage of NMA ranking methods is that readers may focus overly on what treatment ranks best and focus insufficiently on the methods and results that determine the rankings. The reliability of the rankings requires consideration of the geometry and strength of the network, including evaluation of heterogeneity, consistency, and transitivity. The conclusion of an NMA requires scrutiny of the methods and results.
Comparative clinical trials comprise a small portion of the medical literature compared with case series.
However, in areas of clinical controversy in which numerous comparative trials exist, network meta-analysis (NMA) methods allow us to reach wider conclusions by allowing us to compare different treatment approaches in a single review.
An excellent example can be found in the current issue, where Forsythe, Lavoie-Gagne, Patel, Lu, Ritz, Chahla, Okoroha, Allen, and Nwachukwu synthesize Level I evidence regarding the “Efficacy of Arthroscopic Surgery in the Management of Adhesive Capsulitis: A Systematic Review and Network Meta-analysis of Randomized Controlled Trials.”
An excellent start to understanding an NMA is to focus on the graphically illustrated geometry of the “network” of included studies and their relationships. As can be seen in a simple example, illustrated in Figure 1, different treatment options (A, B, and C) are represented as “nodes.” Lines between nodes represent “direct” comparisons (i.e., studies in which different treatments were directly compared). The thickness of the lines represents the number of included clinical trials comparing the different treatments; this is also indicated numerically (4 and 9). A thicker line indicates that more trials (9) compared treatment B with treatment C, compared with the thinner line, where only 4 different trials compared treatment A with C.
The greater number of internodal connections, the stronger or more well-connected the network. A strong network includes ample connections between nodes. For example, the network indicated in Figure 1 would be stronger or more well-connected if there were 1 or more direct connections between A and B, which is not the case. Similarly, a strong network includes more comparisons between treatments, or thicker lines. Here, a sense of the potential of an NMA to deliver valid results can be gleaned. Well-connected networks indicate a diversity of direct treatment comparisons, resulting in greater potential validity. Less well-connected networks indicate a relatively lower number of head-to-head comparisons (i.e., more “indirect” comparisons) resulting in lower potential validity. Quite simply, in an ideal setting, an NMA geometry would show well-connected nodes held together by strong thick lines. As a network becomes less connected, the analysis becomes more reliant on indirect comparisons and a more cautious interpretation of the results is required.
To Interpret NMA, Consider Heterogeneity and Consistency
All systematic reviews of the literature (of which meta-analyses and NMA are subsets) result in articles being included or excluded according to “systematic” search methods. Once the included articles are identified, a vital step is to determine the heterogeneity (as opposed to agreement) among studies that are compared head-to-head. Typically, and unfortunately, even studies meeting strict inclusion criteria may be quite heterogeneous due to diversity in (just to mention a few examples of areas of variability) patient demographics, treatments, outcome measures, or time to follow-up. A clear example of heterogeneity of treatment is if a treatment were defined as “nonoperative” but if nonoperative treatment was immobilization in some studies and physical therapy in others. Both treatments are “nonoperative,” but obviously, the treatments are different or heterogeneous.
In NMA, heterogeneity applies specifically to pairwise comparisons that are directly compared (i.e., node to node such as A with C, or B with C, in Fig 1). Statistical heterogeneity can be quantified and the extent of clinical heterogeneity requires scrutiny (by authors, editors, and readers) when study data are pooled.
Consistency in an NMA refers to agreement between direct and indirect comparisons, noting that determination of indirect effects is dependent on the property of transitivity (reviewed below). Like heterogeneity, consistency can be quantified and requires scrutiny (by authors, editors, and readers) when study data are pooled.
The NMA network geometry illustrated in Figure 2 aids in understanding consistency. Again, treatment options (A, B, and C) are represented as “nodes,” lines between nodes represent “direct” comparisons, and the thickness of the lines represents the number of included clinical trials directly comparing the different treatments; this is also indicated numerically (2, 4, and 9). Next, for example, note that A and B can be compared directly (by synthesizing the 2 head-to-head trials) or indirectly (by comparing how treatment A compares with treatment C vs how treatment B compares with the shared node C). It follows that the indirect effect should be similar to the direct effect, and the extent to which the direct and indirect effect estimates agree describes the degree of “consistency” in a treatment network.
The Hallmark of NMA is Transitivity
Recall the transitive property of equality: if A = C and B = C, then A = B.
In NMA, and as best illustrated in Figure 1, if 2 treatments (A and B) have not been directly compared, but if both have been compared to a third (or control) group (C), then by relying on the assumption of transitivity, the 2 treatments can be indirectly compared. And as mentioned previously, a hallmark of NMA methods is the ability to derive estimates for treatment comparisons that have not been compared head-to-head. In addition, for an indirect comparison to hold, the common comparator should be consistent across trials.
The assumption of transitivity is reliant on uniformity among treatments, and among baseline risks and effect modifiers.
For example, studies reporting identical treatments may show different results because of different inclusion and exclusion criteria, resulting in different expected outcomes among diverse groups of patients (selection bias), or because of varying surgical expertise among the authors of the different studies (performance bias). As nodes become less transitive, NMA results and conclusions become biased and less reliable.
The Conclusion of an NMA Requires Careful Scrutiny
In the end, NMA methods allow quantitative synthesis of different studies investigating different treatments, and, ultimately, determination and reporting of a numeric ranking of the estimated effects of each treatment from most to least effective. The obvious goal is to determine which treatment has the greatest probability of ranking number one. Readers may be tempted to skip right to the conclusion to find out “the winner,” but rankings require consideration of the strength of the network including heterogeneity and consistency.
If readers were to take home a single point from this editorial, we recommend that: A disadvantage of NMA ranking methods is that readers may focus overly on what treatment ranked best and focus insufficiently on the methods and results that determine the rankings.
NMA on the Management of Adhesive Capsulitis
In this issue’s NMA on the management of adhesive capsulitis by Forsythe et al.,
a variety of treatments are included. The networks for range of motion and pain are well connected in many areas, and physical therapy and injection are the 2 most represented treatments. Several other treatments, however, are connected by thin lines, indicating a smaller number of trials contributing to these other treatments’ representation in the included data. As treatments supported by thin lines are under-represented, conclusions regarding these other treatments are less reliable, mandating a close examination of the effect of these treatments in the results section of the report.
The assumption of transitivity is well-addressed. The authors provide detailed information on the similarity of the treatments and patients that constitute the shared nodes in support of a transitive network and, here, more reliable conclusions. In addition, the network harbors a number of closed loops, indicating treatments that are informed by both direct and indirect evidence. Moreover, direct and indirect effects are similar, indicating relative consistency among the comparisons. Thus, the network presented shows significant strength.
NMA allows comparison of different treatment approaches in a single, systematic review including treatments that have never been compared head-to-head. NMA allows numeric ranking of the effectiveness of treatments, and it can be tempting to skip right to the conclusion to find out of the winner. However, NMAs provide information well beyond which treatment ranks the best. The reliability of the rankings requires consideration of the geometry and strength of the network, including evaluation of heterogeneity, consistency, and transitivity. The conclusion of an NMA requires scholarly scrutiny and nuanced consideration of the methods and results.