|
 |
|
EDITORIAL |
|
|
|
| Year : 2021 | Volume
: 26
| Issue : 3 | Page : 139-143 |
| |
A perspective upon systematic review and meta-analysis
Prabudh Goel
Department of Paediatric Surgery, All India Institute of Medical Sciences, New Delhi, India
| Date of Submission | 18-Mar-2021 |
| Date of Acceptance | 20-Mar-2021 |
| Date of Web Publication | 17-May-2021 |
Correspondence Address:
Dr. Prabudh Goel
Department of Paediatric Surgery, Room No. 4002, 4th Floor, Teaching Block, All India Institute of Medical Sciences, New Delhi - 110 029
India
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/0971-9261.316103
How to cite this article:
Goel P. A perspective upon systematic review and meta-analysis. J Indian Assoc Pediatr Surg 2021;26:139-43 |
The medical ethos has moved from 'authoritarian medicine' to 'authoritative medicine.' The 'democratic' approach to medical decision-making has imparted value to evidence over experts' opinions. The scientific methods with respective rational explanations and objective observations have replaced belief, tradition, authority, and superstition. With this understanding, the different tools of collecting evidence can be organized into a hierarchical pyramid [Figure 1].[1] The case reports and case series have evolved into cross-sectional, case–control, and cohort studies which have been superseded by the randomized controlled trials (RCTs). The RCTs have been considered the gold standard for generating evidence pertaining to efficacy of interventions. | Figure 1: The evidence-based hierarchy pyramid for various study designs in medicine
Click here to view |
Over the years, the body of medical literature has assumed overwhelming dimensions. The PUBMED database, maintained by the National Centre for Biomedical Information (NCBI) alone contains more than 32 million citations and abstracts of biomedical literature.[2] Narrowing our domain to pediatric surgery, a preliminary PUBMED interrogation for any common ailment such as “undescended testis” would yield 11,000 results.[3] Similarly, “vesico-ureteric reflux” yields 9000 results approximately.[4] Most of the databases pertaining to medical research offer a portal for focused, systematic inquiry into specific research questions. In return, they would “platter” the results from a multitude of studies representing population from diverse geographies or ethnicities conducted over varied timeframes by different researchers through similar or related research designs. More often than not, these studies would provide mutually conflicting opinions upon the same research question. The end user is entrusted with the responsibility of systematically locating, appraising, and using the contemporaneous research findings to make conscientious, explicit, and judicious clinical decisions. The evolutionary migration from libraries and print versions of manuscripts to hand-held devices and indexing softwares has played the devil's advocate. The clinical evidence is available at a click, yet the body of medical literature has expanded to “chaotic” magnitudes. As Davidoff pointed out, the inquisitive clinician sometimes finds himself, “starving for information in the midst of plenty.[5]”
Kundra et al.[6] conducted a study to compare the analgesic effect of penile block vis-à-vis caudal epidural in hypospadias repair. They reported that all the urethra-cutaneous fistula (UCF) in the study cohort were observed in the caudal epidural group. This single study stirred the hypospadiologists world over. In a short span of time, there were six more reports from different countries of which only three reported a positive association between UCF formation after hypospadias repair and administration of caudal block.
A young pediatric urologist exploring the safety of caudal block in hypospadias repair was confronted with mutually conflicting results from seven different studies addressing the same research question. A systematic review and meta-analysis of the seven studies was conducted to pool the individual data and generate a common conclusion.[6]
Systematic reviews and meta-analyses entail evaluation and synthesis of evidence from previously gathered information as against primary research which collects and reports data directly from the patients or populations.
 Systematic Reviews | |  |
The concept of reviewing the available evidence on any particular research question systematically was mooted by Archibald Cochrane.[7] As the name implies, a systematic review is aimed at generating an exhaustive summary of available literature relevant to a research question. It is a systematic search for data to synthesize evidence in favor of or against a research hypothesis; the research methods are formal, structured, and free of bias.
A systematic review could be applied to any precisely defined research question irrespective of the aspect of the disease being scrutinized. The question could be related to etiology (if there is a role of androgen exposure antenatally and hypospadias), pathogenesis (the anogenital distance in hypospadias patients vis-à-vis control population), clinical presentation (association of chordee with different sub-types of hypospadias), post-operative results (association of UCF formation with the type of surgical procedure, use of stent or technique of dressing after surgery), and long-term outcomes (quality of life or fertility issues in the long-term).
A systematic review of literature differs from traditional narrative reviews in being founded upon analytical methods which are explicit, reproducible, and transparent. The systematic review, by virtue of its design, encompasses the capability of being updated regularly.
| Meta-Analysis | | |
The term “meta-analysis” refers to the quantitative and formal, scientific tools, and techniques used to derive statistical conclusions after integrating the results from multiple primary studies addressing the same research question. Meta-analysis is an elective, subcomponent of systematic reviews and plays a vital role in “evidence-based medicine.” Meta-analysis occupies the apical position on the evidence-pyramid [Figure 1].
| Design of Component Studies | | |
It is suggested that study designs using rigorous methods to assess the effectiveness of any intervention should be included in a systematic review. Under ideal circumstances, only RCTs shall satisfy the eligibility criteria for inclusion in a review.
From the perspective of pediatric surgery and generalizable to any other surgical branch, conducting an RCT between two different surgical techniques is a difficult proposition. It has been observed that surgeons have a general disinclination toward RCTs while comparing two different surgical techniques. Humanly, it is not possible to perform a surgical procedure repeatedly with the same precision and without any variations. Comparing two surgical techniques performed by different surgeons may introduce additional confounding factors. It is difficult to master two techniques for the same procedure with the same dexterity. More often than not, different surgical techniques broadly designed for the same conditions are generally addressing minor variants or subtypes of the same condition. For instance, coronal hypospadias may be corrected either by tubularized incised plate urethroplasty or Mathieu flip-flap urethroplasty. Yet, a pediatric urologist might deploy both these techniques in different patients depending upon local anatomy. Similarly, distal penile hypospadias may be considered a homogenous subgroup under the umbrella of hypospadias, yet it is heterogenous when local anatomic features such as the degree of glandular development, width of the urethral plate, depth of navicular fossa, and presence of chordee vis-à-vis glandular tilt are taken into consideration. Blinding may not be feasible between different surgical techniques. Last but not the least, it is unethical to test a novel surgical technique against the established gold standard. In such situations, it might be pragmatic to include other study designs such as the case–control or cohort studies.
Research questions which are highly specific such as the association of UCF in hypospadias repair with administration of caudal analgesia may not have been subjected to multiple trials. Of the seven articles available for review, only one case was a prospective, double-blinded RCT, five were retrospective cohort studies, and one was a retrospective, nested case–control study.[6] Relaxing the inclusion criteria to include other study designs may be a necessity rather than luxury.
Similarly, there might be conditions such as Sirenomelia[8] which are rare and penile epidermal inclusion cyst[9] which are commoner but seldom reported. In the absence of a comprehensive source of information, a systematic review on the subject may be warranted. The publications available for review on either of these subjects are case reports and case series exclusively.
| Two Prerequisites to the Understanding of Systematic Review and Meta-Analysis | | |
Evidence synthesis through systematic reviews and meta-analysis is expected to provide summary estimates of the effects of interventions which may be used to formulate treatment guidelines or modify existing strategies in the interest of the patients. However, there are certain prerequisites which have to be underscored prior to undertaking or understanding systematic reviews.[10] Congruent to the current theme, the following merit discussion:
- Comparing apples with oranges: The research question has to be defined precisely. In ideal circumstances, homogenous studies should be included as a component of this analysis. Only the parameters which are consistent across individual studies may be considered in evidence confectioning. The basic rule of statistics prohibiting comparison of “apples to oranges” is valid in the context of systematic review and meta-analysis[11]
- Garbage-In, Garbage-Out (GIGO): The origin of the term GIGO can be traced back to a syndicated newspaper article published on November 10, 1957. Mellin coined the term in the context of computers which lack the power to think. Hence, “sloppily programmed” inputs would invariably lead to incorrect outputs.[12] The same holds true for systematic reviews and meta-analysis. The quality of a systematic review or meta-analysis is as good as the quality of studies included in its synthesis.
| Evolution of Evidence Synthesis: Systematic Reviews and Meta-Analysis | | |
Publishing experiences in medicine have been practiced for a long time. Sir William Osler,[13] the Father of Modern Medicine and one of the four founders of the John Hopkins Hospital stated, “Always note and record the unusual … Publish it. Save it on a permanent record as a short, concise note. Such communications are always of value.[14]” The Royal Society of London was probably the first public institution dedicated to experimental scientific research and knowledge. Philosophical Transactions, first published by the society in 1665 is the first and longest-running scientific periodical on record.[15] The quality and format of publications have been changing with the changing requirements over time. Subject nonspecific scientific periodicals were replaced by specialized publications. Guidelines were framed to standardize the format and style of publications which replaced the detailed descriptions by the concise Material and Methods. Newer guidelines have evolved over time while the existing ones have been modified periodically. Case reports were succeeded by case series which gave birth to original articles.
The need to synthesize research evidence has been recognized for nearly two centuries now.[16] James Lind, a Scottish naval surgeon published a “…Critical and Chronological View on what has been published…” on Scurvy in the 18th century.[17] However, critical appraisal of research findings and synthesis of evidence from primary research was formally guised as “meta-analysis” by Glass as late as 1975.[18] This has been followed by development of toolkits and systematic methods for appraising and collating evidence. The concept of systemic reviews and meta-analysis has progressed in leaps and bounds. The Cochrane collaboration was inaugurated in 1993 and the Cochrane Database of Systematic Reviews was established subsequently. What started as an agitation of Archie Cochrane for developing medicine based on RCTs in his seminal book, “Effectiveness and Efficiency: Random Reflections on Health Services” in 1972 has evolved into a global independent network of people interested in health such as researchers and professionals who are working to, “… gather and analyze the best available evidence to help people make informed decisions about health and health care.[19]” They have published more than 7500 systematic reviews till date on different aspects of health care.[20]
| Implications for Authors, Reviewers, Editors, and Journals | | |
The role of systematic reviews and meta-analysis as the next milestone of evidence-based medicine has been realized globally. Systematic reviews and meta-analysis assimilate a large amount of data, assess the studies for quality and reporting standards before comparing them formally and the derived conclusions are precise and prompt. The horizon of component studies has been liberalized; depending upon the research question, case reports and case series are also being considered for inclusion in the systematic review.[21] They occupy the apical position in the evidence-hierarchy pyramid [Figure 1] and a similar status in the authors' résumé. The number of systematic reviews in pediatric surgery is increasing dramatically with time, almost paralleling the rapid growth of biomedical literature.
[Recall: The quality of a systematic review is governed by the quality of its component studies.] Moreover, it has been observed that while conducting a systematic review, a fair number of studies on the research question have to be excluded or are included only partially since the parameters are not uniform across the entire spectrum of inclusion. This is particularly true for relatively uncommon diseases or phenomena which have been reported only scarcely, for instance, the case of pyopagus conjoined twins[22],[23] or pyelovenous backflow.[24]
In the nature, it is the 'Natural Selection' that drives the phenomena of speciation and evolution. What is considered a case report or a cohort study today, will be one of the pillars of a systematic review tomorrow. Therefore, it is imperative that the editors, reviewers, and authors must take note and the quality of publications be updated accordingly. For instance, the case reports have been popular since the time of Hippocrates.[25] Yet the CAse REport or CARE guidelines were developed in 2013 with the objective of supporting transparency and accuracy.[26] Over the years, it was realized that the focus of medical and surgical case reports was different and the CARE guidelines were not tailored for surgical cases. Hence, the Surgical Case Report (SCARE) guidelines were developed in 2016 and have been updated regularly (2018 and 2020).[27] Similarly, guidelines have also been framed for reporting surgical case series which too are updated periodically[28] or RCTs.[29]
The SCARE guidelines ensure all important aspects of the case are reported and there is a certain degree of homogeneity across reports from different authors. Such guidelines can ensure the aspects common to all reports such as a structured abstract, demographic details of the patient, past medical history, and applicable prognostic characteristics. However, it cannot ensure disease-specific aspects which hold higher relevance for a systematic review. PUBMED interrogation for reports on penile epidermal inclusion cyst identified 11 papers.[9] The duration of swelling, presence of keratin in the filling material, and the follow-up duration were not reported in 4, 6, and 5 of 11 reports, respectively. The concept, illustrated herein, with the help of case reports applies to other article types equally: case series, original articles, trials, description of operative techniques, etc.
The successful conduct of a systematic review also depends upon the breadth of the review. It is imperative that the reviewer should search for relevant studies across all possible databases. With a spurt in the number of agencies indexing biomedical or health-care publications, accessing all potential sources of information is becoming tedious, problematic, and expensive. Positive findings are more likely to be published in indexed journals vis-à-vis negative findings which are rendered unpublished or accepted in nonindexed journals.[30] The rise in the number of conferences annually, repetitive presentation of the same clinical material under different titles, and the absence of a universal or central repository for conference abstracts is yet another limitation. The inclusion of nonpeer reviewed research or “gray literature” from different archiving agencies such as medRxiv.org and bioRxiv.org for the purpose of review is yet another contentious issue.[31]
With the vitality, rising importance, and indispensability of systematic reviews and meta-analysis in the context of formulating or updating evidence-based treatment protocols, it is time to integrate the basics into the primary research itself which forms the foundation of the body of medical literature. The medical fraternity has to look beyond the broad guidelines and formulate review-friendly formats for reporting cases or studies. This will ensure that all parameters relevant from the perspective of systematic reviews and meta-analysis are addressed adequately. The higher will be the degree of homogeneity in reporting the primary research, the stronger will be the foundations for evidence synthesis. The provision of a single, central platform for indexing the complete body of medical research with provisioning for conference abstracts in addition to the standard publications and a consensus statement outlining the strategy for utilizing the best from the “gray” literature is the need of the day.
| References | | |
| 1. | Kouimtsidis Ch, John-Smith S, Kemp P, Ikkos G. Evidence based mental healthcare and service innovation: Review of concepts and challenges. Psychiatriki 2013;24:45-54.  |
| 2. | |
| 3. | |
| 4. | |
| 5. | Davidoff F. In the teeth of the evidence: The curious case of evidence-based medicine. Mt Sinai J Med 1999;66:75-83. |
| 6. | Goel P, Jain S, Bajpai M, Khanna P, Jain V, Yadav DK. Does caudal analgesia increase the rates of urethrocutaneous fistula formation after hypospadias repair? Systematic review and meta-analysis. Indian J Urol 2019;35:222-9. [ PUBMED] [Full text] |
| 7. | Stavrou A, Challoumas D, Dimitrakakis G. Archibald Cochrane (1909-1988): The father of evidence-based medicine. Interact Cardiovasc Thorac Surg 2014;18:121-4. |
| 8. | Tomar S, Goel P, Bajpai M. Sirenomelia: A case report and review of literature. J Indian Assoc Pediatr Surg 2020;25 (Suppl):s1-99. |
| 9. | Goel P, Bajpai M, Anand S, Das Prasenjit. Systematic review of epidermal inclusion cyst of penis in chidren. J Indian Assoc Pediatr Surg 2019;24 Suppl S1:1-89. |
| 10. | Linares-Espinós E, Hernández V, Domínguez-Escrig JL, Fernández-Pello S, Hevia V, Mayor J, et al. Methodology of a systematic review. Actas Urol Esp 2018;42:499-506. |
| 11. | Milojevic M, Sousa-Uva M, Durko AP, Head SJ. Mixing 'apples and oranges' in meta-analytic studies: Dangerous or delicious? Eur J Cardiothorac Surg 2018;53:1294-8. |
| 12. | |
| 13. | |
| 14. | Ortega-Loubon C, Culquichicón C, Correa R. The importance of writing and publishing case reports during medical training. Cureus 2017;9:e1964. |
| 15. | |
| 16. | Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval Health Prof 2002;25:12-37. |
| 17. | Cooper H, Hedges LV. The Handbook of Research Synthesis. New York: Russell Sage; 1994. |
| 18. | |
| 19. | |
| 20. | |
| 21. | Jeong W, Keighley C, Wolfe R, Lee WL, Slavin MA, Kong DC, et al. The epidemiology and clinical manifestations of mucormycosis: A systematic review and meta-analysis of case reports. Clin Microbiol Infect 2019;25:26-34. |
| 22. | Bajpai M, Khanna, Goel P. Complexities and challenges involved in the management of conjoint twins. J Indian Assoc Pediatr Surg 2019;24 Suppl S1:1-89. |
| 23. | Tomar S, Goel P, Khanna K, Bajpai M. Conjoined twins – Experience and insights gained during management of seven pairs from an apex institute. J Indian Assoc Pediatr Surg 2020;25 Suppl: s1-99. |
| 24. | Goel P, Bajpai M, Kandasamy D. An enigmatic route to the contralateral pelvicalyceal system on antegrade pyelogram. J Indian Assoc Pediatr Surg 2018;23:236-8. [ PUBMED] [Full text] |
| 25. | Nissen T, Wynn R. The history of the case report: A selective review. JRSM Open 2014;5:2054270414523410. Published 2014 Mar 12. doi:10.1177/2054270414523410. |
| 26. | Gagnier JJ, Kienle G, Altman DG, Moher D, Sox H, Riley D. The CARE guidelines: Consensus-based clinical case report guideline development. J Clin Epidemiol 2014;67:46-51. |
| 27. | Agha RA, Franchi T, Sohrabi C, Mathew G, Kerwan A; SCARE Group. The SCARE 2020 guideline: Updating Consensus Surgical CAse REport (SCARE) guidelines. Int J Surg 2020;84:226-30. |
| 28. | Agha RA, Sohrabi C, Mathew G, Franchi T, Kerwan A, O'Neill N, et al. The PROCESS 2020 guideline: Updating Consensus Preferred Reporting Of CasESeries in Surgery (PROCESS) guidelines. Int J Surg 2020;84:231-5. |
| 29. | |
| 30. | Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics 2018;74:785-94. |
| 31. | Chaabna K, Cheema S, Abraham A, Alrouh H, Mamtani R, Sheikh JI. Gray literature in systematic reviews on population health in the Middle East and North Africa: Protocol of an overview of systematic reviews and evidence mapping. Syst Rev 2018;7:94. |
[Figure 1]
| This article has been cited by | | 1 |
Role of preoperative blockade in pheochromocytoma–paraganglioma: A clinician's perspective |
|
| ManjiriP Karlekar, SabaS Memon, TusharR Bandgar | | Indian Journal of Endocrinology and Metabolism. 2022; 26(1): 30 | | [Pubmed] | [DOI] | | | 2 |
A bibliometric study of the pediatric inguinal hernia publication landscape from the web of science database |
|
| AnjanKumar Dhua, Mohit Garg, AshwaniKumar Mishra, Prabudh Goel, Vishesh Jain, DevendraKumar Yadav | | Journal of Indian Association of Pediatric Surgeons. 2022; 27(6): 689 | | [Pubmed] | [DOI] | |
|
|
|
|
