Having Access to More Studies Does Not Lead to a Better Understanding of Medical Evidence

In Uncategorized by Michael WillisLeave a Comment

Zung Vu Tran PhD

Zung Vu Tran PhD is a leading authority in bio statistics and meta-analysis

Presently, “systematic literature review,” followed by an analysis of the data derived from this literature review (Meta-Analysis) paints a myopic view of medical evidence. Keyword-based searches miss much of the available literature. And, depending on the area of research, the volume of available literature can be staggeringly large.

Medical Journals

There are over 23,000 journals publishing over one million medical research articles every year

Traditional Meta-Analysis

Current practices in quantitative literature review have evolved from when Dr. Gene V Glass published a methodology he called “meta-analysis” in 1976 and when I published the first meta-analysis in a medical journal (JAMA), in 1985.

Glass advocated that literature reviews should include all the scientific evidence. That is, there should be no a priori exclusion of evidence based on the reviewer’s subjective impression of what data were worth examining.

Since then, the number of “meta-analysis” publications currently available is disturbing.

Dr. John Ioannidis has indicated that, “the production of systematic reviews and meta-analyses has reached epidemic proportions. … the large majority of produced systematic reviews and meta-analyses are unnecessary, misleading, and/or conflicted,” and that, “probably more systematic reviews of trials than new randomized trials are published annually.”

Given the ease of access, the volume of data available from any keyword search (e.g., PubMed), and the proliferation of confusing meta-analyses, there is a need for a system that enables investigators to comprehensively review, extract, and aggregate all the evidence in the medical literature.

A New Approach to Meta-Analysis

Human Intelligence + Artificial Intelligence = Higher Intelligence

To address the gap between the amount of available scientific data and the ability to analyze it in its entirety, MedAware Systems, Inc. has developed a process where data extraction is complete, remarkably rapid, and accurate.

This new approach requires two medical research scientists, blinded to each other, to extract data from the same study. Intelligent software guides the rapid initial extraction, then compares each data field for matches (or mismatches).

For mismatches, a senior scientist reviews and reconciles the discrepancy. This combination of human intelligence with “intelligent” software to guide and verify the data curation process results in a complete and precise extraction of data from all available studies on a medical topic.

Those data are then organized and standardized in a database which is continuously updated and maintained.

A Case Study – Alzheimer’s Disease

The volume of Alzheimer’s disease (AD) literature is large and difficult to interpret. However, results produced using MedAware Systems’ integrated process provide far more benefit and value to the researcher, in an ongoing capacity, that no traditional approach can duplicate.

The objective of any Alzheimer’s disease (AD) treatment is to achieve clinically meaningful symptom reductions without compromising the quality of life (QOL) in patients.

MedAware Systems’ literature database and meta-analytic methodology readily profile the diversity, efficacy, and ADL/QOL effects of leading pharmacologic and non-pharmacologic AD treatments:

  • Dynamic characterization and comparison of multiple treatment modalities and outcome types, based on all available scientific evidence.
  • The results represent the most comprehensive evaluation of the effects of leading AD drugs and nonpharmacologic therapies.
  • The findings provide a framework for further studies on defining appropriate efficacy measures.

Despite the promise of emerging therapies, existing pharmacologic treatments have dominated the AD treatment landscape. The acceptance of these treatment norms, especially in comparison to nonpharmacologic treatments, has not been adequately examined.

With this in mind, we set out to identify the complete domain of AD literature; extract all intervention, efficacy, and cohort-level data; and profile the differences in treatment effect across treatment classes and outcome types.

The first step was to codify and organize the complete domain of published, peer-reviewed neurology publications, currently containing over 9,000 papers. There was a total of 905 clinical studies reporting AD, dementia, or cognitive impairment interventions and outcome data. Of those, 340 and 125 publications were identified that investigating drug or non-pharmacologic treatments, respectively, and were included in the analysis.


Pharmacological treatments were classified as Acetylcholinesterase Inhibitor (AChEI), NMDA Receptor Inhibitor, and Anti-Psychotic/Anti-Depressant. Non-Pharmacologic therapies were classified as Physical Activity, Cognitive Therapy, and Integrative (Physical and Cognitive).

Efficacy was defined as the difference in pre- and post-treatment cognitive function scores.

Functional outcome was defined as the difference in pre- and post-treatment quality of life (QOL) scores, including the Activities of Daily Living (ADL) scales.

Study duration was limited to 26 weeks.

No significant difference in reduction of cognitive decline or improvement in QOL/ADL scores was demonstrated between pharmacologic and non-pharmacologic treatments. While acetylcholinesterase inhibitors showed the least overall cognitive benefit, the drug class was associated with significant benefit in ADL/QOL, and the greatest variability in effect

Within non-pharmacologic treatments, integrative therapies showed the poorest efficacy and suggested conserved ADL/QOL with cognitive therapies.

This investigation represents the most comprehensive evaluation of the effects of leading AD drugs and non-pharmacologic therapies.

The findings provide a framework for further studies on defining appropriate efficacy measures and developing effective combination therapies.


Excluding a significant volume of evidence does not advance medical science.

This new, comprehensive approach is critical to informed provider, payer, and research decision-making.

The database is not primarily intended for use in performing meta-analyses; rather, it enables a researcher to access all the evidence, in the form of data extracted from publications of all kinds, and to decide for themselves what the state-of-the- art might be.

Our methodology also provides the possibility of checking why meta-analyses on the same topic reach different conclusions.

Please follow and like us: