The part of research nobody puts in the headline

When a news story says "a new analysis found", it almost never tells you how the analysis decided what to look at. That step is where a lot of the real quality lives, and it is worth ten minutes of your attention because once you understand it, you can read any health claim differently.

Ahmad and Sabbour searched five databases: PubMed/MEDLINE, ProQuest, Scopus, CINAHL and the first 100 results in Google Scholar. They searched from the start of each database up to May 2023.

That search returned 1,445 articles.

They ended up using 13.

How you get from 1,445 to 13

The filter looked like this, and each step tells you something.

First, titles and abstracts were screened by both authors independently. Two people reading the same pile separately is a basic guard against one person's enthusiasm or blind spots steering the whole review.

Then they went hunting in what researchers call grey literature, material outside the main journals, and added 18 more candidates.

Ninety full papers were pulled and read properly. Seventy-seven were thrown out for not meeting the inclusion rules. Thirteen made it.

And even then, only 8 of the 13 were usable for the actual number crunching. The other five got dropped from the maths because one had no mean values, two followed patients for under six months, and two only reported changes without the starting and ending numbers.

So when you later read that "the combination reduced HbA1c by 1.48 points", that figure is coming from a subset of a subset. That is not a criticism. It is simply what the evidence base looked like.

The rules they set before they looked

The detail I like most in this paper is boring on purpose. The review protocol was registered on PROSPERO, the international register for systematic reviews, under number CRD42023434707.

Why should you care? Because registering in advance means you commit to your question and your methods before you see the results. It makes it much harder to quietly move the goalposts once the data comes in. A review that registered its plan is telling you "hold us to this". A review that did not is asking you to trust it.

They also followed the MOOSE reporting guideline, which exists specifically for meta-analyses of observational studies, and used the PRISMA flow chart to document every exclusion.

The inclusion rules themselves were:

  • Adults 18 and over with type 2 diabetes
  • Real-world observational studies: cohort or case-control designs
  • Published in English
  • Followed patients for at least six months, or compared the combination to single drugs at six months

Trials, conference abstracts, case reports and other reviews were excluded.

What the thirteen studies actually were

This is where it gets interesting, because the 13 studies are wildly different from each other.

The smallest had 15 patients, a single-centre study from India. The largest was a global electronic health record dataset (Riley and colleagues, drawing on TriNetX) with 2.2 million people. In between sat a Danish national registry of 66,807 people followed for up to 20 years, and a handful of Spanish, Brazilian, Croatian, Korean, British and American clinic cohorts in the low hundreds.

Patients were aged roughly 50 to 70 on average. Between about 44% and 66% were men. Many had high blood pressure, high cholesterol and obesity. Plenty were also on metformin, insulin or sulfonylureas at the same time.

Every study was scored on the Joanna Briggs Institute checklist, an 11 question quality test covering things like whether the groups were comparable, whether confounding factors were identified and dealt with, and whether follow-up was complete. Ten studies scored in the 6 to 8 range. Three scored 5 or lower, which the authors describe as low quality.

Why this matters to you

Here is the takeaway I want you to carry out of this part.

When someone tells you a treatment "works", the useful follow-up question is not "is it true?" It is "how many studies, how big, how good, and who picked them?"

This paper answers all four in the open. Thirteen studies. Sizes from 15 to 2.2 million. Mostly middling quality, three weak. Picked by two authors working independently against rules they registered beforehand.

That transparency is the positive story here. It lets you, a non-specialist, judge how much weight the conclusions can bear. Most health claims you run into online give you none of that.

In Part 3 we get to the result that makes people sit up: the mortality number. Keep the "subset of a subset" idea in your head when you read it.

This series discusses a peer-reviewed meta-analysis of observational studies. Nothing here is medical advice. Do not start, stop or combine prescription medications without your doctor.