What does drinking do to your health? We can say two things with confidence:

Drinking is associated with lots of health problems.

Heavy drinking is bad for you.

Here’s a graph of some associations:

Someone who averages 10 drinks per day is 50 times more likely to get cirrhosis than someone who doesn’t drink at all (controlling for age, sex, and drinking history).

This looks bad, but there are two caveats. First, it doesn’t establish causality. It could be — if all you had was this figure — that cirrhosis causes hormonal changes that in turn create the urge to drink more.

But we do know that heavy drinking is bad. That’s partly because we know how alcohol causes problems. It causes cirrhosis by destroying liver cells. It causes cancer by getting converted to acetaldehyde and then damaging DNA. There are also randomized controlled trials (RCTs) that take heavy drinkers and get them to drink less. These inevitably show improved health (either health outcomes or biomarkers like blood pressure).

The second caveat is the little dip in relative risk for diabetes and heart disease around 1-2 drinks. Some people think alcohol is causing this dip. Lots of mechanisms have been proposed: Maybe it reduces inflammation. Or maybe it impairs the cells that build up plaques in arteries. Or maybe it creates a hormonal imbalance that changes blood pressure regulation. Or maybe it increases HDL cholesterol or insulin sensitivity or adiponectin levels.

Or, maybe alcohol doesn’t help diabetes and heart disease at all. Mathews et al. (2015) ¹ tried to model how alcohol affects the heart, ending up with this terrifyingly tangled figure.

Alcohol does a lot of different things and interacts with a lot of other factors. It’s great to try to unravel all this, but I don’t trust anyone who says they understand everything with confidence.

If alcohol doesn’t improve heart health, then why the dip? Well, it could just be that the same people who drink moderately are also more likely to exercise and eat well.

So we don’t know if moderate drinking is bad for you. It almost certainly causes harms like cancer, but it might help heart disease enough to offset those harms. In the US, around 20% of adults drink 1-2 drinks per day. Even if the effects are modest, the collective impact is huge. Second perhaps to caffeine, alcohol is humanity’s favorite drug. We need to know what it does.

This is the story of a trial that came close to answering this question and then exploded. At first, this looks like a simple story of corruption, but when you look closely, it’s a very complicated story of corruption.

You might be thinking, “What we need to do is compare the health of people who drink different amounts, while controlling for income, diet, education, exercise, et cetera.” The problem is that “controlling” for things is a dangerous business. It requires tons of different assumptions, like what you control for, how you code stuff, and how you model everything. For example, if you “control for exercise,” do you measure the number of hours people exercise each week? Should you distinguish different kinds of exercise? Reasonable people can disagree about these choices. For alcohol, reasonable people do disagree. Some, like Ronksley et al. (2011) ² find a strong association between moderate alcohol consumption and improved cardiovascular health, and argue that the association is likely causal. But others, like Goel et al. (2018)[^S. Goel, A. Sharma, & A. Garg, “Effect of Alcohol Consumption on Cardiovascular Health,” Current Cardiology Reports 20, no. 19 (2018).] are unconvinced and suggest there could still be confounding variables, while Wood et al. (2018) ³ do a meta-analysis of observational studies that suggests even small amounts of alcohol hurt cardiovascular health.

There’s also some recent research using Mendelian randomization, which suggests alcohol could be bad for cardiovascular health. The idea is that a variant of the ADH1B gene makes it hard to metabolize alcohol. People who have it drink less. If you assume that the gene is random in the population and that it’s causing reduced drinking, then you can treat it like a random assignment to drink less. Holmes et al. (2014) ⁴ did this and found that carriers of ADH1B had better cardiovascular health by every measure. This suggests alcohol makes cardiovascular disease worse, not better.

So what do we do? We take the long, slow, hard path:

1. Get a large group of people.

2. Tell some of them to drink moderately, tell the others not to drink at all.

3. Wait years, monitoring people to make sure they are actually drinking (or not) like they’re supposed to.

4. Follow up and see which group is healthier.

Lots of things make this difficult. Because the expected effects aren’t huge, you need a large group of people. Because culture and genetics vary, you need people from around the world. Because diseases take a long time to show up, you need to wait years. And imagine the challenge of telling people how much to drink and then making sure they follow your instructions.

An international effort monitoring thousands of people around the world for years — does that sound expensive?

Back in 2013, the NIH’s National Institute on Alcohol Abuse and Alcoholism (NIAAA) got interested in funding this. They figured it would cost on the order of $100 million for the full trial. This doesn’t seem crazy given the NIAAA’s $500 million annual budget, but the NIAAA has lots of other priorities and didn’t feel they had the money.

You know who has a lot of money, though? The alcohol industry. Worldwide, $85 million of booze is sold every 30 minutes. In principle, the industry could directly fund a study, but who would trust it?

In 2016, it looked like the NIAAA had found an elegant solution:

• Five alcohol companies would donate money for a trial.

• The NIH would ask researchers to send proposals for how they’d run a trial.

• The NIH would choose the scientifically best proposal, just like they do with any government-funded grant. The donors would have no influence on the process.

• To make the results trustworthy, there would be a “firewall”, with no communication between the industry and the research team.

Sounds promising. But if we go forward a couple of years, everything suddenly blows up.

June 15, 2018

What happened? You might imagine banal corruption, with cocaine and overseas bank accounts, but it’s nothing like that.

The real story is a much more interesting cocktail of science, academia, bureaucratic maneuvering, ambition, politics, capitalism, the “deep state,” secret emails, and slippery ethical slopes. It’s a huge stroke of luck that we know about any of this. You have to ask how often similar things happen and don’t blow up.

If you’re brave, you can read the 165-page report the NIH prepared before canceling the program. But I warn you: it’s mostly out-of-order redacted emails written by people who wanted to conceal what was happening. There’s an executive summary, but it’s written in a frustratingly bureaucratic style. There are also newspaper stories, but they don’t try to give the full timeline.

After way too much time reconstructing things, here’s the full story as best as I can tell.

2001-2013. Kenneth Mukamal, a physician at Beth Israel Deaconess Medical Center and faculty member at Harvard Medical School, published many papers that argue that moderate alcohol consumption has health benefits, usually for heart disease or diabetes. During the same period, John Krystal, a psychiatrist and professor at Yale, also published many papers on alcohol, mostly focusing on addiction and mental health. (Many other researchers were involved in this study, but these two were most prominent.)

Here’s a characteristic sample of Mukamal’s 189 papers on alcohol:

You may notice that all of them find that moderate drinking has health benefits. 

Early 2013. Some NIAAA staff were convinced that moderate drinking is good for you, and an RCT could prove it conclusively enough that doctors might recommend it to patients like they do with aspirin now. They had the idea of getting the alcohol industry to fund the study, but faced two problems. First, the alcohol industry wants lots of details before forking over any cash. Second, the NIAAA isn’t allowed to solicit from industry. They tried to get around these problems by having outside researchers (including Mukamal and Krystal) meet with industry to give details on how such a trial might work. This created a dynamic where everyone (the NIAAA, the alcohol industry, Mukamal) wanted to coordinate with each other, but maintain a pretense of being isolated. There was lots of scheming about how information should flow to maintain this pretense.

They settled on the strategy of having the industry make a “gift” to FNIH, the not-for-profit arm of the NIH that was set up to take industry money and then do NIH-stuff with it.

At the same time, they decided that they could get rid of the appearance of soliciting by getting an external researcher to make the case. They settled on Kenneth Mukamal. The record is silent on exactly why they chose Mukamal. My guess is that it was partly because of Mukamal’s pro-alcohol research record, and partly because it helped to overcome some apparent issues regarding collaborations between Harvard and Beth Israel (BI).

The NIAAA wanted someone else to present the idea of the study to overcome their prohibition of solicitation, even though they’d obviously set this whole thing in motion. The alcohol industry was excited about what they heard directly from the researcher, but wanted the plan to come “from NIAAA.”

July 12, 2013. The NIAAA published NOT-AA-13-004. This was a “planning grant,” which basically means that the NIH would give you some money to do some work that would allow you to successfully submit a much larger grant soon. By NIH rules, this was a public opportunity, meaning any researcher could submit and win the grant if they had the best science. Yet they obviously wanted “their” PI to win:

They stacked the deck in three ways. First, they asked for an extra-short deadline, and said that applications would need to get preapproval before submitting a grant. Both of these tricks were overruled by NIH central, though “prior consultation” was still “strongly encouraged.” Second, rather than a typical open-ended call for research, they asked for a specific trial to be done — coincidentally exactly the trial Mukamal wanted to do. Third, NIAAA staff decided to physically travel to Boston to help Mukamal write the grant. Since this was totally forbidden, they went another way.

All the scheming from the NIAAA worked. Ultimately, they received exactly one application: from Mukamal. 

November 21, 2013. There is a meeting at the Distilled Spirits Council in Washington, DC between the alcohol industry, the NIAAA, and three researchers, including Mukamal and Krystal. Someone from industry later reported to NIAAA staff that “he was tremendously enthused about the project” and that they would need similar meetings with other companies. He specifically wanted to hear more from “the guy from Harvard and the guy from Yale” — in other words, Mukamal and Krystal.

According to The New York Times, representatives of beverage conglomerates Anheuser-Busch InBev, Heineken, and Diageo later confirmed that these meetings were important for their decision to go ahead and fund the trial.

January 2014. The preliminary planning grant was reviewed. One reviewer was concerned about the alcohol industry, but NIAAA staff were able to exclude the reviewer from voting on procedural grounds. When responding to reviewer comments, Mukamal stated that he “tried to be discrete about the industry stuff.” The grant was formally awarded on March 20, 2014. There was a parallel conference grant that was also successfully steered to Mukamal at the same time.

February 26, 2014. There was  a meeting in Palm Beach, Florida, including alcohol industry representatives, at least one NIAAA staffer, and outside researchers. According to The New York Times, Mukamal and Krystal’s slides stated, “A definitive clinical trial represents a unique opportunity to show that moderate alcohol consumption is safe and lowers risk of common diseases,” and suggested that the trial might make doctors recommend alcohol as part of a healthy diet.

February 28, 2014. Wine Industry Insight published “US Govt Asking Industry To Fund Most Of $50 Million Alcohol/Health Study”:

This caused a lot of concern within the NIAAA. Some people had no idea what study it was talking about and sent emails asking what was going on. Someone from the NIAAA communications office was clearly annoyed:

In one e-mail, two senior staff in the NIAAA discussed how they could best conceal information from an NIAAA division director:

June 21, 2014. There was a meeting in Seattle, led by Mukamal, and including NIAAA staff and the alcohol industry. Afterward, representatives from industry sent Mukamal a list of reasonable concerns about the design of the RCT like what outcomes will be measured, how to measure them, who will be eligible to enroll in the trial, how many sample sites will be used, and how to ensure compliance. Mukamal sent back a detailed response which looks sensible to me.

There is, however, a curious passage at the beginning of Mukamal’s response, implying that the final investigator had not yet been decided.

I don’t know how to judge this. Did the alcohol industry really think that other investigators would be involved? Or had the NIAAA winked at them enough that they knew it was going to be Mukamal?

December 8, 2014. A large joint conference call was coordinated between the alcohol industry, NIAAA staff, and researchers including Mukamal. Here are three topics that industry asked about:

1. Would the data be shared with other researchers? Mukamal stated that they would make “controlled data sets” available one year after the study ends.

2. Might industry funding call the study into doubt? Mukamal reassured that it’s fine because there will be a “firewall” between research and industry.

3. Would results be published even if they are negative? Mukamal said yes, but they would “most certainly” see a positive impact at least for diabetes.

You can read the full minutes for this meeting here. (For fun, try to find where “Ken” slipped through redaction.)

February 26, 2015. Mukamal and NIAAA senior staffers coordinated edits to an email that would be sent to someone in industry. This email stated that yes, they really needed $100 million, and “one of the important findings will be showing that moderate drinking is safe.”

Here’s the full quote to show that isn’t taken out of context:

Oct 5, 2015. The NIAAA publishes the funding opportunity for the big RCT, “Multi-Site Randomized Controlled Clinical Trial Research Center on Alcohol’s Health Effects.” Apparently the NIAAA originally requested that this funding opportunity be a limited competition where only people who had won the preliminary planning grant — that is, Mukamal — could apply. NIH central rejected this, but  the funding opportunity still “encouraged” it with language like the following:

Mukamal submitted his application on December 18. 

March-September 2016. The proposal was reviewed by the NIH, and eventually awarded to Mukamal. Little information seems to be publicly available about these reviews. The project began on September 30.

July 3, 2017. The New York Times published “Is Alcohol Good for You? An Industry-Backed Study Seeks Answers.” There’s this quote from George Koob, director of the NIAAA:

There’s also this:

The careful reader will note that in the above timeline, Mukamal just spent several years contacting the alcohol industry about this trial.

In October, Wired published their own story about the study, in which Mukamal again insists, “We have no contact with funders other than NIAAA itself whatsoever,”

February 5, 2018. The trial began enrolling patients.

March 17, 2018. The New York Times published “Federal Agency Courted Alcohol Industry to Fund Study on Benefits of Moderate Drinking.” They interviewed former federal officials and used Freedom of Information Act requests to get emails and travel vouchers related to the grant. This story reveals that, contrary to Mukamal’s claims, there were various meetings in 2013 and 2014. This includes a “working lunch” at a Beer Institute-hosted convention in Philadelphia in October 2013. 

March 20, 2018. Based on the previous article, NIH director Francis Collins ordered an investigation into the trial.

April 11, 2018. Collins appeared before the House Appropriations Subcommittee on Labor, Health and Human Services to discuss the NIH’s budget. When asked about the trial, Collins responded that he was very concerned and was investigating the issue as a matter of priority. (You can watch the video here.)

May 10, 2018. The NIH suspended enrollment in the trial.

June 8, 2018. Anheuser-Busch pulled its funding.

June 15, 2018. Based on a recommendation from an NIH working group, Collins terminated the study

When I talk to people about this story, I’m angry with so many different entities for so many different reasons that I have trouble putting my words in a coherent order. ⁵ So let’s start with the basics.

People did bad stuff: 

The trial started out with some corner cutting and bureaucratic maneuvering. But somehow this escalated into Mukamal and the NIAAA lying to the public. They claimed that this was just like any other NIH trial, where any researcher could propose a study design, and the NIH would choose the best entirely based on scientific merit. In reality, the NIAAA intentionally steered the money to one pro-alcohol researcher. Mukamal claimed he had no idea that industry funding was even involved, despite the fact that he had just spent several years coordinating things with industry.

The incentives encouraged misconduct: 

When I first read about this trial blowing up in the news, I was stupefied — how could everyone have been so shameless? What were they thinking?

Of course, we can’t know for sure — no one knows the heart of man. But I think that if you consider the perspectives of the different actors, it exposes deeper problems in the landscape of research funding.

So the NIAAA staff stretched the rules and misled the public. But imagine you knew a study would be valuable, except there’s some bureaucratic rule that prevents you from doing it. Wouldn’t you be tempted to stretch the rules?

If you’ve ever worked in government, you know that being able to work around bureaucratic obstacles is a key job skill. So think about the NIAAA staff who took “personal vacations” to visit Mukamal to help him write the original planning grant. I don’t get the sense that these people were trying to screw over the public for their own personal gain. Rather, they were trying to work around some rules that they saw as silly and inflexible barriers that were preventing them from accomplishing something important.

Say you’re a scientist and you want to send a grant to the National Science Foundation (NSF). According to The Rules, you will propose a detailed plan of future work. In some (more theoretical) fields this is absurd: you have to do half the work in order to write that plan! And in other (less theoretical) fields, your grant will be reviewed by other scientists who will expect to see “preliminary work” to show your idea has promise. This leads to a funny situation where people do much of the research and then “propose” it afterward.

Everyone involved knows that this is happening. The grant reviewers aren’t fooled. The people at NSF aren’t fooled. (Though if they’ve been around for a while, they might not notice the doublethink anymore.) Everyone is just trying to do the best they can inside of a system that they did not design.

At the NIAAA, The Rules say that you can’t solicit grants from industry. But what exactly is “soliciting”? If you run into someone in industry, can you abstractly mention that they’re allowed to donate money? What amount of detail can you go into? You might imagine there’s some oracle somewhere ready to lend definitive answers, but I doubt it. Instead, what you probably see is some people doing things that are a little like soliciting, and it’s fine. Eventually, someone pushes things slightly too far (or is just unlucky) and gets into trouble. The rules get clarified a bit then, but without acknowledging the institutional incentives that made everyone bend the rules in the first place. The person who got in trouble probably feels like a duck shot out of a flock.

So that’s what I guess happened at the NIAAA. The staffers are used to bending the rules because that’s what everyone does all the time. They think that the alcohol study would be beneficial and go for it, and over time things sort of spiral out of control.

No one thought about the impact of cancellation: 

When the NIH canceled the trial, they didn’t seem to explicitly consider the information that was lost by cancellation, or the fact that there would be little cost to taxpayers. (Though Collins’ letter to Senator Charles Grassley reveals the NIH did pay around $4 million out of pocket.) Could a different principal investigator be put in charge? Could the study design be modified to address the concerns? Could the monitoring bodies have been strengthened so people could trust the results? Maybe the trial was unsalvageable, but it’s telling that the NIH didn’t bother to make that argument.

The public health benefits of a well-designed and well-executed trial would be massive. So before jumping straight to cancellation, the NIH should have asked — if this trial goes ahead, should we trust the results? Despite the somewhat sordid history of this trial, the arguments that we shouldn’t aren’t as strong as you might think.

Clearly, Mukamal thought the trial would show a benefit, but that doesn’t mean he was right. Mukamal didn’t start claiming alcohol was safe as a cynical ploy to get his hands on grant money. He had been publishing on the health effects of alcohol for years. Can Mukamal be trusted? We can look at his track record. In 2002, he published observational research that showed tea drinkers were less likely to die from a heart attack than non-drinkers. In 2007, he was first author on a paper that randomly assigned patients to consume black tea or not. They looked at tons of different biomarkers and found that the tea did … basically nothing. This is the kind of case where it would be easy to p-hack your way to force some conclusion, but they straightforwardly state they found no evidence. Anyone who’s worked in science knows what it’s like to confidently run an experiment, only to get smacked in the face by reality’s indifference to your pet theories and career goals.

But OK, say you don’t trust the research team. What do you think they are going to do, fabricate data? The study was a collaboration of a large team around the world. The data would be stored at a Data Management Center at a different university and inspected every six months by a monitoring board. The organizational structure for the study involves over a dozen boards, committees, subcommittees, and centers:

This isn’t some excel spreadsheet stored on one grad student’s laptop. You’d need a big conspiracy.

Or maybe you don’t think they’d falsify data, but that for publication they would use some tortured data analysis to spin the results. The thing is, it’s not unusual to have researchers who want to find a given result — that’s every researcher everywhere! We have a system for this, which is that studies pre-register their statistical analysis. This study did that, and the plan seems fine (although, see below). There just aren’t many places to hide the bodies.

Even without a conspiracy, there are two concerns about the study design. For one, it’s plausible that the biggest harms of alcohol (e.g. cancer) appear later, while cardiovascular and diabetes benefits (if they exist) happen quickly. So a five-year study might find alcohol reduces mortality while a ten-year study could show the opposite.

Fine, but what’s the principle here? Should we cancel all studies where there’s a much more expensive and difficult variant that would be more conclusive? We know this is an issue now, and we’d still know it when interpreting results after the study is done.

Another concern is that the study population maximizes the chances for alcohol to look good: it would only enroll people who are either ≥75 years old or at elevated risk for cardiovascular disease while excluding anyone with liver disease, a personal history of colon/liver/breast cancer, a family history of breast cancer, suicidal ideation, or dementia. If I wanted to maximize the chance that alcohol could be beneficial while minimizing the chance that alcohol could be harmful, this is the population I would choose.

If you want a final verdict on whether moderate drinking is safe, I agree this seems like stacking the deck. I’d prefer a random sample of all adults. You can call this a “bias.” But you can also call it “refusing to take the sampling scheme into account when interpreting results.” There’s still value in knowing how alcohol affects a restricted population. And we can extrapolate — since the people in this study were exceptionally likely to benefit from alcohol, a neutral result in this study population would suggest alcohol is harmful to the average person.

You might also argue that it’s ethically required to exclude people who are at higher risk for being harmed by alcohol. I don’t really agree, but I’d imagine many people would.

The final NIH report notes that the researchers do not have “the requisite equipoise.” You could interpret this in two ways. One, you might say the whole thing seems rotten and damn the logic of it. The other is that it looks bad for the NIH — that, even if useful, it needs to be canceled to preserve trust in the institution. I understand this. But if that’s the reason to cancel, it makes me sad — in an ideal world, the best way for the NIH to preserve trust would be to ruthlessly pursue knowledge to benefit the public interest.

The problems could have been fixed – but not by these people: 

The study might have been fine. I’ve tried to argue above that there are lots of structural factors that make the behavior of everyone here much easier to understand, and which mitigate some of the risk of bias. But despite that, I think this story also shows why it’s important to have personal principles.

I’m open to industry-funded research. I don’t necessarily mind a lead researcher who was chosen because they believe something that happens to support industry. I can even live with industry having influence on the study design. I stubbornly hold all this even when the study has a goal of proving it’s safe to use humanity’s most harmful drug.

But my (possibly delusional) open-mindedness is based on the idea that it’s possible to compensate for the biases these issues create. That’s not possible if we don’t know about them.

So I can accept that the NIAAA staff might have thought that this study might have had value, despite all the obvious issues. But if they had tried to rescue the study, they would have needed to make that argument openly, not pretend the issues don’t exist.

Many people are also complicit in silence. Maybe the alcohol industry really didn’t think anything underhanded was happening. Well, they knew on July 7, 2017, when the first New York Times story came out, including untrue or misleading statements from Mukamal and the NIAAA. They had months to correct the record, but they did nothing. The same is true for many of the other researchers involved.

Because of the scandal, the idea of industry funding with a firewall — which could be tremendously valuable — was tarnished. If we interpret the NIAAA and Mukamal charitably, what they seem to be suggesting is that there was a “late firewall” with lots of contact with industry early on, but no influence after the trial started. But that didn’t happen! How do I know? Well, did you notice the part where Anheuser-Busch pulled its funding? Having the power to shut down the entire trial whenever you want qualifies as influence.

Media coverage didn’t help either. Take that New York Times article again. Remember that when this was written, the firewall still held, as far as anyone knew. Besides mentioning that the study exists and is funded by industry (which is totally legit) it’s largely a collection of whatever random connections they could dig up between anyone connected to the study and the alcohol industry. (One investigator “has conducted research at institutions that received industry support,” another was paid to speak at an industry conference nine years earlier). There are also quotes about how industry funding skews research, but it doesn’t address that that’s why there was supposed to be a firewall.

Obviously, I’m glad The New York Times followed up on this story and revealed holes in the firewall. I just wish there was a more nuanced tone that engaged with the premise that the problems with industry funding are, in principle, possible to overcome.

The trial didn’t happen: 

Yes, I’m mad the trial didn’t happen. Despite people doing bad stuff, despite the industry influence, despite the imperfect study design, this trial would have provided unparalleled ability to answer some really important questions. Globally, the average person apparently drinks around 6.18 liters of alcohol per year, or around 1.5 drinks per day. What impact is this having? Should doctors encourage moderate drinkers to stop? Should they encourage abstainers to start? Currently, we just don’t know. This trial was a chance for us to get some kind of an answer, and we didn’t get one. Instead of imperfect data (which is all data), we got none.

What did we get instead? Collins says that “three individuals are no longer employed” at the NIH, and they made process changes to avoid similar problems in the future.

That’s something, but what about the researchers? To their credit, Harvard and Beth Israel did do an investigation of Mukamal, which led to him formally apologizing and both institutions creating safeguards to make sure no future employees would do anything similar.

Hahahaha, no. Here’s what actually happened:

1.Mukamal stated, “We stand fully and forcefully behind the scientific integrity,” and “Every design consideration was carefully and deliberately vetted with no input or direction whatsoever from private sponsors.” (Yes, these are real quotes from after the study was canceled.)

2. As far as we know, there were no investigations by Harvard, Beth Israel, or any of the other researchers’ institutions. No one faced any penalty of any kind.

3. In 2020, in a brazen display of academic shamelessness, the researchers published a paper on how awesome the study would have been. ⁶

Here’s a quote from that paper’s “sponsorship” section:

The Foundation for the National Institutes of Health (FNIH) supported the trial financially and managed contact between public and private organizations on behalf of NIH. The funds provided by FNIH for this project were contributed to FNIH by the brewing and distilling industries following contract negotiations that established an intellectual and financial firewall between MACH15 investigators and private contributors. The corporations providing support agreed to have, and had, no contact with trial investigators about any aspect of the study after their commitment of funding, and they agreed to receive no data or updates until they became publicly available. Ultimately, however, the most important safeguard for impartiality lies in the execution of a rigorous, transparent protocol following independent, expert peer review, and in the conduct of the statistical analyses as described in the protocol.

Emphasis mine. You can’t make this stuff up.