On September 14, the Food and Drug Administration released two briefing documents for the joint meeting of two committees. These are the Psychopharmacologic Drugs Advisory Committee and the Drug Safety and Risk Management Advisory Committee. The first briefing was from Pfizer, Inc. (NYSE:PFE) itself, on the merits of its case to remove the boxed warning from Chantix, its smoking cessation aid approved back in 2006 but implicated in neuropsychiatric side effects and suicides in the two years following its approval. A boxed warning was added to the drug in 2009 as a result. The second briefing was from the point of view of the joint review panels, who did not like the way data was collected in the post marketing study to remove the boxed warning.

Key parts of the warning label text are as follows:

Serious neuropsychiatric events including, but not limited to, depression, suicidal ideation, suicide attempt, and completed suicide have been reported in patients taking CHANTIX. Some reported cases may have been complicated by the symptoms of nicotine withdrawal in patients who stopped smoking. Depressed mood may be a symptom of nicotine withdrawal. Depression, rarely including suicidal ideation, has been reported in smokers undergoing a smoking cessation attempt without medication. However, some of these symptoms have occurred in patients taking CHANTIX who continued to smoke. All patients being treated with CHANTIX should be observed for neuropsychiatric symptoms including changes in behavior, hostility, agitation, depressed mood, and suicide-related events, including ideation, behavior, and attempted suicide. These symptoms, as well as worsening of pre-existing psychiatric illness and completed suicide, have been reported in some patients attempting to quit smoking while taking CHANTIX in the postmarketing experience. When symptoms were reported, most were during CHANTIX treatment, but some were following discontinuation of CHANTIX therapy.

Pfizer’s document is 114 pages long. The FDA panel document is a much longer 564 pages. Market Exclusive staff combed through these documents and our conclusion is as follows. Spending that much ink reviewing data is a bad sign for Pfizer, and for various reasons primarily centering on the nature of the FDA itself as a regulatory body, the nature of Chantix as being helpful towards but not absolutely necessary for quitting smoking, and the fact that the FDA looks out for itself first (meaning its reputation as generally competent) and only secondarily for the lives of patients affected by its decisions, we feel that the chances of Pfizer scoring a removal of the unwanted label are very low. At best, the wording may be slightly changed, but the chances of even that are comparatively low.

Granted, the joint FDA advisory panel voted 10 to 9 to remove the label, but it’s not really the vote that counts. It’s the sentiments behind the vote that count. We will know more about those when the meeting transcripts are published, but judging by the scant reporting of the actual meeting, it seems that all or nearly all panelists were quite concerned about the way data was collected in this trial, and that the 10 that voted in favor only did so on the logic that removing the label will increase the use of Chantix and enable more people to quit smoking, thereby saving lives on net. The vote was actually 10-4-5 for remove, reword, retain respectively, but we are counting it as 10-9 for simplicity.

In order to understand why the chances of removal of this label are so small, first let’s consider the nature of the FDA itself. The FDA is a regulatory body subject to public pressures. The worst thing that can happen to it is to look stupid. The most obvious way it can look stupid is to approve a drug that hurts or kills people, or is even suspected of hurting or killing people by the public even though a drug may have absolutely nothing to do with whatever bad thing happened. That’s the starkest way to put it.

In cases where people have terminal illnesses anyway, then of course the chances of getting a drug approved are greater because the pressure is usually coming from the other side in those cases. If the FDA rejects a cancer drug for instance, and then it conducts further studies and it is found to be effective, then the FDA gets some flak for delaying its approval. Any deaths they likely won’t be directly blamed for, because the underlying disease is terminal anyway.

It’s different with a drug like Chantix, because it is not a necessary drug even for what it is trying to help achieve, which is to help people quit smoking. It is not treating anything terminal directly. While it may be true that removing the boxed warning will save people on net by helping more quit smoking, these sorts of results are not headliners. They are what some economists would call the “unseen”. What is “seen” by contrast is if someone commits suicide or hits his or her spouse while on Chantix, and Pfizer and the FDA get blamed when the newspapers are looking for sensationalist stories.

Take the recent example of Tesla Motors Inc. (NASDAQ:TSLA) autopilot system. Car accidents happen all the time. People are killed. Car accidents will also eventually happen on autopilot, and autopilot will get bad press, even if autopilot actually reduces the incidence of car accidents per mile travelled, something that needs extensive statistical analysis. The latter requires thinking at least 2-steps deep, but seeing a headline like “Driver on Tesla Autopilot Killed” is immediately seen, and only 1-step deep. Few delve into statistical analysis to broaden their knowledge in these cases.

One of the most egregious blunders from which medical regulatory bodies in general have not recovered and likely never will, was the thalidomide fiasco of the 1950’s and 60’s. A drug designed for morning sickness in pregnant women but later found to cause severe birth defects as a terratogenic poison. People affected by this in utero are still alive today without limbs as a permanent testament to what the FDA, at all costs, does NOT want to get blamed for. Granted, the FDA never approved thalidomide and the phenomenon was mostly international, but that doesn’t change the point.

So let’s think of some hypotheticals here. Let’s say the boxed warning label is removed and there’s a story of a murder-suicide somewhere in the US, heaven forbid. Let’s say it’s found that this person was on Chantix at the time, and had no prior history of violent behavior. The FDA will have to weather a storm of controversy it can easily avoid, or at least greatly lessen, if the warning label remains on the Chantix box as is. Once the label is there, removing it means the FDA is absolutely sure that there is no connection between psychiatric side effects and Chantix. There will be pressure to put the label back on. There might be congressional hearings on Capitol Hill. The public does not have the patience to wade through statistical studies of course, least of all politicians, and putting the label back on after taking it off will make the agency look stupid, wishy-washy, and incompetent.

Whether or not Chantix is actually related to any of these adverse events does not, in the end, matter all that much for the purposes of removing the label, just like it doesn’t matter if Tesla’s autopilot increases or decreases road fatalities. What matters is the FDA’s reputation. Someone somewhere in the future who happens to be on Chantix will commit suicide at some point. People unfortunately commit suicide sometimes. The phenomenon cannot be stopped entirely, and at least one of these people will be on Chantix. It will make headlines, and the FDA will have to defend itself without the use of complicated statistical graphs because nobody wants to see those.

It’s not like the FDA can point to a graph showing less people dying of smoking, attribute that to removing the boxed warning, and pat itself in the back in defense. The FDA will not be blamed for smoking deaths. It will be blamed for Chantix deaths if and when they happen, regardless of whether Chantix had anything to do with the death or not. That’s just how it is.

The Data Collection Problems

As for the actual problems with the data collection in this post-marketing trial, here are some excerpts from the FDA’s 500 plus page briefing document linked to above.

An example of one of the original narratives…demonstrat[es] the inadequacy of the original narratives in providing information needed to assess events of potential interest. In this case, an event coded to the term “skull fracture” is reported in the narrative without providing any context for how the patient came to sustain the skull fracture. (Subsequent information requests revealed the injury was inflicted by her boyfriend, also a trial participant, a potentially relevant piece of information.)

Obviously, if the boyfriend hit the girlfriend while on Chantix, that could be termed a severe neuropsychiatric adverse event related to Chantix. It was characterized as unrelated, however. Maybe it was unrelated, but that’s beside the point.

Here’s more:

The instructions provided to the investigators for assessment of relatedness do not appear to have resulted in a full capture of potentially related events. Examples include emotional symptoms frequently attributed to “stress” or to specific social situations, worsening symptoms of a preexisting psychiatric illness almost always coded as “not related to study drug,” and assessments such as “The investigator considered the SAE (severe adverse event) of foot fracture to be not related to the study drug but due to fractured right heel” were provided. For this reason, investigator assessment of relatedness has been altogether disregarded in this review.

One more:

These types of cases further underscored the concern that the severity criterion for inclusion in the NPS endpoint may have been inappropriate to capture events of concern. There may have been a disconnect between what subjects with no previous psychiatric issues consider severe (even missing a day of work) and what a health care provider accustomed to caring for seriously mentally ill patients would regard as “severe” (possibly only an event requiring hospitalization). However, even hospitalization may not have been assessed as “severe” by some investigators; an additional case was identified among the SAE narratives, where a subject appears to have been hospitalized for depression after about 3 weeks of treatment with bupropion, but the event was assessed as “mild” by the investigator. (“On , the subject experienced depression which was considered mild in intensity and serious due to hospitalization or prolonged hospitalization by the investigator.”

Qualitative measurements are extremely difficult to quantify, especially in such a large series of studies like this. The whole initiative may have been doomed from the start. Categorizing mental phenomena as severe or mild from one person to another who may both have totally different ideas as to what is what and what the definitions of relevant terms are, is almost impossible to do effectively.

There are other issues cited by various sources that may also further motivate the FDA to maintain the label as is, including not wanting to set a precedent for successful removal of a label like this on the basis of poorly collected data. This is also an issue, but not the primary one in our opinion. The central issue is the FDA’s reputation.

Finally, may we say this: The FDA is a regulatory bureaucratic body. It operates based on strict rules and guidelines set for it by Congress and it never wants to make a pure “judgment call” if it can avoid doing so for fear that its judgment call may be slightly off and it will be blamed for something. There are no regulatory guidelines for the removal of a boxed warning. Says the FDA (emphasis ours):

There is no specific regulation or guidance that has been established to remove a boxed warning. However, if the criteria for including a boxed warning are no longer met, it is reasonable to remove it.

Even if it is “reasonable” to remove it, it is also “reasonable” to retain it. The FDA probably will not risk its reputation for the sake of being “reasonable”. If there is no regulation or specific guidance to remove a boxed warning, with data collection problems like these and the potential for seriously bad press if the label is removed and something bad happens – and something bad will eventually happen, even if it takes years to happen – you can bet that the FDA will not remove this label. Reword it slightly, maybe, but remove it? The chances are 10% at best, regardless of the technical majority vote by the joint advisory panel on September 14.