Thursday, February 16, 2012

Rhode Island Heart Attack Study Makes No Attempt to Control for Underlying Secular Trend of Declining Heart Attacks

Yesterday, I reported on a new study from the Rhode Island Department of Health which concluded that the smoking ban, implemented in Rhode Island in March 2005, resulted in a 28% drop in heart attack admissions in the state (see press release).

The study examined age-adjusted rates of admission for acute myocardial infarction in all Rhode Island hospitals from 2003 to 2009. The report compares the hospitalization rates in 2003 and 2004 (prior to the smoking ban) with those in 2006 through 2009 (the four complete calendar years following the smoking ban).

According to the press release: "The findings reveal a 28.4 percent drop in the rate of acute myocardial infarction (AMI) admissions and a 14.6 percent reduction in total associated cost, representing a potential savings of over six million dollars."

However, as I showed, an examination of the actual data reveals that before the smoking ban, heart attacks were declining at 10.5% per year and after the smoking ban, heart attacks were only declining at 5.3% per year. Thus, when viewed in light of the baseline trend, there is no evidence that the Rhode Island smoking ban led to a decline in heart attack admissions.

The paper, however, concluded that the smoking ban led to a 28.4% decline in heart attacks. If you're wondering where this 28.4% number comes from, the paper simply determined the change in heart attacks from 2003 to 2007.

The Rest of the Story

In other words, the paper made no attempt to control for, or even consider baseline trends in heart attacks in Rhode Island, which were declining rapidly prior to the smoking ban. This is a cardinal error which renders the conclusions of the study invalid.

When one runs a regression analysis that controls for the secular trend of declining heart attacks (which existed before the smoking ban), one finds that the coefficient for the effect of the smoking ban is no longer statistically significant. Basically, what this means is that although there was a significant decline in heart attacks in Rhode Island from 2003 to 2009, this decline was not significantly different from what would have been expected based on the rate of decline in heart attacks that was already occurring in the state prior to the smoking ban.

Here are the heart attack rates reported in the study. The smoking ban was implemented in March 2005, so the data for 2005 should begin to reflect any effect of the smoking ban:

2003: 35.2 admissions per 10,000 population
2004: 31.5
2005: 30.6
2006: 28.1
2007: 25.2
2008: 25.4
2009: 23.1

The study analyzes the data as if the heart attack admission rate in 2004 was also 35.2, just as in 2003. In other words, it assumes that the heart attack rate was steady at baseline. Were that the case, then one could simply report the percentage change in heart attacks from 2003 to 2007 and attribute that to the smoking ban, as the study did.

However, the heart attack rate in 2004 was not 35.2. It was 31.5. This represents a 10.5% decline. And it was already occurring: prior to the smoking ban. One needs to control for this baseline secular trend of declining heart attacks in the baseline period in order to determine whether the smoking ban had any effect. Otherwise, one is simply reporting the pre-existing secular change. This is precisely what the Rhode Island report did.

When you are examining an outcome measure that is changing over time, one must account for the pre-existing trends in this variable before attributing these changes to an outside factor. Failure to do this renders the conclusions invalid.

What I can't seem to understand is why nearly every smoking ban - heart attack study appears to be making the same analytic mistake. It is as if anti-smoking researchers are only interested in reporting favorable results, rather than in reporting accurate ones.

No comments: