Background

Previous articles in this series have documented the secrecy surrounding what have been called “budget justification” reports purchased through no-bid contracts by the Michigan Economic Development Corporation from a company called Longwoods International. (See “Pure Michigan Puffery” Parts I and II as well as “Puffery or Proof?”) Longwoods produces very similar products for agencies in at least eight other states, claiming high returns from taxpayer-funded tourism ads.

The MEDC is this state’s agency in charge of business subsidy and assistance programs. Last year it got a $33 million appropriation to run its “Pure Michigan” ad campaign. The agency would like to get another $33 million or more this year.

However attractive you think the Pure Michigan ads may be, they amount to pure corporate welfare for the travel and leisure industry. This reality, and the fact that even in Lansing $33 million is real money, means that during appropriations season legislators and the public may need some convincing.

That’s where Longwoods comes in. The most recent report it did for the MEDC purports to show a $6.87 return for every dollar spent on Pure Michigan ads in 2014. Whether this is accurate or even plausible cannot be determined because Longwoods refuses to reveal all the details of the methods it may have used to generate the claim.

The MEDC paid almost $150,000 for this product, but agency officials haven’t shown much curiosity regarding their hand-picked vendor’s claims. To the contrary, the MEDC has offered a dubious defense of the secrecy.

Who Watches the Watchers?

Such unanswered questions are not helpful for “budget justification” purposes. (That’s how Longwoods characterized research it produced for another state’s tourism agency. The statement was made on a page that has since been taken down from the company’s website.)

So it is interesting that in 2011, when pressed on similar questions about earlier Longwoods’ reports, the then-head of Michigan’s travel promotion bureau pointed to a “validation study” for which the MEDC paid a different contractor $5,000. While this was supposed to confirm that the Longwoods methodology was valid, it also suggests the agency knew these reports had a credibility problem.

Moreover, even this second vendor said Longwoods had a “lack of total transparency,” and acknowledged that reviewing its process would “require costly replication to totally confirm” that its claims were valid. But even if cost were no object, Longwoods’ secrecy would make confirming its claims impossible.

Dodgy Methods Acknowledged

We do know some things about the method, although the details provide little comfort. Bill Siegel, founder and CEO of Longwoods, has said his company uses a hybrid system based on what are called “conversion” and advertising tracking studies. The former are said to measure how many people responded to a travel ad by later visiting the destination.

Nevertheless, traditional conversion study methods appear to have been what the validation study authors used for their work. Conversion studies have been widely criticized by academic economists, who consider them highly dubious. In his book, “Tourism Analysis: A Handbook,” author Stephen Smith writes that — especially with tax-funded travel promotion agencies — “Conversion studies are increasingly popular … as a tool to justify budgets. Because of survival motivation, many conversion studies fall short of full adherence to rigorous scientific procedures.” Smith wrote this was especially true of tax-funded agencies that promote tourism interests.

Even Siegel (writing with colleague William Ziff-Levine) said in 1990 that, “The main value of a conversion study would appear to be to get diagnostic information on a fulfillment piece, not to provide hard numbers on the bottom-line return on-investment of a campaign.” More recently, he said that this hybrid approach uses the parts of the conversion study system that he believes are its most useful and effective.

And then there is the secrecy. While you can read an online copy of the survey Longwoods used to produce its 2014 report, it is meaningless without a detailed explanation of how the company used the survey to derive its claim of a 587 percent return from an ad campaign. And such an explanation is exactly what you won’t find on any Longwoods website or published document.

The MEDC’s validation contractor was supposed to evaluate the legitimacy of Longwoods’ claim. Notwithstanding Longwood’s lack of transparency, this second outfit chose to attempt its own version of a conversion study.

Process Fails Smell Test

As if doubts about the underlying contract weren’t enough, there are many reasons to question the practice of paying a second contractor to validate the claims of a “budget justification” contractor.

To begin with, the MEDC did not issue a request for proposals, which could have given it a number of validation contractors to choose from. Instead, agency officials hand-picked a contractor, which at the very least limited the scope of possible perspectives. Also, it’s hard to ignore that the co-author of the validation study is a former vice president of Longwoods International, the company being evaluated.

That revolving-door relationship is part of a pattern. The MEDC official who mentioned the validation study in 2011 was George Zimmerman, whose title was vice president of the MEDC's Travel Michigan office, which oversees state promotion efforts. Mr. Zimmerman has since been hired by Longwoods International as its chairman.

The questions multiply when details of this second contractor’s work are examined. For instance, the survey research it used involves asking respondents to report the degree to which the state of Michigan’s advertisements influenced their decision to visit. This introduces the possibility of response bias — a pitfall of survey research — which would influence the output. Moreover, the authors simply lump in two categories of responses at different rates before making their calculations. They claim this is a common technique but offer no supporting evidence. Curiously, one result derived by the validation authors apparently matched the Longwoods International conclusions to the penny ($3.42).

Confirmation Study Confirms Nothing

Perhaps not surprisingly, the validation study authors so thoroughly hedged and qualified their report that they appear to doubt whether “return on investment” claims like Longwoods’ can actually be measured. In one place they wrote, “The ‘real’ conversion rate in [sic] unknown and next to impossible to derive given the many variables involved, many beyond the control of the researcher.”

Elsewhere, contractor No. 2 admitted, that “the possible confidence interval for our estimate is wide,” and that “our bottom line estimate will likely be inflated to the degree that other Michigan advertisers were also engaged in these markets at this time at a level beyond the precision of this estimate.” Translation: They came up with a ballpark estimate in a very large ballpark. Still the contractor suggests that even if they’re off by 25 percent the result is still positive.

The outfit that was paid $5,000 to prepare this report can at least be commended for admitting they don’t know what they don’t know.

Perhaps capping all the preceding, 2011 press reports describe former MEDC and current Longwoods International official George Zimmerman repeatedly pointing to the validation study to support his company’s claims about MEDC ad spending. However, the validation study contract was entered into in 2007 to cover a Longwoods’ report on a 2003 tourism ad campaign. Published reports indicate that the state had validated Longwoods’ technique but failed to mention that the validation work involved ads unrelated to the Pure Michigan campaign.

Conclusion

To sum up:

• MEDC officials have granted a series of no-bid contracts to Longwoods International, a company that has basically advertised itself as a “budget justification” assistance service.
• The agency asked the company to quantify a positive return from tax-funded tourism ads, not to discover whether the spending generates a positive return — meaning it simply assumed a positive return. (See “Puffery or Proof?” for details.)
• The MEDC’s close ties to Longwoods are mirrored by a tie between Longwoods and the firm hired to confirm the validity of its claims.
• Independent verification of Longwoods’ methods is impossible, however, because the company refuses to reveal the details of those methods.
• Recognizing it may have a credibility problem, the MEDC hired a second firm to examine Longwoods’ claims about a particular ad campaign.
• The second contractor all but admitted that the thing it was hired to do — verify Longwoods’ extraordinary “return on investment” claims — was impossible.

All of this amounts to an ongoing campaign by officials in a government agency to justify continued spending on a corporate welfare program that’s a major component of their annual budget, with no credible evidence that taxpayer dollars aren’t being flushed away.

Yet policymakers and taxpayers are expected to presume that these officials are acting solely in the public interest rather than their own interest and that of the industry benefiting from this corporate welfare. The events described in this series make this presumption exceedingly difficult to grant.

Indeed, it has become almost impossible to avoid the conclusion that it all amounts to a self-serving campaign of self-promotion by an agency whose continued existence and legitimacy are the focus of growing skepticism from lawmakers and taxpayers.

Permission to reprint this blog post in whole or in part is hereby granted, provided that the author (or authors) and the Mackinac Center for Public Policy are properly cited.