Search results: research bias

I’ve been posting summaries of studies funded by food companies or trade groups, all of which come up with results that the sponsor can use for marketing purposes.

In each of these posts, I ask for examples of industry-funded studies that produce results contrary to the interests of the funder.

In response, I received this comment from Mickey Rubin, Vice President for Nutrition Research, National Dairy Council.

He gave me permission to reproduce his letter: 

Dear Marion,

By way of introduction, my name is Mickey Rubin and I am a scientist at the National Dairy Council. I understand that you know Greg Miller, and I asked him for your contact information so I could write to you directly after reading with great interest your most recent post on industry-funded nutrition research, in which you selected a sample of 5 studies/papers sponsored by industry all showing favorable outcomes. Although none of the papers you selected were sponsored by the organization I represent (although there is one dairy industry sponsored review paper in the list), what struck me is your focus on the favorable vs. unfavorable dichotomy, rather than the reality of what much nutrition science research results in: null findings.

It seems that there are fewer and fewer nutrition studies published that report the null, or find no effect. I agree with you that the reason we don’t see more of these studies in the literature has to do with bias, but I suspect that it is publication bias as much as any other bias. From my interactions with nutrition researchers, I gather it is quite difficult and sometimes impossible to get a study with no significant effects published regardless of funding source, to say nothing of allegiance bias by some researchers hesitant to publish findings that may go against their own hypotheses. Dr. Dennis Bier of Baylor College of Medicine and editor in chief of the American Journal of Clinical Nutrition has presented eloquently on this issue previously. You may also be aware of David Allison’s papers on other types of bias. So I think it is important to discuss all types of bias, and not just industry bias. You of course wouldn’t want your discussion on bias to be biased to just one type.

At National Dairy Council we have an extensive program of nutrition research that we sponsor at universities both nationally and internationally. While I can’t speak for all of industry, we strongly encourage the investigators of all of our sponsored studies to publish the findings, no matter the results. Thus, we would expect our sponsored studies to have a similar “success” rate as those sponsored by the National Institutes of Health. In fact, that is exactly what one recent analysis – not sponsored by the dairy industry – found, reporting that there was no evidence that dairy industry funded projects were more likely to support an obesity prevention benefit from dairy consumption than studies sponsored by NIH.

We feel this transparency is not only critical to the credibility of the research we sponsor, but we also feel it is important that our research contributes to nutrition science knowledge as a whole. We hope that other scientists take the findings from studies we sponsor and build upon them, and if it is by using research dollars from other sources, even better! I’ll be the first to stand up and say that one favorable study on milk, as an example, does not close the books on the subject. We need many studies in many different labs sponsored by multiple agencies in order to produce a portfolio of knowledge. I suspect that is certainly an example of where you and I are in agreement.

That all said, please allow me to provide some examples of studies the National Dairy Council has sponsored that are published and, rather than showing a clear benefit, do not refute the null hypothesis. These are all studies published within the last 4 years. It’s not meant to be comprehensive, but rather just a sample similar to what you provided. I could also provide you a list of studies we have sponsored that have shown favorable results for dairy, but you seem to have that covered, and I’ll instead wait until one of our sponsored studies appears in a subsequent blog post J.

Thanks for taking the time to read. I appreciate the dialogue.

Here’s his list of papers:

Studies with null finding:

Bendtsen et al. 2014: http://www.ncbi.nlm.nih.gov/pubmed/24168904

• No unique benefit of dairy protein over other proteins for weight maintenance

Maki et al. 2013: http://www.ncbi.nlm.nih.gov/pubmed/23901280

• No effect of three servings of dairy on blood pressure

Chale et al. 2013: http://www.ncbi.nlm.nih.gov/pubmed/23114462

• Whey protein supplementation offered no additional benefit over resistance training alone in older individuals

Lambourne et al. 2013: http://www.ncbi.nlm.nih.gov/pubmed/23239680

• No change in body weight or composition in adolescents performing resistance training and supplemented with milk, juice, or control

Van Loan et al. 2011: http://www.ncbi.nlm.nih.gov/pubmed/21941636

• Recommended dairy servings offered no additional weight loss benefit over calorie restriction without dairy servings 

Studies with mixed findings (some outcomes changed, others null):

Maki et al. 2015: http://www.ncbi.nlm.nih.gov/pubmed/25733460

• The main finding from the study was that dairy intake had no effect on glucose control whereas sugar sweetened product consumption contributed to a worsening of glucose control in at-risk adults.

Dugan et al. 2014: http://www.ncbi.nlm.nih.gov/pubmed/24236646

• Waist circumference and BMI were lower in women after consuming the dairy diet as compared to the control diet. Fasting glucose was lower in men following the dairy diet as compared to the control diet. There were no differences in blood pressure, serum lipids, fasting insulin, or insulin resistance between the treatments.

Here’s what I wrote in response:

I am familiar with charges of bias against independently funded researchers (“White-hat Bias”), which equates industry biases with biases that result from career objectives and other goals.  I do not view the biases as equivalent.  Industry-sponsored research has only one purpose: to be used in marketing to sell products.   As I have said repeatedly, it is easy to design studies that produce desired answers.

When I was in graduate school in molecular biology, we were taught—no, had beaten into us—to do everything we could to control for biases introduced by wishful thinking.  I don’t see that level of critical thinking in most studies funded by food companies.

You may be correct about the influence of publication bias with respect to dairy studies, but how do you explain the situation with sugar-sweetened beverages?  Studies funded by government and foundations typically indicate strong correlations between habitual consumption of sugary beverages and metabolic problems, whereas studies funded by the soda industry most definitely do not.   The percentages are too high to be due to chance: 90% of independently funded studies show health effects of soda consumption whereas 90% of studies funded by soda companies do not.  This is troubling.

We’ve seen the results of studies funded by tobacco and drug companies.  Are food-industry studies different?  I don’t think so.   What seems clear is that industry-induced biases are not recognized by funding recipients, a problem in itself.

That’s why I’m posting these studies as they come in and begging for examples of industry-funded studies that do not favor the interests of the donor.

Thanks to Mickey Rubin for writing and for permission to reproduce his letter.

Let the discussion continue!

BMJ (formerly the British Medical Journal) has just released an editorial and four papers on ties between the sugar industry and public health scientists who advise the government on health policy.  Some health policies involve recommendations about intake of sugars.

The BMJ press release explains

Recipients of research funding from sugar and other related industries include members of the Scientific Advisory Committee on Nutrition (SACN), which is currently updating official advice on carbohydrates consumption, and researchers working for the Medical Research Council’s Human Nutrition Research unit (HNR).  HNR scientists have received research funding and funding in kinds from companies including Coca-Cola, Mars, Nestlé, Sainsbury’s, the Institute of Brewing and Distilling, Weight Watchers International and others…Of the 40 scientists affiliated with SACN between 2001 and 2012, only 13 have had no interests to declare.

This, of course, is no different than what we see here.  Food and beverage companies support food and nutrition research as well as professional societies, and conflicts of interest are rampant.

Even so, these well documented studies are alarming and worth serious attention.  And be sure to look at the map.

• Sugar: spinning a web of influence
• Sugar’s web of influence 2: Biasing the science
• Sugar’s web of influence 3: Why the responsibility deal is a “dead duck” for sugar reduction
• Sugar’s web of influence 4: Mars and company: sweet heroes or villains?
• Map showing links between public health scientists and the sugar industry ​

MedPage has a nice summary (I’m quoted).

The furor over these articles

• UK scientists object to the accusations
• Responses to the BMJ
• BBC commentary on the dispute
• Susan Jebb responds
• The Obesity Society’s Position Opposing Ad Hominem Attacks (its term for articles like those in the BMJ)

In the past few weeks, I’ve been sent several questions asking my opinion of food professionals’ relationships with food companies.   I thought I would deal with them at one time (all are edited for succinctness and clarity):

Q.  I am a food science student looking into career options in the food industry. I love food science and truly believe that processing food is a good idea that can positively impact the planet and its people.  I want to do something worthwhile,, but I still need to eat.  Can’t the food industry be changed from the inside? Can’t you advise good companies, people, or places where I could start my search? 

A.  I have met social entrepreneurs who strongly believe that businesses can be ethical, do good, and still make heaps of money.  Maybe so.  If you are going to try that route, I think it essential that the company be family or cooperatively owned, and not publicly traded.  You might take a look at food companies incorporated as Benefit Corporations. These are now authorized in about half the states to consider the interests of all stakeholders, not just shareholders, when making decisions.  They are different from B corporations certifying companies that meet certain sustainability criteria.

Many companies work hard to reduce their environmental impact.  But the real question is what they are doing about health impact.  Are they going overboard on health claims?  Are they marketing to children?  These are questions I’d want to ask.  In your shoes, I’d start by looking at companies making products that you like, feel good about, and would be proud to be associated with.   And then take a closer look at how the companies operate.   Working for food companies is always a good learning experience, but if you really want to change the world, you might be better off with a nonprofit agency.

Q.  I’m a graduate student in nutrition and I would like to know what recommendations you may have for students to navigate conflicts of interest with food companies when beginning a career.  I intend to pursue an academic career but am concerned that my credibility as a scientist could be compromised by my participation in industry-funded publications and research. 

A.  It’s great that you are asking such questions. From the standpoint of ethics, that’s an important first step.  You should most definitely publish your research, no matter how it is funded.  Be sure to disclose potential sources of funding bias and conflicts of interest.  While you are doing your research, you can take special care to control for potential biases—conscious and unconscious—in your study design, conduct, and interpretation to ensure that they are not influenced by the funder.  In searching for jobs, you might consider those in academia, government, and NGOs that are less likely to require you to compromise principles.  Finding such jobs may not be easy, but you will be OK if you are always ethically transparent and as straightforward about biases as it is possible to be.

Q.  Do you believe that relationships between the food industry and nutrition professional organizations like the Academy of Nutrition and Dietetics (AND) and the Association for Nutritional Science (ASN) are problematic? Why?

A.  Sponsorship by food product makers puts nutrition professionals in conflict of interest.   Nutritionists ought to be advising clients and the public about what to eat to stay healthy and prevent chronic disease.  This necessarily means promoting consumption of some foods but discouraging consumption of others.  Nutritionists cannot speak truth to clients and protect corporate sponsorship at the same time.  If nothing else, food industry sponsorship gives the appearance of conflict of interest and makes AND and ASN appear as arms of food company marketers.  But it also affects—or appears to affect–AND ’s and ASN’s positions on key issues in nutrition and health.  Overall, financial relationships between these organizations and their food industry sponsors undermines the credibility of their positions on food issues.

Q.  What sort of changes do you think the Academy needs to make in order to make food industry relationships more beneficial to its members and the public overall?

A.  Nutrition and food professional organizations need to establish a firewall between corporate sponsorship and content or opinion.  This requires setting up rigorous guidelines for what food companies can and cannot expect from their donations.  They should not, for example, be permitted to sponsor content sessions at meetings, not least because opinions expressed at sponsored sessions rarely appear objective.  The organizations should have complete control over how and where corporate donations and company logos are used.

Q.  How can relationships between health professionals and the food industry be beneficial for public health overall?

A.  The role of health professionals is to give the best advice possible about diet and health.  The role of food companies is to provide profits to shareholders.  These goals are not the same and are only rarely compatible.  In my experience, people who want to work for food companies to change corporate culture from within do so from good motives, but soon discover that corporate imperatives take precedence over health goals. If health professional organizations want their advice to be taken seriously, they must establish and adhere to rules and guidelines designed expressly to protect their integrity.

Q.  I read your post on the revolving door,  It seems to me that your underlying premise is the notion that any company that makes food is indicted as part of the big evil food conspiracy. Surely, you can’t really believe that. 

A.  Of course I don’t. But food companies are not social service agencies.   Their job—their legal responsibility—is to continuously expand sales and distribute ever-increasing profits to shareholders.   If they can do this and promote health at the same time, more power to them.  But people would be healthier eating food, not food products.  In our present system, products are far more profitable and the focus on them is rarely works in the interest of public health.

Food and nutrition professionals need to make a living.  Unfortunately, jobs in industry pay better–and sometimes a lot better–than jobs in government or NGOs.  That’s the real dilemma that underlies all of these questions.

The big nutrition scare last week was the study in Nature finding that in mice and, maybe, humans, artificial sweeteners mess up the microbiome and make some people even more intolerant of glucose.

The authors conclude that their results call for a reassessment of massive use of artificial sweeteners.

The study is complicated and difficult to read but the Wall Street Journal has a nice summary.  It explains why the study is getting so much attention:

The new Nature study marks a significant advance because it brings together two separate areas of research—the role of sweeteners in raising blood sugar levels, and the complex workings of the vast colonies of bacteria that inhabit the gut. Individuals can have differing bacterial colonies in their gut, meaning people respond differently to what they consume.

The study involved several experiments.  These found:

• Mice fed saccharin, sucralose, or aspartame had significantly higher blood-glucose levels than mice whose diet included sugar, or just water.
• Mice with sterilized digestive tracts, who were given bacterial transplants from artificial-sweetener-fed mice, displayed higher blood sugar levels than those receiving bacterial transplants from sugar-fed mice.
• People who typically use artificial sweeteners have different kinds of bacteria in their intestines than those who do not.  They also are more glucose intolerant.
• Seven volunteers fed artificial sweeteners for four days displayed higher blood-sugar levels as well as altered populations of bacteria in their gut.

The Wall Street Journal quotes the Calorie Control Council (the trade association of makers of artificial sweeteners).  The CCC said:

The results from the mouse experiments may not apply to humans, while the human experiments had a small sample size. It said further research was needed.

Despite my lack of enthusiasm for artificial sweeteners, I think the Calorie Control Council has a point.

The excellent report by Kenneth Chang in the New York Times explains why.

At present, the scientists cannot explain how the sweeteners affect the bacteria or why the three different molecules of saccharin, aspartame and sucralose result in similar changes in the glucose metabolism.

Chang ends with this:

Dr. Frank Hu, a professor of nutrition and immunology at the Harvard School of Public Health who did not take part in the study, called it interesting but far from conclusive and added that given the number of participants, “I think the validity of the human study is questionable.”

Here’s why I’m not fond of artificial sweeteners:

• They taste bad (to me)
• They have no demonstrated effectiveness in helping people lose or maintain weight.
• They are artificial, and violate my rule to “never eat anything artificial.”

Do they mess up the microbiome and cause glucose intolerance, insulin resistance, and metabolic syndrome?

That would be fascinating, but I’m reserving judgment pending further research.

In the meantime, I’ll take sugar—in moderation, of course.

Sunday’s New York Times discussed how foreign governments are dropping millions on Washington DC think tanks to fund reports explicitly designed to influence government agencies to set policies favorable to the donor country’s interests.  One such interest is food trade with Asia.

The line between scholarly research and lobbying can sometimes be hard to discern.

Last year, Japan began an effort to persuade American officials to accelerate negotiations over a free-trade agreement known as the Trans-Pacific Partnership (TPP), one of Japan’s top priorities. The country already had lobbyists on retainer, from the Washington firm of Akin Gump, but decided to embark on a broader campaign.

Akin Gump lobbyists approached several influential members of Congress and their staffs…seeking help in establishing a congressional caucus devoted to the partnership, lobbying records show. After those discussions, in October 2013, the lawmakers established just such a group, the Friends of the Trans-Pacific Partnership.

To bolster the new group’s credibility, Japanese officials sought validation from outside the halls of Congress. Within weeks, they received it from the Center for Strategic and International Studies, to which Japan has been a longtime donor. The center will not say how much money the government has given — or for what exactly — but an examination of its relationship with a state-funded entity called the Japan External Trade Organization provides a glimpse.

In the past four years, the organization has given the center at least $1.1 million for “research and consulting” to promote trade and direct investment between Japan and the United States. The center also houses visiting scholars from within the Japanese government…one Japanese diplomat…said the country expected favorable treatment in return for donations to think tanks.  “If we put actual money in, we want to have a good result for that money — as it is an investment,” he said.

Ah yes, the Trans-Pacific Partnership (TPP).  Let’s start with the context: The US has trade agreements with 20 countries.

These, says the US, provide many benefits:

Trade is critical to America’s prosperity – fueling economic growth, supporting good jobs at home, raising living standards and helping Americans provide for their families with affordable goods and services.

Obviously, Japan must think the TPP will do all that and more for Japan.

But trade negotiations run into trouble when one partner has higher food safety standards—Sanitary and Phytosanitary (SPS) Measures—than another.

This is an issue in current negotiations over the TPP.

Like other trade agreements, the negotiations are conducted in secret.

Over the past two years there has been a steady drip of stories about the secretive negotiations regarding the Trans Pacific Partnership (TPP). Members of Congress and congressional staffers have been stymied in their efforts to perform some measure of oversight while major corporations have reportedly been given unfettered access and influence over the deal. The public has been kept almost completely in the dark regarding negotiations that affect everything from food prices to our ability to innovate on the Internet…Historically, international trade negotiations have happened through the World Trade Organization…Trade negotiators have had trouble closing major deals over the past decade, in part because the public, and sometimes their elected officials, have stood up to decry these secret negotiations, demanding oversight opportunities and setting up websites to post leaked drafts of the agreements.

According to PoliticoPro, negotiations on TPP with Japan and 10 other countries in the Asia-Pacific region are close to conclusion, but pork producers, other farm groups, and Republicans are insisting that Japan eliminate all agricultural tariffs and completely open its market to U.S. agricultural exports.

Japan says it can offer improved market access but cannot eliminate all tariffs because of pressures from its own farm groups.

At the end of July, some House members demanded that Japan—and Canada—be dropped from the TPP if they do not agree to eliminating tariffs.

Trade negotiations work in predictable ways.  for example, despite the North American Free Trade Agreement (NAFTA—and also see this link— which allows unfettered imports from Mexico, the US is imposing tariffs on Mexican sugar to prevent flooding the US market with cheap supplies that undermine sales of US sugar (NAFTA allows this).

The moral: In trade agreements, each country looks to maximize its own interests.  Hence: Japan’s lobbying via Washington think tanks.

Yesterday, Food Navigator reported that Cornell economists calculated that GMO labels would cost the average family of four a whopping $800 per year.

This seemed so improbable that I immediately wondered:  Who paid for it?

I clicked on the link to the study: Bingo!

The work on this report was supported financially by the Council for Biotechnology Information.

You won’t find the list of companies and groups that support the Council on its website, but Source Watch fills the gap.

I am increasingly alarmed by the increasing extent of industry research sponsorship—it’s become a huge issue in  studies of nutrition, diet, and health.

The influence of funding source on research outcomes is so predictable—many studies have now shown that industry-funded studies almost invariably produce results that favor the sponsor—that I’m batting nearly 100% on conflict-of-interest  checks, of which this GMO study is a particularly blatant example.

It’s not that industry pays investigators to find the desired answers to questions.  It’s more complicated than that.  It has to do with the way investigators ask and try to answer the research questions.  The industry favored biases get built into the study’s assumptions and controls, often (I think) unconsciously.

This study, for example, is based on an elaborate set of assumptions leading to the $800 per family estimate.  Other assumptions might give different results.   The authors do not discuss the limitations of their estimates, nor are they required to in this type of report.

But I’m willing to hazard a guess that independently funded studies would come to considerably lower estimates.

Moral: if a study produces surprising results that favor an industry position, look hard to see who sponsored it.

Addition, May 24:

A reader sent in further information about the Council for Biotechnology Information:

Council for Biotechnology Information

1201 Maryland Avenue, SW., Suite 900, Washington, DC 20024 USA

Phone: 202-962-9200 web site: http://gmoanswers.com

(CBI: http://www.sourcewatch.org/index.php?title=Council_for_Biotechnology_Information.

http://www.powerbase.info/index.php/Council_for_Biotechnology_Information.

Experts: http://gmoanswers.com/experts. Founding members and supporting partners:

http://gmoanswers.com/about. There are also offices in Saskatoon (SK, Canada)

(http://whybiotech.ca)  and Mexico City (AgroBio Mexico: http://agrobiomexico.org.mx.)

I am a member of the American Society for Nutrition (ASN), the organization that publishes the American Journal of Clinical Nutrition (AJCN) and the Journal of Nutrition.

I’ve become increasingly worried about food company influence on ASN.  Food companies fund sessions at ASN annual meetings.

But I’m even more concerned about food company sponsorship of scientific studies published in AJCN.

The results of sponsored studies almost invariably benefit the sponsor.  Exceptions are scarce.

The conflicts are so blatant that I can often guess from reading an abstract who the study’s sponsor must be.

A look at the conflicts of interest disclosed by the editorial board of AJCN suggests why this problem is occurring.

Of the 12 members of the editorial board, only 3 disclose no corporate conflicts of interest, and 2 others disclose minor conflicts.

But the majority—7 of the 12—list major corporate affiliations.  The list of food companies for which they consult or advise is too long to reproduce but it includes Coca-Cola, PepsiCo, The Sugar Association, The National Restaurant Association, ConAgra, McDonald’s, Kellogg, Mars, and many others.

This raises uncomfortable questions: How does this editorial board deal with papers suggesting harm to health from consuming products from these companies?  How does it deal with sponsored papers suggesting benefits of the products?

Affiliations with food companies may or may not lead to publication bias, but at the very least they give the appearance of serious conflicted interest.  This affects opinion not only of sponsored studies, but also of the overall credibility of research published in the journal.

For the results of papers published in the AJCN to be considered credible, the editorial board should:

• List the editor responsible for review of published papers in the conflict disclosures.
• Recuse individual members with conflicts from reviewing papers in their area of conflict.
• Phase out conflicted editors as quickly as possible.
• Appoint editors who have minimal or no conflicts.
• Give special editorial scrutiny to papers sponsored by food and beverage companies.

ASN is not the only nutrition society raising doubts about its conflicts of interest with food company sponsorship.  The Academy of Nutrition and Dietetics (AND) is the subject of two recent reports analyzing its conflicts of interest.

• The Food Ties that Bind
• And Now a Word From Our Sponsors

I will say more about these reports tomorrow, but it looks like a similar report could be written about ASN, alas.

Imagine this.  A professional journal, Preventing Chronic Disease: Public Health Research, Practice and Policy (Vol 8, Issue 5, Sep 2011) has published a series of papers on the ethics of childhood obesity interventions. 

It is about time that these kinds of ethical issues are getting focused attention.  Applause for the journal’s editors and authors!

Ethical Issues In Interventions For Childhood Obesity

A92: Protecting Children From Harmful Food Marketing: Options for Local Government to Make a Difference
  Jennifer L. Harris, Samantha K. Graff

A93: Childhood Obesity: A Framework for Policy Approaches and Ethical Considerations
  Rogan Kersh, Donna F. Stroup, Wendell C. Taylor

A94: Childhood Obesity: Issues of Weight Bias
  Reginald L. Washington

A95: Children With Special Health Care Needs: Acknowledging the Dilemma of Difference in Policy Responses to Obesity
  Paula M. Minihan, Aviva Must, Betsy Anderson, Barbara Popper, Beth Dworetzky

A96: Public Policy Versus Individual Rights in Childhood Obesity Interventions: Perspectives From the Arkansas Experience With Act 1220 of 2003
  Martha M. Phillips, Kevin Ryan, James M. Raczynski

A97: A Question of Competing Rights, Priorities, and Principles: A Postscript to the Robert Wood Johnson Foundation Symposium on the Ethics of Childhood Obesity Policy
  Shiriki K. Kumanyika

A98: The Ethical Basis for Promoting Nutritional Health in Public Schools in the United States
  Patricia B. Crawford, Wendi Gosliner, Harvey Kayman

A99: Ethical Family Interventions for Childhood Obesity
  Mandy L. Perryman

A100: Public Policy Versus Individual Rights and Responsibility: An Economist’s Perspective
  Frank J. Chaloupka

A101: State Requirements and Recommendations for School-Based Screenings for Body Mass Index or Body Composition, 2010
  Jennifer Linchey, Kristine A. Madsen

A91: Ethical Concerns Regarding Interventions to Prevent and Control Childhood Obesity
  John Govea