User login

Enter your username and password here in order to log in on the website:

Forgot your password?


If you've not registered yet, you can now register.

Only registered users can write contents of discussion.

Registration site

Organic Research Forum

Thanks for visiting the Organic Research Forum

Welcome to the Organic Research Forum, a platform to exchange ideas and discuss the latest important topics related to organic agriculture research.

The forum is for registered users only!

If you've already registered as a user, please log in to use all of the functions of this forum. If you haven't yet registered, click on either of the links on the left or right to register.

If you are registered, you can post questions, comments and information to the Forum.

The left navigation bar has link to "Private Messages" to facilitate direct private contact with other forum members. 

Brian Baker

Forum administrator

Unanswered topic

The Stanford study: The challenge to make organic betterThe challenge to make organic better

Author Message
Written on: 07. 09. 2012 [17:45]
Brian Baker
registered since: 23.08.2012
Posts: 5
The study performed by Stanford University scientists Smith-Spangler, et al., (2012) adds little to the on-going literature and debate over whether organic food is ‘different’ from conventional food. They claim to have conducted a comprehensive analysis, but a number of key studies were not included and several that were included were interpreted in a misleading way.

With great caution, the authors also concluded that organic food had two health benefits supported by scientific evidence: 1) reduced risk of exposure to pesticides and 2) lower likelihood of antibiotic resistant foodborne bacteria. On the whole, the study cast organic in a positive light. In addition, the literature reviewed found that organic food is not significantly different from conventional food in the risk of food-borne pathogens, despite unsubstantiated claims by various proponents of chemical disinfectants and food irradiation to the contrary.

The main conclusion, however, is that the studies do not show a significant difference in the levels of nutrients. The one case where organic food has significantly higher nutritional value, phosphorous, is rarely deficient. In other cases where nutrient content is higher in organic, the results are insignificant on aggregate and not robust.

The results found in Smith-Spangler are not consistent with other recent reviews performed. Researchers led by Kirsten Brandt at the University of Newcastle went over many of the same studies and reached a different conclusion (Brandt, 2011). Brandt’s performed a meta-analysis of the published comparisons of secondary metabolites and vitamins in organically and conventionally produced fruits and vegetables. Secondary metabolites, such as anthocyanins, tocopherols and bioflavonoid were found to be consistently higher in organically grown fruits and vegetables by a significant margin. The Brandt analysis also found vitamins to be higher, although not with as much significance as the secondary metabolites.

An earlier study conducted by Chuck Benbrook and others also reviewed the majority of the same articles cited by the Smith-Spangler article and also concluded that the literature supports claims that organic food has significantly higher levels of various nutrients, including omega-3 fatty acids and many different antioxidants (Benbrook, 2008 ).

While both Brandt et al. and Benbrook et al. noted the heterogeneity of the studies, these studies found the differences in antioxidants and vitamins still to be significant and consistent, although not universally supported. Growing practices, varieties, weather and other confounding factors can reduce the significance of analyses, as can small sample sizes, but the overall indication is that organic food has higher levels of a number of nutrients.

Smith-Spangler et al. used statistical methods that are relatively unusual in the field. One measure they used is called “Risk Difference” (RD). The researchers used RD to compare the relative risks of the different methods, rather than the absolute risks. This understates the difference in risk by a substantial margin. In addition, the authors ignore the difference in risks posed by multiple pesticide residues and multiple antibiotic resistant pathogens, both of which are far more likely to occur in conventional than organic products. While the methodology is designed to be cautious in not accepting false positives, it is more likely to result in false negatives and quantification of that differences found can be misleading or misinterpreted.

The Challenge
Even with the caution and understatement, the study’s qualified conclusions that organic food contains health benefits, the Stanford Study has gotten media attention from many sources with the main message that organic food is no different from conventional. The irony is that the researchers’ caution and extremely conservative methodology has led to misleading statements in the media. The popular press repeated the conclusions of no difference without much depth of reporting on the methodology involved.

Healthy skepticism is an important part of science. The methodology used in the study was extremely conservative at finding differences between the two systems. As weak and as qualified as they are, that makes the findings in favor of organic all that much more powerful. Rather than condemn the researchers for their findings, researchers who are working on organic farming systems should offer their gratitude for conducting such a far-reaching review.

The authors evaluated the abstracts of almost 6,000 articles and ended up picking 237 as worthy of comparison. Many of their screening criteria are reasonable and should be followed by the scientific community. One of their screening criteria was to eliminate studies where they found a funding bias. The organic community has long criticized conventional farming research for biases in favor of farm chemicals when the research is paid for by the fertilizer, pesticide, feed additive and pharmaceutical industries. It is only fair and reasonable that organic farming research be held to the same high standard for objectivity. The systems needed to be clearly and fully organic and not simply share common practices with organic or in transition to organic farming. Other comparisons have expected no less. Another reason some studies were not included is that they didn’t report their sample sizes.

The challenge is to develop a better scientific understanding of what growing practices yield healthier food, but not in a crude, reductionist way. The American journalist Michael Pollan critiques the pseud-science of ‘nutritionism’ and the illusion that foods can be fortified with individual nutrients to optimize nutritional needs.

Poorly designed experiments, weak methodologies, small sample sizes and poorly written articles will not withstand close scrutiny from a skeptical scientific community. Story-telling can only go so far without hard data to back it up. To address the pressing issues of how to feed the world for this and all future generations, the world needs more research on different farming systems, agro-ecology, and food quality. The science behind it will need to be first-rate. Researchers who work on organic farming systems cannot be just average, we need the best scientists because their work will be torn apart by editors and peer-reviewers to be published in journal in ways that those who work in the dominant paradigm do not face.

Calls for more research funding for organic farming and food are unlikely to be taken seriously unless researchers are able to justify the expenditure on results. Research methods in organic farming systems must improve if the science is to be of suitable quality to draw meaningful conclusions. That means working tightly within the existing paradigm.

At the same time, we need create and build a whole new paradigm. Basic research, particularly in agroecology, That new paradigm is not just going to invent itself. Doing so will take hard work by top scientists who are willing to risk their careers and reputations on path-breaking work, rather than settle for the safer routine work of the dominant paradigm. Agricultural science will need to shift from a linear input-output model cranking out the calories to a model that efficiently cycles nutrients with minimal losses to the environment. And (here’s my bias as an economist coming out) it will take a restructuring of market incentives so that people get what they pay for and pay for what they get, both good and bad. The medical profession will need to be restructured to look at wellness and prevention, rather than treatment and cure.

Despite the Stanford study’s methodological flaws and misinterpretations by the media, researchers in organic food should carefully and critically read it and take up its methodological challenges. While many of the conclusions reached in that study are not completely accurate or substantiated, they are not so easily refuted either. The only way to refute the challenge posed by the authors is to conduct the research in a way that meets rigorous scrutiny. Research is judged not by how well the conclusions please its patrons, but how much they satisfy its critics.

Organic food and agricultural research should work to improve on the existing differences to make the advantage more clear cut. More and better studies are needed to compare foods grown in different system. However, a higher priority is the development of methods, technologies and varieties that deliver healthier food. Rather than have the standards drive the research, rigorous research based on health, ecology, care, and fairness should drive the standards. Where research shows we can improve, we should make the needed improvements through best practices and through revision of the standards.

Because of the scrutiny with which organic food is examined, it is crucial for researchers on organic farming systems to conduct their research impeccably. While the reductionist science is challenging at best for organic farming researchers and is wholly inappropriate to address the most important questions that face agriculture and the food system of the future, it is still the dominant paradigm against which organic farming will be measured until a better paradigm is developed. That means organic farming research must do double the work. Evidence shows that the benefits of organic food are real, tangible and well-supported. These benefits should not be overstated and organic food’s quality and safety can be improved.

Organic agriculture does not yet have all the answers to the world’s problems, but neither does conventional agriculture. We must openly, critically and rigorously examine organic farming systems, accept its flaws, faults and shortcomings as much as we promote its benefits and embrace the promise for a better future that organic farming has to offer.

More Information


Charles Benbrook, Xin Zhao, Jaime Yáñez and Neal Davies. 2008. New evidence confirms the nutritional superiority of plant-based organic foods.

Kirsten Brandt, Carlo Leifert, R. Sanderson and C.J. Seal. 2011. Agroecosystem management and nutritional quality of plant foods: The case of organic fruits and vegetables. Critical Reviews in Plant Science 30: 177-197.

Johannes Kahl, Ton Baars, Susanne Bügel, Nicolaas Busscher, Machteld Huber, Daniel Kusche, Ewa Rembia?kowska, Otto Schmid, Kathrin Seidel, Bruno Taupier-Letage, Alberta Velimirov and Aneta Za??cka. 2012. Organic food quality: a framework for concept, definition and evaluation from the European perspective. Journal of the Science of Food and Agriculture in press.

Crystal Smith-Spangler, Margaret L. Brandeau, Grace E. Hunter, J. Clay Bavinger, Maren Pearson, Paul J. Eschbach, Vandana Sundaram, Hau Liu, Patricia Schirmer, Christopher Stave, Ingram Olkin, Dena M. Bravata. (2012) Are organic foods safer or healthier than conventional alternatives? A systematic review. Annals of Internal Medicine. 157: 348-366.