What can you believe in drug discovery?

I was intrigued to read in the September issue of Nature Reviews Drug Discovery (http://www.nature.com/nrd/journal..) that Bayer halts nearly two-thirds of its target-validation projects because in-house experimental findings fail to match up with published literature. It seems that we can’t always believe what we read in the scientific literature; I guess we all know that you can’t rely on every publication but I’m staggered that as much as 65% can’t be relied on.

The Bayer team reviewed 67 target-validation projects, covering the majority of Bayer's work in oncology, women's health and cardiovascular medicine over the past 4 years (http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html). Irreproducibility was high both when Bayer scientists applied the same experimental procedures as the original researchers and when they adapted their approaches to internal needs (for example, by using different cell lines). Surprisingly, the confirmation of any given finding by another academic group did not improve data reliability.

Some of the causes of irreproducibility may simply result from poorly understood science in an emerging field but there may be more systemic causes resulting from the pressure to publish high impact papers. Other factors may include a failure to work to industry standards and a lack of industry perspective (e.g. 'safe and well tolerated' may mean one thing to an academic scientist and something quite different to an industrial scientist.

The fact that even findings reported by more than one team tend to be irreproducible in industry hands highlights another potential problem: the sharing of materials between academic groups without checking them out properly. Prof Phil Cohen of the Division of Signal Transduction Therapy at the University of Dundee commented “We stopped borrowing materials back in 1996, because of problems we inherited from the clones and samples sent in by other labs.”

This lack or reproducibility is impacting significantly on both large and small companies engaged in drug discovery. Bayer has already changed its approach to target validation and become much more cautious when working with published targets. Bruce Booth of Atlas Ventures relies on contract research organizations (CROs) to confirm published data before moving forward with an investment opportunity arising from academic research. He also cultivates ties with researchers whose work has repeatedly proved to be reproducible.

Booth also argues that whereas many university technology transfer offices (TTOs) have set up seed funds to spin companies out of academic research, the TTOs would better serve both the universities and the broader community by funding either CROs or other research teams to independently validate claims. “Their data packages would then become so much stronger and better able to attract early-stage investors,” he says. “A lot of biotechs are currently formed sooner than they should be,” he adds.

There is plenty of food for thought in both the Bayer study and the comments by Bruce Booth. Certainly our own experience at Ithaka Life Sciences is that many university spin out companies are formed far too early. Maybe a new kind of business model is required for drug discovery spin outs; for example, an entrepreneurial CRO might partner with a university to jointly produce a robust data package that then forms the basis of a stronger spin out company in which both partners have founder’s equity. Anyone know of an entrepreneurial CRO?