Scientific knowledge is drowning in a flood of research

A comic about the problems with the -omics, illustrated by Matteo Farinella

What does it mean to do scientific research? When I was an undergraduate this question seemed obvious.
First we formulate a theory. Then we perform experiments in order to reject or validate the theory. However in reality things are rarely that simple… For example, in 1963 Bernard K. Forscher published a letter in Science to call attention to the fact that many scientists, driven by socio-economic pressures, were becoming too focused on data-collection, neglecting theory building.
Today, in the age of "big science," the gap between data and theory has grown even wider. Enormous amounts of time and resources are invested in mapping any sort of -omes: genomes, proteomes, microbiomes, connectomes…often driven by strong commercial interests.
In this new post-academic world funding and career dynamics seem to value volume rather than depth, and leave little space for scientists to connect this ever expanding universe of data points. While the scientific community struggle with these issues, I think Forscher's story is still useful to keep the conversation alive. So here is my adaptation of the original letter.
Chaos in the Brickyard, by B.K. Forscher

Illustrated by Matteo Farinella

Once upon a time, among the activities and occupations of man there was an activity called scientific research and the performers of this activity were called scientists. In reality, however, these men and women were builders who constructed edifices, called explanations or laws, by assembling bricks, called facts.
When the bricks were sound and were assembled properly, the edifice was useful and durable and brought pleasure, and sometimes reward, to the builder. If the bricks were faulty or if they were assembled badly, the edifice would crumble, and this kind of disaster could be very dangerous to innocent users of the edifice as well as to the builder who sometimes was destroyed by the collapse.
Because the quality of the bricks was so important to the success of the edifice, and because bricks were so scarce in those days the builders made their own bricks. The making of bricks was a difficult and expensive undertaking and the wise builder avoided waste by making only bricks of the shape and size necessary for the enterprise at hand.
The builder was guided in this manufacture by a blueprint, called a theory or hypothesis. It came to pass that builders realized that they were sorely hampered in their efforts by delays in obtaining bricks.
Thus there arose a new skilled trade known as brickmaking, called junior scientist to give the artisan proper pride in his work. This new arrangement was very efficient and the construction of edifices proceeded with great vigour.
Sometimes brickmakers became inspired and progressed to the status of builders. In spite of the separation of duties, bricks still were made with care and usually were produced only on order. Now and then an enterprising brickmaker was able to foresee a demand and would prepare a stock of bricks ahead of time, but, in general, brickmaking was done on a custom basis because it still was a difficult and expensive process.
And then it came to pass that a misunderstanding spread among the brickmakers (there are some who say that this misunderstanding developed as a result of careless training of a new generation of brickmakers). The brickmakers became obsessed with the making of bricks.
When reminded that the ultimate goal was edifices, not bricks, they replied that, if enough bricks were available, the builders would be able to select what was necessary and still continue to construct edifices. The flaws in this argument were not readily apparent and so, with the help of the citizens who were waiting to use the edifices yet to be built, amazing things happened.
The expense of brickmaking became a minor factor because large sums of money were made available; the time and effort involved in brickmaking was reduced by ingenious automatic machinery;
The ranks of the brickmakers were swelled by augmented training programs and intensive recruitment. It even was suggested that the production of a suitable number of bricks was equivalent to building an edifice and therefore should entitle the industrious brickmaker to assume the title of builder and, with the title, the authority.
And so it happened that the land became flooded with bricks…
It became necessary to organize more and more storage places, called journals, and more and more elaborate systems of bookkeeping to record the inventory. In all of this the brickmakers retained their pride and skill and the bricks were of the very best quality.
But production was ahead of demand and bricks no longer were made to order. The size and shape was now dictated by changing trends in fashion.
In order to compete successfully with other brickmakers, production emphasized those types of brick that were easy to make and only rarely did an adventuresome brickmaker attempt a difficult or unusual design. The influence of tradition in production methods and in types of product became a dominating factor.
Unfortunately, the builders were almost destroyed. IT became difficult to find the proper bricks for a task because one had to hunt among so many. It became difficult to find a suitable plot for construction of an edifice because the ground was covered with loose bricks.
"It became difficult to complete a useful edifice because, as soon as the foundations were discernible, they were buried under an avalanche of random bricks. And, saddest of all, sometimes no effort was made even to maintain the distinction between a pile of bricks and a true edifice."

Reprinted with permission of the family of Bernard K. Forscher, with special thanks to Geoffrey P. Forscher.

Peer Commentary

Feedback and follow-up from other members of our community

Allison Fritts-Penniman

Ecology, Evolutionary Biology

I had not read Bernard K. Forscher’s letter before, and it’s awesome to experience it for the first time through this comic, because it makes the metaphor to academia and the scientific process so incredibly clear. I’m surprised that in 1963 Forscher already saw a trend towards the overproduction of data (bricks) and PhDs (brickmakers). 

However, I would like to tinker with the metaphor a bit to reflect what I actually see happening. I think there is always a particular edifice in the minds of the brickmakers; scientists are still driven by hypothesis-testing as they collect data. But instead of coming up with new hypotheses, we are using new data to test the same ones over and over again. 

We brickmakers aren’t given enough time to come up with new building plans, so we focus on those that are well established. 

My field was initially very theoretical, and now has been flooded with studies that use genomics to re-test the same theories using different techniques and organisms. It’s not that we’ve created a pile of bricks and no edifice, it’s that scientists are building over the old bricks with fancier bricks on the same edifice.

 Dan Samorodnitsky

What's that joke about computational biology solving the same three problems over and over?

Devang Mehta

Genomics

University of Alberta

This is such a fantastic illustration of a problem with some modern science. I, however, think the problem’s not as big as it’s made out to be. 

I am one of a small set of scientists who think that we don’t yet have enough data/bricks. In my view, we still need data to form predictive models and hypotheses about scientific phenomena, but we need to diversify and connect the various data-types we use to do so. A lot of us are doing this now, using different methods of data production to correct and refine the overall structures of modern scientific thought. Oftentimes more bricks can demolish bad foundations and replace them with something stronger. 

Yes, science – especially through new techniques – does sometimes build on older theories with fancier-seeming bricks. But sometimes using fancy bricks reveals facades that would have remained crudely formed, or even incorrectly built. 

For example, only through the advances in genomics (and the massive amounts of data produced) were we able to leave behind the terrible gene-centric  theory of molecular biology that impeded further discovery for decades.

Dan Samorodnitsky

Biochemistry

SUNY Buffalo

I don't think the issue is that there's too much or too little data. There's always been more data than anyone knows what to do with, even before the -omics era. It's like baking bread: you end up with a loaf but your kitchen will be covered with random scraps of dough that end up in the garbage. Every project I've worked on produced interesting data which didn't fit in anywhere and wasn't interesting enough to pursue. 

The greater issue that the letter and comic illustrate, in my mind, is that -omics approaches to science prioritize correlating data sets with each other while leaving behind any thought about mechanisms, why things are the way they are. 

That, to me, is the edifice. Drawing a map from point A to point B is worthless if you don't pave the road between them. I can't count the number of papers I've read that did mind-bending analysis to show that, say, drinking red wine increases expression of a bunch of genes without stopping to consider why or how or if it even matters. You can base an entire lab on finding new things that haven't been sequenced and sequencing them. That's a viable approach to research today.

 David Haggerty

But even with the -omics breakthroughs, we still don't have the tools to understand what the mechanisms are. Obviously some of the problem is our approach, biases, what is profitable, but we neuroscientists can't figure out how to explain a microchip that runs donkey kong, let alone a neural circuit. It's hard to deem what's important and why, when we still don’t have the full picture in front of us.

 Dan Samorodnitsky

All of this makes me feel uneasy, because I've spent my career so far focusing on "small" projects about how proteins function. I literally lie awake at night wondering if I'll be able to get a job in the future with my... old-fashioned skill set.

I don't think -omics approaches are bad or wrong or don't have their place. I just want to live in a world where there's money and appetite for both the forest, which computational approaches look at, and the trees.

 David Haggerty

You’ve hit it on the head.

 Allison Fritts-Penniman

This touched the job anxiety nerve for me as well, thinking about the droves of specialized PhDs coming out of academia, many of whom are competing for a small set of jobs that don’t actually require that level of specialization.

Matteo Farinella

Neuroscience

Columbia University

I love the bricks/buildings metaphor because even if it certainly oversimplifies some things (as any metaphor necessarily does) I think it's a great way to visualize scientific practice, especially for our non-scientist friends and policymakers.

Just to reply to a few points in random order:

1. I didn't write the letter, so I don't know what Forscher actually meant, but for me the message is not that too much data is 'bad' or theories are somehow more 'noble.' It's just a reminder that to make science we need both: a pile of data without theory doesn't explain anything (it's  a description, as Samorodnitsky said) just like a theory unsupported by data is completely useless (just like a beautiful drawing of a building that never gets built).

2. By that I don't mean that we all need to do both! And I certainly wouldn't want to suggest that 'brickmakers' are a lower caste, or untrained PhD. Especially in today's age I think it makes perfect sense to have experimental scientists mapping all the -omes they want, and theoretical scientists trying to connect the dots. The problem is that training and funding often favors data over theory (probably because, unlike data, theories are difficult to quantify).

3. Also, Mehta makes a good point: we clearly can't generalize across sciences. Some fields are much more balanced (for example I think physicists are pretty good at giving credit to both theory and experiments). Other are producing mountains of data; others get a bit carried away with their theories (believe me – my PhD was in computational neuroscience!)

Anyway, I hope the job anxiety doesn't get too bad, especially over the holidays! Just keep going, one brick at a time 😀

 Allison Fritts-Penniman

The letter itself equates brickmakers with junior scientists, compared to the builders, so it seems to me that Forscher was drawing attention to the fact that many students and postdocs are hired to collect data for their PI rather than being encouraged to develop their own theories to test. I guess I would take that further and say they are getting a different (not better or worse) kind of training as someone who achieves their degree by pursuing their own independent projects.