The importance of failure in graduate student training

Running the winch at dusk

The A-frame shuddered as the box core, heavy with mud and reeking of sulfur, emerged from the water. We knew that it had found its mark 2300 meters below. Soft sediment from the seafloor oozed out the sides as I slid the safety pins into the spade arm. There was nothing visibly special about this mud. No ancient arthropods or primeval polychaetes crawled through this muck. It was a cubic meter of sticky, stinking glop. My first sample.

We were in the middle of the Gulf of Mexico, aboard the R/V Cape Hatteras. Our cruise objectives were to characterize the pelagic and benthic fauna associated with deep-sea methane seeps. For me, it was a ship of opportunity. In exchange for and extra set of hands to work the gear and process samples, I could add my own small research project to the cruise objectives. My goal was to collect sediment cores from multiple sites and survey the diversity of fungi associated with these methane seeps.

The 12 hour shifts rarely left me enough time to eat meals. Though I had never seen the equipment before we left port I became the acoustic tracking technician, out of necessity. Things consistently went wrong. Nets tore, gear broke, a misfired box core almost crushed my leg. Two hurricanes, one a category 5, hit the Gulf of Mexico while we were at sea. Work was exhausting and rest was brief, when existent. I loved every minute of it.

The end of that cruise was the high point of a 4 year project that began with unbridled optimism and early, exciting results, only to decay into drudgery, failure, desperation, and collapse. In the end, it would rise from the past for one small victory. In hindsight, so much of those four years seems painfully trivial, but this story is really about how much of a human being is poured into a scientific manuscript.

When I entered my PhD program, as is often the case, I had a grand vision for my research. The field of deep-sea mycology was wide open. Only one major research program and a few opportunistic studies had been conducted on fungi from the abyssal depths. I wanted to be a deep-sea biologist since I was a child, but had spent my undergraduate years working in a mycology lab. Looking out from a position of inexperience, it felt like I was the perfect candidate to merge the two fields.

Inexperience is a funny thing. Sometimes, if you don’t know when limitations exist, you can plow through them and make unexpected discoveries.

That’s not how things turned out for my project. During the first year, I had some success, a few dozen sequences amplified from the deep sediment. I was navigating with course work and the bureaucracy associated with graduate school, so those first few results felt so much bigger than they were. I entered the fall of my second year with a plan, based on those early successes, to explore not only fungi from Gulf of Mexico methane seeps, but also a rare artifact on the Carolina coast – a natural shallow water methane seep. I celebrated my first sampling trip as one of the few people who have ever SCUBA dived on a methane seep, collecting filamentous Beggiatoa by hand.

And that’s when things fell apart.

Our early results in the laboratory were not indicative of future success. We had lucked out by choosing the one sediment core among 400 that contained abundant, easy to amplify, environmental DNA. In retrospect, I should have expected declining yields, as we picked the core that had the best chance of success to start on. The rest of the cores yielded few products. On top of that, several grants I had applied for fell through – reading back through the comments, it’s clear the reviewers could see the problems with my experimental design that I could not. There was no real question being asked, no vision for how the project could progress, and no solid justification for it’s intellectual merit. “Characterizing Fungi” is a poor proposal. My second year was a desperate one, with weeks of all nighters in the lab, months of disappointing failures, and no hope in sight. At my worst, there was moments when leaving my graduate program seemed like the only way out. Not used to academic failures, I broke down, and spiraled into months of insomnia, anxiety, and heavy drinking.

So where was my advisor? I couldn’t see it at the time, but she had been supporting me the entire way. The strange reality was that I had placed myself in a trap of my own creation, unwilling to see the wider scientific world that I could explore. While always encouraging me to pursue my own interests and create my own research program, she had ensured that I was embedded in other projects, sending me out on research cruises to support other research programs. When the moment came that we finally sat down together and said “enough”, set a drop dead date for my deep-sea fungus project, after which it would be terminated, I was already positioned to transition seamlessly into another project.

And that was it. For the next two and a half years, my research has focused on population genetics in the deep sea. No fungus. That project failed, and in a way, looking back on those two years, I am thankful for that failure. It made me realize what the real cost of research is, resolved my commitment to the process of doing science, and left me with experience that could not be otherwise gained. I learned how to value my time and balance research with having a life that didn’t depend on success and failure in the lab. There’s a paper that frequently makes the rounds among new graduate students – The importance of stupidity in scientific research. Perhaps we should also recognize the importance of failure in scientific research. When creating a young scientist, failure is as important as success. At least it was for me.

I’m trying not to be too self-indulgent, but given how close these events still are, it’s proving difficult. Entering my final year as a graduate student, I’ve seen the same events play out for dozens of new students. Some are fortunate, like I was, to have an advisor who made sure I landed on my feet after that first fall. Others are not, and find themselves leaving their program, and science, for good. The best advice I can pass on to new students is accept failure, and accept that sometimes that failure is yours alone.

There’s a saying we have out here, where the shoals move daily, merging and breaking, blocking channels and creating new ones, – “If you haven’t run aground in Beaufort, you haven’t spent enough time on the water.”

Sometimes you should expect surprises.

More than a year after my fungus work ended, a paper popped into my RSS feed from a researcher in Japan looking at fungi from Pacific methane seeps. In it they had identified a unique phylotype that had never been sequenced anywhere but the deep-sea and was strongly associated with microoxic sediments around methane seeps. I starred at the phylogenetic tree for a few minutes, recognizing the position, deeply embedded within the Ascomycetes, but far out on its own, only loosely related to anything else. It was eerily similar to one of my own, long abandoned, fungal phylotypes.

Suddenly, I found the project reignited, but this time, I had a hypothesis, a plan, and an understanding of what was necessary, and the experience to bring the project to a conclusion. My original vision of characterizing deep-sea fungi from the Gulf of Mexico evolved into a concrete test of the hypothesis that a deep-sea fungi from Pacific methane seep sediment was identical to a deep-sea fungi from Gulf of Mexico sediment. As it turned out it was, and Ascomycete phylotypes recovered from a Gulf of Mexico methane seep are identical to an uncultured deep-sea fungal clade from the Pacific was just published in Fungal Ecology. It was short, simple, and elegant (at least in my opinion). Had I started with a smaller project, instead of some grandiose vision, I probably would have had a much better time those first few years.

I don’t have any big closing statement. I can’t boil down this experience into some concrete finale. If you’re a graduate student, sometimes life just sucks. You may be able to deal with it, you may not. You will fail, and probably fail hard. But your experience is not unique, most young scientists will go through the same thing at some point.

Thaler, A., Van Dover, C., & Vilgalys, R. (2011). Ascomycete phylotypes recovered from a Gulf of Mexico methane seep are identical to an uncultured deep-sea fungal clade from the Pacific Fungal Ecology DOI: 10.1016/j.funeco.2011.07.002


  1. Kevin Z · September 13, 2011

    Well done Andrew. I love how you framed your experience, knowing hindsight is 20/20, as a lesson learned. I still think you are completely full of shit, but I continue to admire your response to a situation that would have ended many grad students’ experiences and how you’ve embraced a totally unrelated field and made it your bitch. Looking forward to seeing how grow after grad school is put behind you.

  2. Dan · September 14, 2011

    Thanks for the post.
    Being someone with a somewhat similar story, but probably leaving science, it’s good to read one with a good ending (despite all the troubles). Gives me a bit of hope =)

  3. Mike · September 14, 2011

    Interesting piece. Makes you really appreciate when things go right, after having them go so wrong. Definitely worth sharing with other grad students.

  4. Danna Staaf · September 16, 2011

    I am very impressed by your mature and thoughtful response to an extremely trying situation! However, I disagree with the idea that, systemically, scientist trainees benefit from failure–just as I don’t think they benefit from feeling stupid. In my opinion, Schwartz confused “feeling stupid” with “asking questions that no one knows the answer to.” The latter is an exciting part of science; the former is a painful and humiliating result of feeling (and being told) that it’s your fault when things go wrong. One builds confidence, the other undermines it.

    Similarly, failure could refer to many things. But when it’s on the scale of conceiving and pursuing a line of research that a more experienced scientist could recognize as a dead end, I have a hard time not seeing that as a failure of the mentor–or of the entire training system. One of the neat things about humans is that we can learn from other people’s mistakes, and I think a Ph.D. training program ought to provide students with some guidance on their first large research projects. The sink-or-swim “training” that I and so many of my peers experienced may occasionally produce good science, but along the way it wastes time, resources, and goodwill. And maybe potential scientists, as well.

    Just to be clear, I’m not saying that your experience was necessarily a waste of anything–it sounds like you’ve been able to glean a lot of good from those years of struggle. In may case, though I slogged through plenty of failure and feeling stupid to get my Ph.D., it left me with very little desire to be a scientist.

    Fortunately, I’m very happy with what I’m doing instead, so maybe it was all for the best. =)

  5. Rick MacPherson · September 19, 2011

    …I broke down, and spiraled into months of insomnia, anxiety, and heavy drinking.

    You sure you’re not a PhD candidate in the Philosophy Dept. at Harvard?

  6. TheBrummell · September 21, 2011

    I failed out of a science PhD program, about 3 years ago. I’m now close to 1/2 through a second attempt at a PhD, in a different field, different University, and with a very different advisor.
    Your story is well told, and valuable. Thanks for putting this up, I think I’ll re-read it in a little while after I’ve had a chance to think some more about the similarities and differences between your experiences of grad-school-failure and my own.

Comments are closed.