Every few years, book publishing is hit by a major scandal. Someone like Jonah Lehrer is caught making up quotes. There were supposedly major factual inaccuracies in the controversial Primates of Park Avenue, and passages in Ben Carson’s book were lifted and published without attribution. A few years back it was revealed that Herman Rosenblat’s planned Holocaust memoir was entirely fraudulent. Just this month, Gay Talese disavowed his own book and refused to promote the forthcoming The Voyeur’s Motel because one of his sources proved to be uncredible—which he discovered only after the Washington Post fact checked his book when an excerpt of it ran in the New Yorker.
Though these high profile instances get a lot of attention (a headline for a 2015 New York magazine piece begins with Will Book Publishers Ever Start Fact-checking?), the reality is that these cases are really just examples of hubris and stupidity—misinformation is passed on to readers on a regular basis. It’s usually unintentional.
There is a popular business book, Eat That Frog! by Brian Tracy, that says in its introduction:
“Mark Twain once said that if the first thing you do each morning is to eat a live frog, you can go through the day with the satisfaction of knowing that that is probably the worst thing that is going to happen to you all day long.”
It’s really a brilliant observation. The only problem is that Mark Twain never said such a thing. There’s a loosely translated remark from Nicolas Chamfort along similar lines (“we should swallow a toad every morning, in order to fortify ourselves against the disgust of the rest of the day”) but certainly nothing from Twain along those lines. Imagine, misattributing the quote which not only serves as your title but as the entire thesis for a book.
And with the exception of a site called Quote Investigator, almost nobody has noticed. This is for a book that has sold, at least according to the claims on its cover, 1.5 million copies.
I don’t mean to single out the author. I’m just trying to point out something that has been pointed out before but most people still don’t understand: Books are rarely fact checked.
While publishers do many rounds of editing for spelling, grammar, clarity, and even legality, they, for the most part, defer to the authors when it comes to the validity of the material. Certainly, none of the books I’ve worked on in ten years in the industry have been given more than a cursory check by a publisher. Even college papers are checked for plagiarism, but books aren’t. While we know that blogs and most online journalism isn’t always trustworthy, books carry a weight with readers—imbued in them by centuries of leather bindings and cultural bibliophilia.
For a busy, overworked author—especially one without a team of assistants—this can be a recipe for disaster (and abuse). Particularly when there is so much to gain from neat writing that tells readers what they want to hear. As a fellow young author burning the candle at both ends, I wasn’t so quick to see Jonah Lehrer as a monster. He was closer to a cautionary tale (especially when I found out that we had the same speaking agent and later, the same editor). There but for the grace of God go I...
Which is why in 2015, with all this weighing on my mind, I decided to subject the book I had finished, Ego is the Enemy, to a vigorous round of independent fact checking. Given that my book was primarily about the dangers of overconfidence and the toxicity of self-absorption, I figured it was only appropriate at this point in my career—on the heels of a successful book—that I take my own assessment out of the picture and submit to an objective accounting from some third parties. So after the manuscript was accepted by my publisher but before it went to be copyedited and laid out, I embarked on what would be one of the most uncomfortable and interesting experiences of my writing career. I would put myself through the ringer—voluntarily.
Despite my enthusiasm, the journey didn’t get off to a great start. First off, there is no service designed specifically to check manuscripts for plagiarism. This struck me as odd in the world where more than a quarter of a million books are published every year (if that’s not a market, what is?) In college, I had remembered hearing of a service that my professors used, so I reached out on Twitter for help.
I was sent to something called Turnitin, but without a university ID, I was out of luck. With some work—at one point, attempting to break the manuscript into many paper-sized smaller pieces—I was able to submit the entire 58,000 word manuscript to WriteCheck by Turnitin, purchasing 12 credits to cover the word count, at the cost of $101.70. Unlike the professional edition for instructors (Turnitin), WriteCheck for the general public does not show the sources of the suspected plagiarism and the papers submitted are not added to the Turnitin database, but did promise to check my work against 45+ billion webpages, 337 million student papers, and 130,000+ published works. Better than nothing.
Obviously I knew I had not intentionally plagiarized anyone, but what if I had accidentally written something down during the research process and the later confused it with an original thought? What if a line had come to me in the course of writing that was actually a quote or was dangerously similar to one? These are the fears that keep a writer up at night, especially If something like that could happen to talented and ethical writers Stephen Ambrose and Doris Kearns Goodwin. Certainly, they were much more talented and organized than I ever hope to be, and yet…
As Goodwin later explained in Time Magazine, “the more intensive and far-reaching a historian's research, the greater the difficulty of citation. As the mountain of material grows, so does the possibility of error.” This is compounded for those of us who still do even a portion of the work longhand instead of digitally. I have taken notes on hundreds of books over the years, which I then transfer to thousands of notecards, organized in file boxes by theme. These cards are what ultimately make up the details, quotations and facts I have used in my books. It seemed to me that it was inevitable that I would eventually slip up—and the result might be some career ending mistake.
Which is why, as the book was being analyzed, I also asked my editor to refer me to a fact checker. I’d already used a research assistant as I’d written the book to check as we went, but I wanted someone with fresh eyes. There is not exactly a file of fact checkers on call at Penguin, but she was nice enough to help me track someone down, and I was able to reach out to Chris Morgan, a former editor for Vibe (now a writer for Vice) who also worked freelance as an editor and fact checker for other magazines and projects. We settled on $1,500 to check the manuscript (which I paid out of my advance).
I explained to Chris that my interest was in having an impartial third party audit and fact check every word in the book. Though it was important that all the literal facts be true—dates, names, places, attributions—I also asked that he check assertions and claims I made. I wanted to make sure that each of these was backed by at least one trustworthy third party source. Since what I had written was a book of historical stories plus analysis, I needed to be sure that my analysis was as grounded in truth and fact as I believed it to be. And what we worked out was that for any claim that he couldn’t find verification for, he would mark and I would provide him my claims—which put me in the unique position of defending my work to a total stranger.
Once both of these two inspections were underway, the nervousness set in. I recalled a Mitch Hedberg joke about an AIDS test—it doesn’t matter what you’ve been doing, waiting for the results is terrifying. I knew I hadn’t plagiarized anything. I believed I’d done my research and erred on the side of caution with my facts. And yet, I was incredibly anxious.
Again, what if I’d messed up? What if one of my important points turns out to be untrue? Would the publisher give me time to go back and fix things? Would I have to delete arguments I had come to believe in? Could my premise be wrong? Could I be a fraud and not even know it? I imagine it’s a bit like finding yourself being grilled in a police interrogation room, after a long enough time, you might think to yourself, “Hey, maybe I am guilty.”
Mercifully, the results soon came in. On the plagiarism front, I was cleared—but only after some review. WriteCheck had given the manuscript a similarity index of 3 percent and with a total of 122 matches. Seeing the swaths of yellow across the document was not a sight I expected. As I went through the flagged passages one by one, it became pretty obvious what had happened. In several instances WriteCheck flagged quotations not technically inside quotation marks (block quotes, poetry, acknowledged paraphrases, and allusions). On page 15 a verse from Shakespeare was highlighted, though obviously I was not claiming to have written “To thine own self be true.” In other cases, WriteCheck picked up instances of so-called “self-plagiarization.” On page 33, it highlighted a sentence from an article I had written months previously about the dangers of passion, while I was testing out material and exploring ideas I would adapt in the book. Obviously the software had no idea that I was the creator of the sentence I was supposedly stealing—and if it had been, I would still argue that self-plagiarism is not a sin. It’s my writing, I can do with it what I want.
I don’t believe I made any significant changes to the manuscript based on the data from WriteCheck, though I certainly would have had it found anything. My interactions with the fact checker were far more fruitful.
Chris was able to help me get a quote by Bernard Baruch right, which I had either written incorrectly or had found mistaken at the source. In the book, I had also attributed the expression “oceanic feeling” to the French philosopher Pierre Hadot, when in fact, Hadot references the term throughout his writing, but is not the one who coined it (Romain Rolland deserves the credit). In another instance, I said that Glacier Bay was named by the naturalist John Muir when in fact there is a glacier in Glacier Bay that is named after Muir. I had mentioned a letter that Ben Franklin wrote urging restraint to a particularly frustrating and obnoxious colleague—Chris found that Franklin ultimately declined to send this letter, meaning I had to fix my error but I feel like the new language actually strengthened my point. In one of the book’s most critical sections, Chris took issue with a quote from Howard Hughes. Not aware of my source, Chris had discovered Hughes saying nearly an identical line: “If you had ever swapped places in life with me, I would be willing to bet that you would have demanded to swap back before the passage of the first week,” to the one I had quoted but in a sarcastic context. Research revealed that he must have made this remark more than one time—so the line could stay.
As Ta-Nehisi Coates wrote a few years ago, “Good fact-checkers have a preternatural inclination toward pedantry, and sometimes will address you in a prosecutorial tone. That is their job and the adversarial tone is even more important than the actual facts they correct.” I wouldn’t say that Chris was ever prosecutorial, but I certainly benefited from every inquiry. Even in areas where I disagreed with Chris’s questioning or felt he was splitting hairs, I was motivated to address his concerns—even if only to hedge or adjust my language to eliminate even the possibility of having to later defend a given remark or assertion. In other cases, being forced to go back to confirm against the source material allowed me a second chance to review and reconsider my position on the people I was writing about. In a book about ego, which requires harshly judging many historical figures, this second look was essential in developing the fair and considerate tone I hoped to achieve. The process made the entire book better—not just more technically accurate.
After making the changes, I was able to proceed back along the publishing track. In traditional publishing, the book is then sent for copy editing, which, depending on the editor you’re assigned, can be either a miserable experience or quite helpful. I was surprised to find myself much less defensive with the fact checker’s questioning of critical assertions in the book—whether Howard Hughes actually said this or that, or whether my characterization of this historical figure was fair or not—than I was having a copy editor (who one never actually meets) question stylistic choices I had made. Or worse, finding that this copy editor had taken the liberty to revert these flourishes or delete important words and simply mark the changes with “OK?” No, ma’am, it’s not OK.
A book then moves on to the legal review, which was relatively easy. I had referred to a certain hedge fund as “notorious” in the book—the lawyer asked if I would consider something a bit less judgmental. That was an easy change. We discussed the rights issues on two single lines of lyrics (which are notoriously difficult to get permissions for) and then we were done. One of the benefits about writing about mostly dead people is that they are unlikely to sue.
The book was then laid out and designed and printed in galley form for media purposes. It’s at this point that changes are most difficult. In a somewhat unique circumstance, at this stage the audiobook rights to my book were acquired by a publishing company owned by the author Tim Ferriss, who was gracious enough to give a round of extensive notes (mostly cuts). Publishers hate making significant cuts at this stage, but my editor was patient enough to allow it. We then finished a final pass and the book was wrapped.
I mention those final rounds of edits in this article for one reason—and it’s a point I think most writers will appreciate. After what amounted to nearly a dozen different passes—by me, the book agent, the research assistant, the editor, the fact checker, the copy editor, the lawyer, the managing editor and my own passes between these two as well as all the early readers—all aimed at the complete elimination of errors and mistakes, one begins to feel pretty confident that they’ve got everything right. Jonah Lehrer told Jon Ronson in So You’ve Been Publicly Shamed that he wished he “had the temerity, the courage, to do a fact check on [his] last book.” Well, I had done that. And I patted myself on the back for it.
Yet even with all these eyes, not just diligently looking and parsing the words of the book, but some specifically hired to find errors, we missed something. My sense of elation came crashing down when the designer responsible for the book trailer sent me an email. He had no idea what the book had gone through, that I’d been preparing this article to discuss fact checking in publishing. He just wanted to alert me to something he’d noticed.
“You probably already know,” he said. “On Page 200 there was a minor error about Tom Brady—he went to the University of Michigan, and the copy lists Michigan State as his alma mater.”
Figures. Although if that’s our biggest miss, I’ll consider myself very lucky.
Ryan Holiday is the bestselling author of Ego is the Enemy and three other books. His monthly reading recommendations which go out to 50,000+ subscribers are found here.
