Rakesh Agrawal's blog

The New York Times public editor writes this week about an increasing problem: incorrect information from the Times that lives forever in search engines. The Times has started surfacing its archived content in a way that search engines can crawl. This presents problems for people who were treated unfairly.

The Times tells the story of Allen Kraus, a deputy commissioner for the New York City Human Resources Administration. The Times reported that he resigned under pressure because of an investigation. If you Google “Allen Kraus” today, the second link is for the Times archive with stories titled “6 Held in Welfare Fraud Scheme; Inquiry Uncovered Worker Bribes” and “A Welfare Official Denies He Resigned Because of Inquiry“.

That’s not the best impression to make on a prospective employer or client.

People are coming forward at the rate of roughly one a day to complain that they are being embarrassed, are worried about losing or not getting jobs, or may be losing customers because of the sudden prominence of old news articles that contain errors or were never followed up. … Kraus is hardly alone in claiming real or potential harm. A person arrested years ago on charges of fondling a child said the accusation was false and the charges were dropped. The Times reported the arrest but not the disposition of the case.

The Times says that if they worked to correct errors, that’s all they would be doing:

But what can they do? The choices all seem fraught with pitfalls. You can’t accept someone’s word that an old article was wrong. What if that person who was charged with abusing a child really was guilty? Re-report every story challenged by someone? Impossible, said Jonathan Landman, the deputy managing editor in charge of the newsroom’s online operation: there’d be time for nothing else.

Although Wikipedia is often slammed for having inaccurate information, at least with Wikipedia, you’ve got a more than fair chance of getting an error corrected. Bloggers are generally more than willing to fix genuine errors — and they are much more approachable than the Times to point them out in the first place.

A big part of the problem is that many newspapers consider their archives to be a permanent record of what was in print. Newspapers view themselves as the first draft of history.

Removing anything from the historical record would be, in the words of Craig Whitney, the assistant managing editor in charge of maintaining Times standards, “like airbrushing Trotsky out of the Kremlin picture.”

In newsrooms, the archives are called “the morgue.” Now that the Times is bringing those dead stories to life, a different approach is needed. In the past, finding old newspaper articles required you to search through microfilm, dusty newspapers or expensive commercial databases made it clear that you were looking at something old. With one-click Googlability, ancient articles look as fresh as something published minutes ago.

Some news outlets still show the incorrect version of a story with a footnote at the bottom showing the correction. That is absolutely wrong. If you make an error and you know it, fix it where the error was made, not someplace people might not get to. (Especially if they’re just seeing an excerpt on a search results page.) The first thing someone should come across online is the best version the newspaper can offer.

Bloggers do this all the time. When an error is pointed out, they correct the main blog entry and usually indicate that a previous version contained incorrect information. If the error was pointed out in the comments, they usually point readers at the comment.

For newspapers worried about the “permanent historical record”, that can be maintained as a separate link off the current page with a prominent disclosure that shows that the article has been superseded.

The archives of the Times presents another problem: you have to pay to see the archived story. The free headline and excerpt might say “John Smith arrested in connection with fondling neighbor’s child”. But someone would have to pay $4.95 to see the update that says “The Times incorrectly reported the name, it was Justin Smith.”