Pretty damning article published this week in the Journal of Cell Biology showing that Thomson Scientific (formerly ISI) can’t produce the data that they use to determine journal Impact Factors. The Rockefeller University Press, which publishes JCB bought their own data back from Thomson as part of an analysis of their properties. Much to their surprise, the data didn’t add up to the impact factor that Thomson has declared for their journals. This led to the revelation that Thomson keeps two sets of books, but neither of these sets of figures matched up with the Impact Factors. As the article notes, “If an author is unable to produce original data to verify a figure in one of our papers, we revoke the acceptance of the paper.”

While Impact Factors play a big role for those of us in the publishing game, they’re probably even more important to the working scientist, as job offers, promotions, tenure and grant funding are all often determined by one’s record of publishing in journals with high Impact Factors. This harkens back to a PLOS Medicine editorial from 2006, where they noted that in their opinion, determining a journal’s Impact Factor is “unscientific and arbitrary.”

As both articles note, perhaps it’s time to re-examine such measurements, and start using more scientific and meaningful tools to determine the value of a given publication.