No there there
Thursday, September 27, 2007 at 10:22 am UTC by David Crotty permalink
Interesting post over at Nature’s Nascent blog regarding scientists and social software. While they do make a great point, that new tools need to focus more on the level of the groups in which scientists work, I still get the feeling that there’s an essential point being missed by so many of these “myspace for scientist” efforts. Just building a tool, just taking advantage of technology that’s available isn’t enough. For real large scale uptake, any such tool has to serve a need of the community, and it has to do so in a compelling manner. And on top of that, it has to be much, much better than whatever is currently filling that need.
There’s a very telling comment on the post from William Gunn, who details his efforts at getting his labmates to adapt to these new tools. His two examples were using a Wiki for keeping track of lab protocols, and using Connotea for the lab’s journal club. He blames the failure of both efforts on a lack of credibility for these methods, and suggests that if a well-respected publisher got behind them and promoted them, you’d see more acceptance in the community. I’d argue that they failed for a different reason–that what those two methods offer isn’t significantly better than what’s already in use, and in some cases requires more effort than the older tools.
For the lab notebook–which is easier, going to the lab’s shared drive, grabbing a Word document used to keep the protocols and annotating as needed, or going to the Wiki site, creating an account, signing in and editing there? Add in the worries of publicly exposing that Wiki to both annoying trolls (easily fixed, but annoying nonetheless), and more importantly, to competing labs. Factor in the need for someone to monitor and do upkeep on the Wiki. How is the Wiki any better than the Word document for an average lab member? Why should they spend the time and effort in learning something new if the payoff is not going to be significant?
And while Connotea is a very useful site, again, I’ll ask which is easier, circulating the paper to be discussed as a pdf by e-mail, or requiring everyone to create an account on Connotea, circulating a link by e-mail, following that link to Connotea, logging-in, and then following another link to get the actual paper? It seems to me you’ve gone from a one-step process to a 5 step process. This is progress?
In both cases, you’ve created a situation where the new, supposedly superior tools create more work than the primitive stone-age tools already in use. It reminds me a lot of the electronics industry over the last decade or so, an industry that continually creates new products for which there is no demand. It’s kind of a “build it because we can” philosophy, with a hope that because the product exists, a demand for it will be created. Does anyone really need a refrigerator with a computer and internet connection? Is it really that hard to throw away spoiled food on your own?
Given the lack of interest in and lack of uptake of the currently available social networking type tools, they’re looking more and more like computerized refrigerators. Most of what’s been offered is merely a repurposing of tools other groups found useful. I suspect that you won’t see mass participation from the life sciences community until completely new tools are developed that serve their specific needs. Science is a very different culture, and each individual field has its own personality. The tools that work for physicists or computer scientists won’t necessarily work for biologists. Trying to shoehorn a tool that worked in another aspect of society into the lab may not be the best approach. Go to any biology meeting and notice who sits where, notice who asks questions during talks. Notice which labs refuse to show any unpublished data for fear of getting scooped by a competitor. Think about the hierarchy that is in place, think about how status is conferred, and how one builds a career and reputation. Think about the costs of reagents used in a typical experiment, and how tight funding is in this day and age. Without a deep understanding of the social nature of a group, building social tools for them is fruitless. And while these new social tools offer incredible potential for democratizing what is currently a top-down hierarchical field, that’s a monumentally more difficult task than just trying to build tools that make scientists lives easier and more efficient.
The other relevant piece of consumer electronics to keep in mind is the iPod. There were plenty of mp3 players available before the iPod and a huge swath of the music-listening world was already intimately familiar with digital music from the heyday of Napster. Clearly there was a need already existent for portable music, and digital players offer much more than previous formats (carrying around one’s whole music collection, playlists and shuffling, small size, etc.). Despite all of this, you still didn’t see mainstream acceptance until there was a tool that was really easy to use, the iPod. So that’s the second hurdle here, no matter how useful the tool is, you’re going to have to overcome a lot of inertia to make people change their ways. If it’s not compelling and straightforward, it’s not going to work. And even if it is compelling, expect it to take a long time. Walk through a poster session and think about how long it’s taken for even partial uptake of the use of large scale printers for making posters. One could also at this point go back to William Gunn’s argument above about the role of endorsement and awareness in acceptance, and factor in Apple’s advertising campaigns and celebrity placement as key factors in the iPod’s success. It’s certainly true, but wouldn’t have made a bit of difference without first creating the compelling product that filled a need.
So until there’s a “there” there, I don’t expect to see the mainstream of science adopting these new technologies. Which is not a knock against the early adopters, and efforts like the Nature Network should be applauded. When the right tools are there, scientists will come flocking in, but until that time, I suspect we’ll see a lot of frustration and a lot of efforts thrown up against the wall without sticking.
Posted in General, Science Publishing, Social Software, Web 2.0 | 8 Comments »
RSS feed for comments on this post. | TrackBack URI
Add to:
Del.icio.us
Digg
Technorati
Blinklist
Furl
reddit
Saturday, October 6, 2007 at 7:48 pm UTC
Hi, David. I’m glad you found my comment interesting.
I disagree that there’s no compelling need for a better way to manage a reading library or protocol collection. In fact, here’s a blog post by one of my colleagues where he complains about how to organize a collection of PDFs.
Your concerns are similar to the concerns raised by my labmates. If I may, I’d like to address those concerns, because it seems to me that your view of the problems is a little shallow.
Grabbing a copy of the word document has the following problems, mostly having to do with distribution and version control.
You end up with multiple different version of the same document, with no clear and easy way to merge the changes back into one document. As a result, you’re never quite sure which version of a protocol was followed, an issue that can lead to fairly insidious problems, as our lab found out when someone wrote rpm instead of x g in a centrifugation instruction. That error got copied and pasted into at least one publication, requiring us to write an embarrassing erratum. If every set of eyes was always looking over the same document, errors would be more likely to be caught, and you’d only need to know when an experiment was done to be able to reference exactly which version of the protocol was being used at the time. Perhaps if more people had been looking over the protocols, they would have asked questions which could have led to us discovering the error. It could have even been a wiki visitor that noticed the error. Since you can give users coming from a set of IPs automatic access to the wiki, they’d only have to have an account or log-in if they wanted to work on it from home or from a laptop, during a conference, for example. Even then, they could always set the browser to remember their log-in details for them. Set up effort is a little more than banging out a Word document, but medium to long term useability is much greater with a wiki. Distribution from a wiki is as easy as it gets, with search built in. No silly shared folder which can’t be accessed from non-workgroup computer or ones that don’t have Word installed on it or ones that have networking issues.
Even if you don’t like the wiki idea, you could at least have a subversion project set up or use CVS to track changes. Yes, all those things require one-time registering for an account, but surely that’s not that big of a hurdle to get the very real and significant benefits of having a revision log of changes, one canonical version which can be accessed from anywhere, and improved error checking. The benefits on a larger scale are improved collaboration abilities that come along with easier sharing of information online, and isn’t that what the web is all about, easier sharing of information?
The next crop of graduate students are definitely going to be checking out the websites of prospective labs, and having a useful and well thought out website can only help recruiting.
Now to talk about Connotea vs. Endnote. There’s really no other comprehensive option short of manually restyling and renumbering your manuscripts every time you change or reorder something, is there? If you think about Connotea as a part of your whole keeping-up-with-the-literature process, it’s far less work.
Say you’re writing a manuscript and you’re reading Pubmed for papers you want to cite. Let’s compare the whole workflow for Endnote vs. Connotea:
Endnote
One time efforts
1: Pay for Endnote
2: Install Endnote
3: Install Endnote Word plugin(which may conflict with macro settings and will prompt you for missing files, and which is incompatible with different versions of Word, all of which assumes you’re using Windows on a PC to begin with.)
4: Set up folder structure, either on PC or in a drawer. You have to do this hierarchically if you want to avoid ending up with a big blob of papers.
Reading
1: Search for terms in Endnote using the distinctively crufty UI.
2: Read abstracts in Endnote.
3: If abstract looks relevant, click through to pubmed.
4: Click through to publisher’s site
5: Click PDF link
6: Navigate to proper folder or create it if it doesn’t exist.
7: Save PDF
8: Add link to PDF in Endnote
Repeat 1-8 as necessary.
Potential points of failure: Someone tweaks the search interface at Pubmed(has happened twice in 5 years) so that Endnote can no longer search.
Setting up the folder hierarchy improperly, so PDFs are either miscategorized or end up in a big useless blob.
Anything happening to your computer will break the whole system. Even upgrading the OS, which breaks your Endnote install or dereferences your PDF links, will break the system
Writing
1: Open Word plug-in
2: Search for a citation you want to insert.
2: Insert citation
Points of failure:
Upgrade of Word which breaks plug-in
Change in security settings which disables Word plug-in
Anything happenning to the compuer you’ve got the programs installed on. You can’t even edit the paper on a different computer without messing up the citations.
You can’t find the paper you want to cite(mentioned because the search function is terrible in practice unless you remember an author’s name.)
Having to use a different computer than the one you’ve got Word and Endnote installed on.
Connotea
One time efforts
1: Go to the site
2: Register for an account
3: Add the browser button to your toolbar.
4: Install the Zotero plug-in.
Reading
1: Go to pubmed or hubmed or any publisher’s site.
2: Click bookmarklet or browser button
3: add a couple descriptive tags
Point of failure: This works anywhere you’ve got an internet connection, on any computing platform. Even if you’re using a public computer on which you can’t modify the browser, you can still bookmark the page by entering the URL in a Connotea window.
Writing
1: Open Zotero
2: Search for article Since you tagged the article, and zotero can import connotea tags, it’s easy to find things according to labels that you give.
3: Insert article
Point of failure: There’s no citation style in Zotero for the journal you want. This makes writing a citation style an item in one-time efforts.
If you’re using a different computer, you can re-download your citations from Connotea, and Zotero is free so you can download it all you want.
The benefits of Connotea are that it can actually reduce the amount of effort it takes to find the most interesting and relevant research. Because the site automatically shows you which other users are bookmarking articles similar to any given subset of your bookmarked articles, you can benefit from the reading work that they’ve done. This gives your library a discovery value that accumulates over time. Having an interesting library allows you to meet people online, opening up a communication channel which normally only exists a couple times a year at meetings. What’s not to like about that?
Now that I’ve outlined some of the very real and significant problems with the existing ways of managing PDF and protocols, and outlines some of the very real and significant benefits to using the existing tools, I’d like to know whether you still stand by your comment that there’s no “there” there.
BTW, the only way I found out about your post was by checking my server logs for referrers, which I do maybe weekly. There’s no way for someone reading my blog to know about your post. These are the problems that trackbacks are designed to solve. Ain’t technology great?
Sunday, October 7, 2007 at 7:29 am UTC
Thanks for the input, and the further thoughts on the way these programs can be used. First off, I’m in general agreement with you, I like experimenting with the new tools and obviously they’re the way of the future. As someone who does a lot of commissioning, and needs to frequently track down scientists and their contact information, it’s appalling how few are using the simplest of available tools (websites) in an effective manner.
And yes, there are obvious benefits to the technologies you mention. The question is whether those benefits are compelling enough to overcome the inertia needed to learn something new. Every scientist I know is pressed for time. There has to be an extremely good reason to devote a lot of effort into learning a new system when there’s already one in place that’s “good enough”.
Your protocol example is a great reason, but probably one that’s pretty rare. In your experience, how regularly do your lab’s protocols change? Does everyone make alterations or just one or two more “technical” members of the lab? Does everyone accept these changes and implement them? Most labs that I know are not in the business of technology development. They take the published techniques, usually established by others, and apply them to their own experimental subjects. Is a Wiki really a useful tool for most labs?
Connotea versus EndNote is a different argument than e-mailing Connotea links versus e-mailing a pdf for a journal club. Personally, I use a program called “Yep” for managing my pdf collection ( http://www.yepsoftware.com/ ) that serves a similar function, albeit not online. Regardless, either is a more personalized individual function than the sorts of social interactive tools I was discussing.
And I do stand behind my comment–if there were an overwhelmingly compelling “there” there, you’d see more adoption. Many of the tools being offered aren’t tailored for the needs of biologists, and clearly just trying to make another “myspace” doesn’t fit the social needs of the group. There’s a lot of “well, this worked elsewhere, I’m going to do the same thing but call it ‘for scientists’” going on. I’m enjoying watching Nature’s efforts to see what does work (and Nature is a company that can afford failure in a big undertaking like this, unlike most smaller less well-funded publishers, so they’ve got a freedom many of us lack). But I don’t see a lot of scientists rushing to set up their blogs there and to join in on conversations. It can’t just be a question of awareness, as the Nature Network got a huge amount of press and Nature is about as well known as they come.
As you say, this all may change over time as younger scientists enter the lab, people who have grown up with these tools.
Sunday, October 7, 2007 at 8:28 am UTC
[...] David Crotty argues that mabe the right tools have not been popularized enough, and that the available tools have not necessarily proven to be totally useful. Take my last post for example: papers and ipapers, the two reference managers, are totally useless! They crash like crazy and have more problems functioning than my 85 year old grandfather. Good in principle, but they aren’t quite there yet. [...]
Sunday, October 7, 2007 at 8:52 am UTC
Great post. . . My experience lyes in developing web infrastructure for research institutions. I’ve made some headway at Tulane University, however, with larger universities it has proven much more difficult.
In my ideal world, it is not up to the individual investigator who decides what tools are available for use in their lab, but rather, it is the institution who needs to define the standard, and get with the times. A perfect example of this is the hesitance and inconsistencies with laboratory websites. Lab sites are not fun projects for graduate students, they are a necessary tool for the research institution to convey the research that is happening to the public, the faculty and current and prospective students. This encourages collaboration, investment, and correct placement of students in labs. Everyone knows what the web is capable of doing, yet the people at the forefront of science, are ignoring it! It is unacceptable to have out any out of date information conveyed to the public.
So why is it the research institutions to seek out and implement current and useful tools, including social software? Check out my rant at oracy.wordpress (Science, Music, and Adapting to Change) for a full explanation. In brief, it is the responsibility of the research institution to provide the tools for communication and organization of all of its scientists. Sophisticated guidelines can then inspire sophisticated technology development that may then be adapted by other institutions. Obviously such a thing is only going to happen in baby steps; i don’t think institutions are going to be enforcing that their researchers all start blogging like crazy! (I’m waiting for Tulane public relations to hunt me down for using their name in this comment).
Wednesday, October 10, 2007 at 8:27 am UTC
I think you’re exactly right, David, that there is inertia in many situations. Please forgive my contentious tone here and in the previous post, but the reason I am a little put off is that it seems like you’re ready to dismiss the whole lot after only a cursory glance. You may be right that your system, developed over the years, is so good that no change would be worth it. However, if that were the case, I believe that would make you the rare example, not me.
Your main argument seems to be that it’s too soon for most of the social web applications in science. Of course, everyone has to make that decision for themselves, but I think this is a great way for a beginning scientist to get noticed and to distinguish themselves. Let’s not forget the “founder effect” that accrues to the early adopters. As you contend, it can also be a great way to waste a lot of time unproductively. The key is to get in at just the right point, when the technology is ready to take off, such that the startup effort is minimal. When the startup effort is minimal, your argument that it’s not worth the time to learn a new system becomes a minor objection, and indeed, the startup effort for use of Connotea is about as minimal as it gets.
That leaves us with the question of whether existing systems are “good enough”. Again, this all depends on how long you’ve been using your system and how comfortable you are with it. If you’ve been doing something for a long time, and it’s always worked, the only argument for change is to adapt to coming changes that will break your system. I don’t intend to argue that what has worked for you for the past decade isn’t a good system. However, why wouldn’t a new investigator, just starting out, pick a system that was as future-proof as possible?
I admit I’m a bit of an early adopter, and this probably comes from working in the relatively new field of stem cell research, where protocols aren’t really set in stone just yet, and scientists do do a fair amount of technology development. The lack of the tried-and-true in my field has led me to search around for solid methods used in related fields that I can adapt for my own use. The three most solid applications I have found, borrowed from the information technology crowd, are blogs, wikis, and social bookmarking.
Blogs are a great way to open your notebook to the world, or just to the lab. They’re a rich medium(you can’t imbed an animation of cells exchanging mitochondria in your paper notebook), accessible from anywhere, and automatically backed up. They also make you improve your writing skills. From a social perspective, they’re the root of your identity on the web: the place where you communicate your thoughts, your CV, and where you receive communication from others. On a blog, this kind of interaction can happen 24/7 with anyone in the world. In person, you only interact with people in your lab, university, or company.
Wikis, and their propellerhead cousin, CVS, are the sine qua non of modern software development. They’re the single best way to coordinate the efforts of many people working on developing the same protocol, so it shouldn’t matter whether that protocol specifies the input to output transformation of digits or DNA. For a small lab that works in an established field, it’s probably overkill, but for those on the cutting edge, you must have a system for rapidly developing, validating, incorporating, and then distributing changes to protocols and techniques. If you don’t, you’re going to end up lost in the minutiae or slowed down by repetition of mistakes. If you want to move past hierarchical control and its attendant inefficiency, a wiki is the only way forward.
Social bookmarking like that provided by Connotea is relatively new, but the analog in the IT world, del.icio.us, has been widely adopted over the past 4 years of its existence. Social bookmarking allows you to easily keep, share, and discover interesting pages or papers. Sure, you can email a journal club PDF around, filling up everyone’s mailbox, but why send 3MB * 100 emails = 300 MB when you could send 300KB that’s accessible anywhere? If you’re already using Connotea, and everyone should simply for the discovery features, you can have your own journal club group that the presenter simply adds the article to. No having to maintain a list of current email addresses for everyone on the journal club list, and the article is immediately placed in context of related work.
These three applications are the essential components of a good laboratory website. If my experience is anything to go by, the website is the first thing a prospective graduate student sees. Since all three can literally be up and running in minutes with minimal maintenance required, it’s trivially easy for anyone, even professors under tight grant deadlines, to get started. To further illustrate this point, I’ll put a up a series of posts on my blog giving a step-by-step rundown of how to get started from the very basics. By doing so, I’ll simply be returning the favor done by the bloggers who I read as I was starting things up for myself. I call my blog Synthesis, no doubt annoying some synthetic chemist, but the reason behind the name is that I see my blog as a place for the synthetic product of my reading and learning on the web. By doing this, I’m lowering the barrier to entry by taking what I spent two or three hours learning about and adapting to my needs, and explaining so that it only takes 15 minutes for the next person whose needs are close to mine. The promise of the web is that someone, somewhere, has asked the same question as you and has solved it, and this is one promise that has been kept.
In the end, each lab has to make the decision whether or not to support blogs, wikis, and social bookmarking. However, they owe it to themselves to fully understand the present and future advantages before making a decision. Not everything picked up by early adopters is a winner, as shown by the lackluster penetration of Apple’s iphone into the mainstream(Nokia FTW!), but the time is right for blogs, wikis, and social bookmarking. I’m sure of it.
Wednesday, October 10, 2007 at 8:55 am UTC
[...] the components of a good laboratory website. However, my thoughts remained incomplete until I read David Crotty’s post at the CSHL blog that I was able to form an answer, and even then it took me two long comments to get my thoughts [...]
Wednesday, October 10, 2007 at 9:38 am UTC
Thanks for the further thoughts William, I don’t think we’re disagreeing here, I think I’m just a little more pessimistic at this early stage. I’m not arguing that it’s “too soon” at least as far as anyone being able to grasp the concepts of what these tools offer. I’m arguing that it’s “too soon” because we have yet to see the “killer app” that suddenly makes sense to everyone. When you’re talking about new investigators, early adopters and those on the bleeding edge of technology development, you’re talking about a very small minority. It’s often hard to see this if one is a part of the blogosphere, surrounded by that small minority. Yes, the current tools can be adapted in some interesting ways, but they don’t seem to have caught the interest of Joe Scientist. The original argument was that it’s because Joe Scientist doesn’t know these tools exist. My argument is that the tools just aren’t that compelling yet (key word is “yet”).
You are right in that there can be benefits to being a founder. At the same time, there are also lots and lots of people who have put in lots and lots of effort into technologies (and companies) that have gone nowhere. Most scientists are already pressed for time, so investing a large effort at this point that may or not pay off is perhaps too risky. Those hours spent blogging or tagging articles could instead be spent doing experiments or actually physically networking with other scientists at meetings/talks, which still is a stronger currency than leaving comments on a blog. Founders can be rewarded, but remember, they’re always going to be a small group–if everyone jumps in at the early stage, then there’s no founder effect. Those who take the risk and do the work will, if they’ve chosen wisely, reap the rewards. You can’t ask everyone to be a founder–it doesn’t work that way.
I’ve been thinking a lot about this lately, particularly the concept of, as you put it, “opening your notebook to the world.” The question that immediately comes to mind is whether that is antithetical to the way biology is done these days. Theoretically, it’s a great idea, but given the competitive nature of research these days, how open are people really going to be with their unpublished data? While I’m a firm believer that it’s better to have collaborators than competitors, not everyone shares this philosophy. Are you willing to allow for the possibility of being scooped because you put up all your data in a publicly visible forum? Will the editors of Science care that you had the data up on your blog first when someone else has already published a paper showing the same concepts (which they got from reading your blog)? With so many scientists funding their labs/homes with their patents and universities so dependent upon those same funds, how supportive are they going to be toward a premature release of data? Labs at a biotech company obviously don’t want anyone to know what they’re doing, let alone what they’re reading, what protocols they use, etc. As a publisher, we’re often asked for private versions of our journals/book websites that can be run from within a company so no one has any access to the user stats that company generates.
Clearly this cut-throat manner is not the best climate for doing research. If you haven’t read it, Jim Watson’s “The Double Helix” gives a fascinating portrait of what it was like to be a biologist before the molecular biology revolution. Things were done in a much more “gentlemanly” manner (pun intended, at least as far as the gender emphasis goes). Watson and Crick were told they couldn’t work on the structure of DNA because a different lab at another institution was already working on that problem, and that it wouldn’t be fair to butt in. Now that’s the other extreme, you have problems that would end up not being solved because you didn’t want to step on anyone’s toes. There’s got to be a happy medium somewhere. As computational biology continues to rise, we’re starting to see the influence of the open source movement inching its way into biology. Hopefully this is something that will continue to grow and alter the competitiveness and secrecy of modern research.
The other main point I was trying to make is that to overcome the inertia we both agree exists, applications need to be easier to use. They need to be obvious, simple and attractive. It’s interesting you bring up the iPhone here, as I used the iPod as my earlier comparison. I wouldn’t call the iPhone’s penetration lackluster at all, given that it’s outselling all other smartphones on the market (at least in the USA). While the Nokia N95 is a sweet phone, it’s overkill for what most people are willing to do on their phone. The iPhone is still rough in its execution, but it has the right idea, a simple, easy to use, compelling package that doesn’t take much futzing about, doesn’t require you to go to the command line, etc. Again I think you’re seeing things from the minority viewpoint–the few thousands of folks out there who modded their iPhones and are now being very vocally negative about it online are a drop in the bucket compared to the million plus phones sold. They’re the equivalent of those currently building and using the new tools for science. Pioneers to be sure, doing interesting work, but there should be no surprise when Joe iPhoneUser refuses to break open his case and take a soldering iron to his hardware.
It’s a lesson we learned when we spent hundreds of thousands of dollars to create the now defunct molecularcloning.com, which was essentially an online version of our best-selling manual (“Molecular Cloning”, AKA “Maniatis”). It never saw the mass uptake we expected, and the reason is that when given the choice of going to the computer, going to the site, signing in, finding the protocol you want, printing it out, then going to the bench, or, grabbing the book off the shelf and going to the bench, most people are going to opt for the latter. There’s no “there” there because the tools are still too clunky. Back to Connotea–most journal clubs are a lot smaller than 100 people, usually just one lab or a few small labs coming together. The scale of tracking 30 e-mail addresses isn’t all that daunting, and bandwidth doesn’t seem to be much of a pressing issue for e-mailing out pdf files to a small group. As far as the Connotea link being accessible anywhere, you’re assuming an always on internet connection, whereas I can read the pdf I’ve been e-mailed on an airplane, or take the printout to sit poolside. Ink on paper, as old an interface as we have, and it’s still one of our most effective and most popular ones.
Keep in mind that familiarity trumps superiority every time (as shown by this study in particular http://tinyurl.com/ytz8j7 ). To get people to switch, you need to go beyond just “better”, you need over the top compelling. And that’s my point. There’s nothing compelling out there yet, nothing tailored so well to biologist’s needs that switching to it is an obvious decision.
Monday, May 26, 2008 at 3:51 am UTC
[...] the cogitation that Richard’s post awoke in my under-rested brain, I came to this thoughtful essay by David Crotty on why some of the tools that ought to be helping scientists, don’t really [...]