Friday, July 27, 2007

HBS Cases: How Wikipedia Works (or Doesn't)



via HBS Working Knowledge on Jul 25, 2007

Published:July 23, 2007
Author:Sean Silverthorne

HBS professor Andy McAfee had his doubts about Wikipedia, the online encyclopedia created and maintained by volunteers. "I just didn't think it could yield a good outcome or a good encyclopedia. But I started consulting it and reading the entries, and I said, 'This is amazing.' "

So when the concept of "Enterprise 2.0"--a term coined by McAfee on the general idea of how Web 2.0 technologies can be used in business--popped up on Wikipedia, McAfee beamed. "I was bizarrely proud when my work rose to the level of inclusion in Wikipedia." Then, however, a turn of fortune took place. A "Wikipedian" nominated the article for deletion as unworthy of the encyclopedia's standards. McAfee thought, "It's not even good enough to get on Wikipedia?"

He left the sidelines to join the online discussion about whether the article should be kept or jettisoned. It was also that moment that would eventually lead to an HBS case study, written with professor Karim R. Lakhani, on how Wikipedia governs itself and faces controversial challenges.

The case offers students a chance to understand issues such as how online cultures are made and maintained, the power of self-policing organizations, the question of whether the service is drifting from its core principles, and whether a Wikipedia-like concept can work in a business setting. (See related story below.)

The wisdom of crowds

Even by online phenomenon standards, Wikipedia is huge. Begun in 1999 by Jimmy Wales under the name Nupedia, the service today claims 1.8 million articles in English, 4.8 million registered users, and 1,200 volunteers who regularly edit Wikipedia articles.

Anyone can submit or edit an article, which is why Wikipedia has been lampooned for high-profile inaccuracies, such as a biography of journalist John Seigenthaler Sr., who, according to the anonymous contributor, "was thought to have been directly involved in the Kennedy assassinations of both John and his brother Bobby." Not so. A recent cartoon parodied, "Wikipedia: Celebrating 300 Years of American Independence!"

But Wikipedia also employs a series of consensus driven vetting processes that strive to ensure the information is accurate, is verifiable, is built on solid sources, and excludes personal opinion. Just as anyone can submit an article, anyone can also start an "Article for Deletion" (AfD) review process if they believe the piece does not live up to those standards. After online debate about the worthiness of the piece, a Wikipedia administrator reviews the arguments and decides the fate of the article.

The result has been a product that even academics regularly consult. In late 2005, the scientific journal Nature conducted a study comparing 42 science articles in Wikipedia with the online version of Encyclopaedia Britannica. The survey revealed that Encyclopaedia Britannica had 123 errors while Wikipedia had 162 (for averages of 2.9 and 3.9 errors per article, respectively.) For the editors at Britannica, that may be a little too close for comfort.

It's the kind of success that attracted McAfee, whose research centers on the use of technology in business, and Lakhani, an expert on distributed innovation.

"We had these completely overlapping interests, and we were kicking around the idea of how we were going to write a case on Wikipedia, what research could we do: What's the right way in on this phenomenon?" McAfee recalls. "And we just got very lucky with timing, in that this article appeared about my Enterprise 2.0 concept."

Into the thicket

In May 2006, someone unknown to McAfee, but who had read his seminal article "Enterprise 2.0: The Dawn of Emergent Collaboration" in the MIT Sloan Management Review, posted a 34-word Wikipedia "stub"--essentially a brief starting point for others to build on the concept. McAfee's article detailed how so-called Web 2.0 technologies such as blogs, wikis, and group messaging, employed in a business setting, could encourage more spontaneous, knowledge-based collaboration.

Shortly after the posting, however, Wikipedia user "Artw" recommended the article for deletion, characterizing the entry as "Neologism of dubious utility." An administrator eventually deleted the work, but Enterprise 2.0 was resurrected as a lengthier piece. An AfD was again tagged on the article. The debate was on.

"So we got to watch the governance process up close and personal on a topic that I cared a lot about," recalls McAfee. "I participated in the Article for Deletion process, and got to understand how Wikipedia works as a Wikipedian. At the end of all that we said, 'Well, regardless of what else we do at Wikipedia, we've got a really, really good teaching case right here.' "

Why Wikipedia works

From the outside, Wikipedia may look like chaos barely contained. "When people look at these sorts of phenomenon at Wikipedia, they misread the anarchy," Lakhani says. "All these people, thousands of people, there must be no rules! But there is a very ornate and well-defined structure of participation. One of our big learnings was to actually dive into the structure: What is the structure that enables these guys to produce this great resource?"

One element instilled by founder Wales is an ethic of self-governance and treating others with respect. In many online communities, personal insults fly freely, often fueled by youth and anonymity. Wikipedians, however, do not cotton to personal attacks. "The elbows are sharp on Wikipedia. It's not cuddly. But at the same time, I'm not entitled to call someone a bleep," says McAfee.

Another reason the governance structure works, adds Lakhani, is that it is transparent--everyone's edits can be read and commented upon by anyone else.

But the real basis of Wikipedia governance is a collection of policies and guidelines developed over the years that defines everything from article evaluation standards to the etiquette surrounding debate.

"When I got involved in this Article-for-Deletion process, they kept citing chapter and verse the policies and guidelines to me," McAfee says. "It really showed me how much Wikipedians rely on these--they really are the foundations that Wikipedia uses.

"So you've got a very clear set of criteria for telling your fellow Wikipedians, 'Here's my contribution, here's why it's valid and needs to be included,' " McAfee continues. "Now, you can argue about the wordsmithing and the structure of the article, but as far as the core question of what goes into an article, they've got that largely nailed."

Or was it Enterprise 2.0 that was getting nailed by the rules?

The endgame

McAfee thought the Enterprise 2.0 article did, in fact, live up to those standards. So why was it being considered for deletion? As the arguments dragged on, McAfee began to feel that the debate might be about something more than just the article.

"It seemed to me that some of the people arguing against it were entrenched, and they were using Wikipedia's policies as doors, as barriers, without being willing to engage in a real debate about them. So the policies had become for them a way to keep out articles they just personally didn't like."

And although Lakhani believes part of the entrenchment was because a Harvard professor was in the middle of the fray--"I think what happened was that people took even firmer stances"--Lakhani agrees that rules seemed to be used in an exclusionary way. "Now the question is, is what we saw just a tempest in a teapot, or does it tell us something interesting? I think it does tell us something interesting."

An ongoing tension within Wikipedia is characterized as the inclusionists versus the exclusionists. The inclusionists argue that one of Wikipedia's core values is that it should be open to all ideas, that truth emerges from a variety of directions. Better to include than exclude. The exclusionists see Wikipedia's utilitarianism diminished if too much froth clouds the valuable information inside.

"There is always a tendency in communities or in any social organization to have this boundary and say in or out," Lakhani says. "This might be happening in isolated places inside Wikipedia. The tension that they need to deal with is how to keep it as porous as possible."

Porous is good, says Lakhani, because most content on Wikipedia appears to originate at the fringes of the community from anonymous or infrequent contributors. (A central core of about 1,200 volunteers refines the pieces over time and generally tends the Wikipedia garden.) If exclusionists began to make it more difficult for outside contributions to populate Wikipedia, the product's secret sauce could be spoiled.

"That kind of ossification, if that happened, could be really dangerous," says McAfee. "But my feeling is this existential debate about the inclusionist versus the deletionist is not going to cripple Wikipedia. What's lost there, though, is that some people who have a lot of energy to bring--and I'm one of them--get turned off by these deletionists trying to slam doors in our faces."

But in its 8-year life in several forms, Wikipedia has shown institutionally that it is open to evolution of the rules. "They continuously keep tweaking the rules as they encounter new situations," Lakhani says.

Win some, lose some

In the end a Wikipedia administrator, serving as judge, reviewed the 17 pages of debate about deletion and decided Enterprise 2.0 should stay in. Victory was short lived.

"After that," McAfee says, "one of the people on the other side of the debate took it upon himself to truncate the article greatly and change the title of it. And I left him a message. I wrote, "Hey, did you not see that the result was 'Keep'?" And he replied, 'Look, Wikipedia is this very freeform environment. This is what I feel like doing. If you don't like it, feel free to change it.' Which left me a little unsatisfied, I have to say."

Q&A: Wikipedia in Pinstripes

Companies interested in tapping into the shared expertise of their workers--the wisdom of crowds writ for business--are looking towards models such as Wikipedia that encourage collaboration.

Can Wikipedia work in pinstripes? Harvard Business School professor Andy McAfee has his doubts that a corporate encyclopedia would have much value. But the underlying wiki technology--basically an electronic document and repository where participants can throw out ideas, comment on the work of others, and share documents--has more promise.

McAfee and collaborator professor Karim R. Lakhani discuss their research into wikis and other collaboration tools for the enterprise.

Sean Silverthorne: Is Wikipedia a good model that transfers to a corporate environment?

Andy McAfee: No is the short answer here, simply because (a) how valuable is the corporate encyclopedia, and (b) how much enthusiasm or incentive do we have to contribute to the corporate encyclopedia? But an encyclopedia is only one of the things you can build with wiki technology.

Karim Lakhani: Wiki is another experiment in how to generate more collaboration inside companies, but I've seen mixed results. It can be as simple as "We're having an office party, please sign up on a wiki page, and tell us what you're going to bring," to "We're going to run this project, bring in all your knowledge assets together, and then we can self-organize."

What Wikipedia has shown is that self-selection is critical. Peer review is critical. So there is a challenge for firms that are used to managing employees and allocating the resources in a very top-down kind of way. Now we have a technology that enables self-selection, transparency, openness--how does a manager or management deal with the technology? Do they implement it in a way that's true to the spirit, or is it top-down? And, again, there are some very successful examples and some not so successful examples.

McAfee: There are a couple of things that explain a lot of the not so successful ones. There is the fact that this is a different technology, and you have to be, at this point, kind of a technology enthusiast or an early adopter. There's another problem, though, which is when you think about the percentage of Wikipedia users who have contributed anything to Wikipedia, it's got to be way less than 1 percent. Only a tiny, tiny fraction have done anything, but they have huge reach and huge impact. So the participation percentage is not big enough for this to spontaneously happen inside an organization. You've got to give it a push somehow. And management is my shorthand for where that push comes from. If you just say, "Employee base, here's a cool new technology, use it for your collaboration and coordination activities," you get back a big corporate blank stare.

Silverthorne: Wikis rely on the foundation of free expression. But can employees feel free to express their opinions to everyone in the company as Wikipedians do in their world? The CEO might be reading it, after all.

McAfee: You have to create an organization where you feel free to share your thoughts, and you don't care that your boss and the CEO can see it. And that's a much bigger challenge, I think. But then the benefits go up dramatically.

Silverthorne: Have you used wikis yourself?

McAfee: I can give you a couple of examples because I try to use wikis in a fair amount of my own work. I was organizing a 40-person conference of academics and needed to take care of all these administrative tasks that I really hate doing, like putting the schedule together. And I thought, "Ding, I'm going to outsource this to the people who are coming to the conference." So I put up a couple of initial wiki pages and e-mailed them to everyone. I said, "Here is the bare -bones schedule. You guys tell each other and tell all of us what you think we should do in each of these slots, and if you want to present in one of these 4 daily slots, just add your name to the list." And with very little pushback, the Web site for the conference self-assembled, and most people were quite happy with it. The amount of overhead went through the floor.

I also use them in my MBA course Managing in the Information Age. I tell my students that about half their grade will be based on wiki contributions. So I solve the incentive problem that way. And then I have to deal with all the problems of, "Well, what do you want us to do?" ("I'm not telling you.")

Lakhani: I think the other thing is that many companies are realizing that there's lots of knowledge in the outside world and are asking, "How do we enable the outside world to interact with us?" Many are thinking through wiki-like technologies that enable them to collaborate with outsiders and enable customers to give input.

Silverthorne: Will your students be using these tools and concepts when they leave HBS?

McAfee: I find it really hard to believe that all of my students are going to go out into the corporate world and never think about this category of tool. I don't buy it. When they get to their jobs, they're going to have collaboration, coordination, and knowledge -sharing challenges. Are they just going to send e-mails to each other? Darn, I hope not.

Lakhani: The new generation of students, the MySpace and Facebook generation, will be hitting the HBS campus soon; they are already here to some degree. They are so used to collaboration and sharing in a distributed fashion, for instance, going to a friend's page and leaving a note. They have these asynchronous ways of coordinating and collaborating.

McAfee: The distinction I draw is between channel technologies like e-mail and platform technologies that are universally visible and transparent and open to everybody. I think the communication bias of young people today has migrated from channel to platform.

Lakhani: They look at e-mail as being antiquated. And so I think that's eventually going to hit corporations.

Silverthorne: Are companies equipped to design these kinds of products?

McAfee: One of the things you learn is that designing a good user interface is really hard work. I know that companies like Google and Facebook have spent person-years just getting it to the point where it feels very intuitive and easy for us to use. It wasn't easy to get there. One of the things I worry about is that companies will go, "OK, we need an internal Facebook. Why don't we put a three-person coding team together, and we'll throw one of these things up there?" And it's just going to be an inferior product, and employees are going to vote with their feet.

Silverthorne: If you were to counsel companies that need more cross-collaboration and need to break down silos, what technologies would you recommend?

Lakhani: I would say technology's not the answer. It's the information and the flows of the information you've architected and the rules around flow of information that matter. If you look at open-source communities and what they're beginning to accomplish, they did that with some very rudimentary technology--e-mail lists and simple source code repositories. But the outcome has been incredible and is based on the architecture and rules of participation. If you bolt on wikis to an old set of rules, it would collapse and die.

McAfee: I'd say it a little bit differently. Wikis are about 10 years old, but there are modern wikis that are kind of corporate-ready--these are recent technologies. Tagging systems and a lot of other things are recently available technologies. But I agree that the technology toolkit is basically in place; that's a necessary condition, but it's completely insufficient alone. What I usually tell companies is, "Look, if you want to activate this Web 2.0-style energy inside your company, management is going to make all the difference. And if you manage it the old-fashioned way, or if you don't manage it and you just have the if-we-build-it-they-will-come philosophy, you're probably going to be disappointed."

You need to be actively involved--I'm going to fall back on buzzwords--in coaching to get desired behaviors and leading by example, and not shooting people when they step a little bit out of line. The organization is going to be watching what happens, and you're going to send very, very strong signals one way or another that are going to be picked up very quickly.


Things you can do from here:


No comments: