The Realities and Responsibilities of “Crowdsourcing”

March 5, 2009

The CrowdThe straight-faced definition of crowdsourcing is using the ability to communicate with thousands of people efficiently on the internet to get those people to do something for you. To me, "crowdsourcing" is also one of those annoying internet neologisms that's overinflated by "Web 2.0" marketing hype, so I qualify it with quotes. Even though I "crowdsource" information all the time, quality results require stringent editing, checking, and yes, curation. I'll be on a panel at South by Southwest Interactive in Austin, TX called "Curating the Crowd-Sourced World" on March 13 to discuss just this.

As a part of their SXSW coverage, The Austin Chronicle quoted me as saying,

For a blogger, crowd-sourcing is just outsourcing your research. Without fact-checkers, why not?

Since the reporter asked for a definition to go in his "glossary with a sense of humor," I said that with tongue lodged firmly in cheek. The truth is that crowdsourcing research--and editing and even fact-checking(!) the results--is a serious subject with lots of issues. Because I felt like being cute, I crowdsourced what "crowdsourcing" means, by asking about 9,000 people on Twitter. My followers didn't disappoint; several nailed some of the thornier aspects of the issue in their 140-character responses.

Here are a few things people said (some of which I edited--haha).

Crowdsourcing means:

  • two heads are better than one, times a million.
  • getting fans to do something for free
  • asking the people you know will tell you what you want to hear
  • torches and pitchforks
  • conducting a survey and stealing all the good answers
  • the smart Mom who asks her kids what they want for dinner B4 spending hrs cooking something they won eat!
  • a red flag meaning "LOW STANDARDS AND BUDGET ISSUES"
  • a technique originally pioneered by Tom Sawyer
  • "too lazy to do real research"
  • delegation, using your audience's combined knowledge to save time and do work that would take you longer on your own

I'm particularly stuck on the third item, the great point that crowdsourcing done in certain ways creates an echo chamber. That happened at Lifehacker all the time, especially with our weekly Hive Five feature: we'd rave about Product X that does Y. Then, weeks or months later, we'd ask, "What's your favorite product to do Y?" Unsurprisingly, readers would mention X. There's obviously a lot of danger in asking people you know to tell you what you want to hear.

I think I may be the only person not from the art world on the SXSW panel, so I'm hijacking the word "curation" to describe what I did at Lifehacker and do here and do with my open source software projects--which is to be an editor, to pick out the best stuff to include, and ignore the rest. It should be a fun discussion. If you're going to be in Austin I'll hope you'll attend.

Now, it's only appropriate to ask you to post what makes you most excited (or freaked out) about the brave new "crowdsourced" world--in the comments.

I don’t particularly like the word crowdsourcing- it honestly sounds like some boring marketing term used when budgets are tight and the boss provides it as the solution – I know! Let’s crowdsource our resources! That should bring our costs down and shoot revenues up!

In all seriousness, though, I like the idea. On many sites like Askville, Yahoo Answers, etc. it’s common practice to ask others who may be more knowledgeable than you for advice. And anyone who’s ever posted a problem on a technical forum already understands the many benefits of crowdsourcing. It’s not perfect by any means, but it has lots of genuine uses.

izzy9985 [+3]
Mar 5 09 at 2:56 pm

My online networks aren’t huge, but I’ve probably crowdsourced in the past without really thinking that that’s what I was doing. Posing a question to an online audience, gleaning what you can from their responses. Sure, I’ve done that.

Crowdsourcing seems like a great way to find popular wisdom from the majority. “A lot of people are doing this and it works for them. I’ll try that.”

But if you’re looking for innovation, that will likely be drowned out by the sound of the majority. Unless that innovator can single-handedly capture your attention and sell you their vision (all in 140 characters, perhaps) that great idea will be lost.

So, as with most things, it has its strengths and weaknesses.

Andrew Ledwith
Mar 6 09 at 8:20 am

I suppose I participate in “crowdsourcing” (I too subscribe to your thought Gina that the word belongs in quotation marks, or better yet, air quotes when saying it!) most often by utilizing ask.metafilter.com, my go-to site for having questions answered and answering others’ questions.

And really, I never think of it as “crowdsourcing” just as I don’t think that I’m being all “web 2.0″ish by using things like Firefox, Facebook, Flickr, or the like. I’m just taking advantage of what’s out there, because it works for me, not because it’s the hip thing to do or it uses AJAX.

Interesting tidbit Gina about the Hive Five features at LH. I did always wonder how much of a self-fulfilling prophecy they became.

Did you guys ever brainstorm on a different way of running them?

I wonder how interesting (more or less so) they’d be if you had attempted to limit votes to things that had never gotten a LH mention.

mrhaydel [+7]
Mar 6 09 at 9:06 am

Good post, Gina. Naturally, I have a lot to say here.

I’m going to spin them off one at a time. Do what you will with them, such as they are, half-formed and semi-coherent….

1. “Crowdsourcing” is a purposeful play on the term “outsourcing” and as such BOTH suffer from negative connotations. I think, unjustly. It’s important in your panel to not take for granted that baggage and the history of the term.

2. Crowdsourcing per se is different from fact checking and fact prediction. Relying on the Wisdom of Crowds, as it were, only works if there’s a known answer. Crowds could not have predicted the stock market performance of the past year. It was an unknown quantity. However a crowd can essentially guess the number of beans in a jar correctly. In other words, in sourcing information, a crowd can actually prove handy (as your example above) but in predicting how many beans I will put in tonight’s black bean soup…not so much.

3. You hint at this above with your High FIve example: crowds don’t necessarily provide you with unexpected results. In fact, if you are the one cultivating the crowd and you ask them to think outside the crowd, as it were, they will have difficulty doing so. Have a read up on the subject with these two posts:
http://lsvp.wordpress.com/2007/10/09/wisdom-of-crowd-or-crowdiness-of-crowds-ii/

I’ll summarize here:

A: Crowds can actually reduce diversity especially when there is a lack of historical data or preference on particular topic.
B: New to me is not new to us. In other words, to the crowd, the clear choice for email might be Gmail, but to the person asking the question “What email program should I use?” the answer is a revelation. In other words, the question and answer pair stands the chance of being incredibly monotonous.
C:Over time, recommendations ossify. That is diversity of answers decreases over time. What’s more, in the beginning, a random (and probably less than ideal) answer has more effect at an early stage. Think Butterfly Effect for fact-finding.
D: Design matters. Really, this is important. In systems where users can see other people’s responses, the answers will be different from those where the ballot is hidden.

Source:
The Crowdiness of Crowds, Part 1
The Crowdiness of Crowds, Part Deux

Kotke does a great job of summarizing THe Wisdom of Crowds here.

Sorry I won’t be at your panel. I’m going to be celebrating my birthday at home this year instead. Good luck!

Kelly Abbott [+1]
Mar 6 09 at 10:17 am

One of the Vegas outfits was looking for ways to cut expenses and increase income without cutting staff. Execs were out of ideas so they asked employees to brainstorm.

Some of the resulting ideas were remarkably simple and profitable. Outcome: for now, no new layoffs.

I wonder, is that crowdsourcing through coercion?

drmelho [+28]
Mar 7 09 at 12:24 am

I bet something similar to this was on your Twitter, Gina.

I once knew an investor who only bought raw commercial land. He bought land nobody else would build on.

He’d then put a sign on the property that said something along the lines of, “All joint venture propositions considered.”

Crowds with no money but lots of ideas came calling.

None went home with money but a few left behind a pot of gold.

drmelho [+28]
Mar 7 09 at 12:37 am

I guess that the problem with CrowdSourcing is not the editing/fact-checking itself (these are critical tasks each time an item is delegated) but – even before – the need for some form of “Thread Managing” to get the sources tight to the original answer and avoid those ever blobbing lateral threads and debates.
Maybe this is against the CrowdSourcing concept itself but I’m pretty sure this is the only way to save time and get the best contributions still keeping room for that popping fresh new idea.

Stefano_TT [+1]
Mar 7 09 at 7:52 am


Comments are closed. Thanks for reading!