Crowdsourcing journalism: The Guardian way

From the Nieman Journalism Lab by Michael Andersen:

Four crowdsourcing lessons from the Guardian’s (spectacular) expenses-scandal experiment

Okay, question time: Imagine you’re a major national newspaper whose crosstown archrival has somehow obtained two million pages of explosive documents that outed your country’s biggest political scandal of the decade. They’ve had a team of professional journalists on the job for a month, slamming out a string of blockbuster stories as they find them in their huge stack of secrets.

How do you catch up?

If you’re the Guardian of London, you wait for the associated public-records dump, shovel it all on your Web site next to a simple feedback interface and enlist more than 20,000 volunteers to help you find the needles in the haystack.

Your cost for the operation? One full week from a software developer, a few days’ help from others in his department, and £50 to rent temporary servers.

Journalism has been crowdsourced before, but it’s the scale of the Guardian’s project — 170,000 documents reviewed in the first 80 hours, thanks to a visitor participation rate of 56 percent — that’s breathtaking. We wanted the details, so I rang up the developer, Simon Willison, for his tips about deadline-driven software, the future of public records requests, and how a well-placed mugshot can make a blacked-out PDF feel like a detective story.

He actually offered SIX lessons. Here they are in a gist:

1. MAKE IT FUN. Willison lured the readers by making it feel like a game. The Guardian’s four-panel interface — “interesting,” “not interesting,” “interesting but known,” and “investigate this!” made categorization easy. And the progress bar on the project’s front page, immediately giving the community a goal to share. He added the Guardian’s mugshots of each MP to their pages in the database, which gave a personal element. “You’ve got this big smiling face looking at you while you’re digging through their expenses.”

2. MAKE IT COMPETITIVE. Willison posted lists of the top-performing volunteers. “Any time that you’re trying to get people to give you stuff, to do stuff for you, the most important thing is that people know that what they’re doing is having an effect. It’s kind of a fundamental tenet of social software. … If you’re not giving people the ‘I rock’ vibe, you’re not getting people to stick around.”

3. LAUNCH IMMEDIATELY. Before Parliament released its records Thursday, Willison’s team thought they might be able to postpone their launch to Friday if necessary. When they saw Thursday’s newsbroadcasts, they realized they’d been wrong. The country’s imagination was caught. “It became quickly clear on Thursday that it was a huge story, and if we failed to get it out on Thursday, we’d lose a lot of momentum."

4. USE A FRAMEWORK. Willison’s project was built on Django, the custom Web framework “for perfectionists with deadlines” that he and Adrian Holovaty created for the Lawrence Journal-World. Other frameworks and languages would have worked, too. “You absolutely could build this in Ruby on Rails or in PHP,” Willison said, but “as far as I’m concerned, this is absolutely Django’s sweet spot. This is absolutely what Django is designed to do. Once I had a designer and a client-side engineer working on the project, I could really just hand it over to them and I didn’t have to worry about the front-end code any more.”

5. HAVE SERVERS READY. As well as the Guardian’s first Django joint, this was its first project with EC2, the Amazon contract-hosting service beloved by startups for its low capital costs. Willison’s team knew they would get a huge burst of attention followed by a long, fading tail, so it wouldn’t make sense to prepare the Guardian’s own servers for the task. In any case, there wasn’t time. With EC2, the Guardian could order server time as needed, rapidly scaling it up for the launch date and down again afterward. Thanks to EC2, Willison guessed the Guardian’s full out-of-pocket cost for the whole project will be around £50.

6. SAVE COSTS. Willison used open-source, freely available software that anyone else who might want to imitate them could use.

MORE.

Twitter rules for journalists

Julie Posetti has posted a set of guidelines for journalists when using Twitter.

You will find some of these obvious and others contradictory and meaningless (be human, be honest but don't bitch about your workplace), but kudos for trying to frame Twitter in a context that some journalists can understand -- or choose to ignore.

Top 20 Take Away Tips for Tweeting Journos

1) Think before you tweet -- you can't delete an indiscreet tweet! (Well, you can, but it will survive in Twitter search for three months and it's likely live on as cached copy somewhere.)
2) Think carefully about what you're re-tweeting and acknowledge if it's unsubstantiated.
3) Be an active twit: tweet daily if you want your followers to stick.
4) Determine your Twitter identity.
5) Be human; be honest; be open; be active.
6) Don't lock your account if you want to use Twitter for reporting purposes -- this fosters distrust.
7) Twitter is a community, not just a one-way conversation or broadcast channel -- actively engage.
8) Check if your employer has a social media policy.
9) Be cautious when tweeting about your employer/workplace/colleagues.
10) Be a judicious follower -- don't be stingy but avoid following everyone as your list grows to avoid tweet bombardment.
11) If you quote a tweet, attribute it.
12) Expect your competitors to steal your leads if you tweet about them.
13) Don't tweet while angry or drunk.
14) Avoid racist, sexist, bigoted and otherwise offensive tweets and never abuse a follower.
15) Scrutinize crowdsourced stories closely.
16) Find people to follow. Foster followers by pilfering the lists of other twits.
17) Twitter is a 'time vampire' (via @anne_brand) -- you don't need to keep track of all tweets, so dip in and out through the day.
18) Prevent information overload by using an application such as Tweetdeck.
19) Add applications to your Internet-enabled mobile device to allow live-tweeting on the road.
20) Add value to your tweets with links, Twitpic and other applications for audio and video.

More.

How to create a social media strategy

David Griner has outlined a simple strategy to kick off a social media initiative in this presentation.

How to create a social media strategy

View more Microsoft Word documents from David Griner.

R.O.I of social media

Some interesting thoughts from a social media panel at New England Xpo for Business at Boston by Beth Perdue, the editor of the New England Business Bulletin:

1. TRANSPARENCY: The marketing mentality is shifting to one that emphasizes authenticity and transparency, creates two-way conversations, and looks to build assets in relationships including those with influential bloggers and Twitter followers.

"I used to talk about manipulation," said Bob Cargill, creative director for Nowspeed Marketing. "Now I preach the truth and nothing but the truth. That's a big shift."

2. CONTENT MORE IMPORTANT THAN EVER: Part of that shift is because, with a Web-based presence, content becomes more important than ever. Hubspot vice president of inbound marketing Mike Volpe, said companies should be thinking more like publishers than salesmen.

Each part of a business' Internet presence becomes one piece of its content, he said, rattling off a list that includes blogs, video and sites like Facebook, LinkedIn and Twitter.

"Promoting this content in social media helps attract more people to it, which in turn makes it more likely you will get links. Links are what power SEO, so more links will attract more people to your content, which can contribute more followers and friends in social media," he said in a blog posting that followed the panel discussion.

3. CUSTOMERS ARE IN CONTROL OF YOUR BRAND: One idea raised by the panel that companies may need to adjust to is that with social media, a company's well-crafted message gets picked up by customers who reshape and comment on it. In other words, companies no longer control their own message.

"Your consumer is taking that message ... they are co-creating with that message and the content that you developed," said Marc Fireman, vice president for Fleishman-Hillard's Digital Group.

Where marketing used to be done by getting your message out to millions of people and expecting a small percentage to connect back, today the message gets sent and resent, bouncing around to friends, bloggers, and others, potentially growing from an initial 500 top influencers to 5,000 then to 5 million, Fireman said.

"You need people to be the voice of your brand," he said.

4. R.O.I. IS LONGTERM: When considering what content to provide, Fireman said it should provide relevance and benefit to consumers. Content, he said, can't just be about getting your message out.

"You have to think outside the 'I have to make a sale today,' (box)," he said.

The hard truth, the group said, is that creating content is an investment in time and the ROI you receive may be long term.

But it exists, panelists said. In fact, Volpe said that for Internet marketing software start-up Hubspot, "social media is one of our top five sources of leads."

Volpe stressed that building followers today is comparable to how companies have traditionally invested in physical assets - like building a new manufacturing plant.

"These are assets — these aren't things that go away," he said.

5. DIVE IN NOW: For those still on the fence, Fireman suggests just starting.

John Kranz, author of "Writing Copy for Dummies," agreed. Just watch and read, he suggested.

"Don't feel like you have to immediately say something," he said.

Volpe suggested your company may already be the subject of online conversation and you're better off knowing that — whether the news is good or bad.

To find out, he said, go to Twitter Search and type in your company name.

"You might as well be there to hear and respond," he said, adding, "You cannot hide."