CrowdFlower Lawsuit Could Change Crowd Labor Industry Forever

Should crowd laborers be paid as contractors or employees? According to Christopher Otey, who has launched a lawsuit against crowd labor platform CrowdFlower, the answer is the latter. The results of this suit could carve a huge gash in the face of crowd labor providers, and forever change how the industry functions.

The Rundown

I’ve reported on this case before, back when I was writing for Ziptask, and you can see the full article there. The gist is that CrowdFlower, much like Amazon Mechanical Turk, is a labor platform where workers perform “tasks” for employers and receive payment according to how many they complete. It’s typically not a substantial amount of money;  CrowdFlower’s own CEO has allegedly claimed that its workers are paid far below minimum wage, sometimes as low as $2 an hour.

This does not sit well with Otey, who submits that CrowdFlower’s workers should be classified as employees and receive all the monetary benefits that go with that status. He has launched a lawsuit against the crowd labor provider, and intends to obtain collective and class-action status for it, which would allow other people to join in if they also feel that CrowdFlower has treated them unfairly.

Deciding Factors

I’ve personally spoken to representatives from both sides of this case. On the CrowdFlower side, we have Rich Arnold, the Chief Financial Officer of the company, who claims that Otey’s contributions and relationship with the company are nowhere near the level that would qualify him as an employee. Otey had contributed less than twenty hours of work to CrowdFlower in the two years he worked for the site, and had also done work for many other sites like CrowdFlower. As such, he was not solely dependent on their compensation as his primary source of income. He had also failed to develop the worker/employee relationship that typically defines these arrangements; prior to this suit, Otey had not met and could not name a single person working for CrowdFlower, and no one from the site had ever heard of him, either.

Otey’s representative, labor law attorney Mark Potashnick, approaches the issue with a more big-picture perspective, viewing Otey as a stand-in for the millions of individuals contributing to CrowdFlower. These individuals, he states, do the bulk of CrowdFlower’s work, making the site completely reliant on them. Their contributions are essential to CrowdFlower’s business model, and as such, they are as crucial as any traditional employee would be. He also cites a number of other Ninth Circuit and Fair Labor Standards Act factors that would place Otey and those like him squarely in the “employee” category.

It seems like the primary deciding factor, however, will be the amount of control CrowdFlower has over its contributors while they complete tasks for the site. One of the main sticking points may be how each side defines “control”. Arnold reminds us that by the nature of crowd labor, much of the control is in the hands of the contractor in that they have the ability to decide where and when they work, and on what projects. Whether they’re passing the time completing tasks at Starbucks on their lunch breaks, or completing them at home in their underwear while watching Pulp Fiction, the power is ultimately in the workers’ hands.

Potashnick would rather direct our attention to the rigorous methods CrowdFlower has of vetting their contributors, grading their labor, and assigning high-level tasks. Through the sophisticated infrastructure CrowdFlower has set up, they have all sorts of powers, Potashnick claims, that are not dissimilar to those one would find in a traditional employer. They can track success rates of individual workers to provide feedback, or exclude them from certain jobs altogether if their skills prove unworthy. Again, since the two sides are approaching this topic from very different perspectives, it is difficult to tell which side has more heft to their arguments.

Further complicating matters are recent discovery rulings passed down to CrowdFlower, barring them from further investigating the work history of Otey. The court has ruled that such knowledge is irrelevant to the case, and that the only thing that matters is the relationship between CrowdFlower and Otey.

Endgame: The Fate of an Industry

No matter who wins the suit, crowd labor’s role in the workforce could be forever altered. There are multitudes of sites that operate under a similar structure as CrowdFlower. If it’s found that Otey, and by extension those like him, are employees rather than contractors, the ramifications would be far-reaching.  This ruling would force CrowdFlower to bar its contributors in the U.S. from taking on low-paying tasks. Other companies similar to CrowdFlower may find themselves forced to close up shop rather than face the dramatic restructuring this verdict would require.

On the flip side, if it’s found that Otey and his ilk are indeed contractors, the exact opposite would occur. The precedent set by this case would protect companies similar to CrowdFlower, instead of undermining the very concepts that helped them flourish in the first place.

I found myself at a difficult crossroads when considering which side of this case has more leverage. The fact that the industry will be forever altered, regardless of the verdict, put me in a position where I really had to consider what either side’s victory would represent. A win for the plaintiffs would force some crowd labor sites to restructure and others to cease existing, but would also bring them in line with modern-day U.S. labor laws and perhaps even cement those remaining as a legitimate way to earn a living.

A victory for the defendants, on the other hand, would establish that there is a place in America for this type of labor. I like to think of crowd labor as akin to a snack vending machine; the work is convenient and there when you need it, and anyone can access it, but it’s not really substantial. It’s best used to supplement a main source of income, or as filler when you don’t have one, and I think such a service is unique and useful. Were there a defendant victory, I would love to then see U.S. labor laws restructured to allow such companies to continue comfortably existing.

Either way, I’ll be keeping a close eye on this case as it continues developing. I just hope I don’t have to watch the industry that I love die a slow, red-tape-swaddled death.

Reflections on Boston: People Screwed Up, Not Crowdsourcing

Image © 2013 Next Media Animation

Ohh, these have been a maddening last few days. The U.S. sort of went to hell last week, and at the center of it all we had the Boston Marathon bombings and the subsequent panic, finger-pointing, racism, media incompetence, armchair investigation, death threats, and basic reminders that we, as a species, are not that great.

But of course, you’re here because you want to read about crowdsourcing. Throughout this story unfolding, internet denizens gathered on social media outlets, mainly Reddit, to collect information, speculate on as-yet-uncovered details, and attempt to reduce confusion. Which is fine! That’s what Reddit is for; collecting all of the Internet into one place where just the good stuff rises to the surface. The problem arises when certain people decide to take this information and, because they are obviously smarter than the FBI, CIA, and Boston PD combined, attempt to find the perpetrators of the bombings before the authorities do.

Alright, let’s do a rundown of who in this narrative is making me angry (hint: everyone):

• Redditors. Not as a whole. Reddit is everyone; that is its beauty. I’m mad at the Redditors who had the gall to think they were smarter than the tens of thousands of investigating officials. What leads a person to believe that just because they have a few blurry citizen photographs and some other Internet Detectives on their side, they are better at solving crimes than entities with sophisticated investigation techniques, access to innumerable surveillance sources, a network of collaborators, and the support of the U.S. Government? Glory, maybe. The idea that they can achieve Internet fame for cracking the case. But, that’s The Internet. Some people on it are idiots. I know that, and you know that, but do you know who apparently didn’t know that?
  
• Mass Media. The Internet will wildly speculate on anything and everything, but that doesn’t make it fact. What makes a fact is confirmation, proof, and sources. You know, things major news outlets are supposed to get before they report that some random student is probably the bomber. But, of course, as soon as Reddit came up with the name of a dark-skinned male who was possibly a little suspicious, news outlets unfortunately ran with it. With the help of Reddit itself, this poor individual’s family was harassed with countless accusations that their relative was the Boston Bomber. All false, of course. No one knew the identity of the suspects until (surprise!) their names were released by the FBI. Not Reddit, not NBC, not Twitter: the real, honest-to-God government agents who were investigating the case. Turns out they can do their jobs after all!
• Internet Journalists. Specifically the ones who are liberal with their use of the word “crowdsourcing”. After the events of this story shook out, many were quick to blame crowdsourcing for the colossal amount of incompetence that went down. I’ve got a news flash for all of them: what happened here wasn’t an example of “crowdsourcing” by any definition of the word. What many forget is that aside from the presence of a crowd, the equally important component of crowdsourcing is the controlling entity, the person or people directing the crowd. It’s what separates this story from the time that crowdsourcing actually did solve a murder mystery. What we have here is crowdsourcing with a complete lack of compartmentalization; without a leader steering them towards a common goal, the crowd governs themselves. I should hope I don’t have to tell you how well that sort of thing typically works out.
• Media Consumers. Yeah, I’m in this boat and so are all of you. We’re the reason for the 24-hour news cycle, we’re the reason that fact-checking is passé and editorialized headlines are the norm. We’re the reason the media will jump on the opportunity to place the blame on any brown kid they can find. And we’re the reason that Reddit posts saying “hey guys, maybe we shouldn’t jump to conclusions and let the authorities do their jobs” got downvoted straight to hell. We demand answers more than we demand correct answers, and our constant yearning for entertainment has turned the news into what at times feels like a constant stream of barely-relevant information. I know this is well-trodden ground at this point, and that I’m not saying anything that hasn’t been said fifty times before by people much smarter than me. I don’t care. I’m angry anyway.

I think if there’s a lesson to be learned here, it’s that the Internet has made many people forget their places. The fact that information can travel so quickly from brain to fingers to keyboard to THE WHOLE WORLD and onto a new brain makes people think that they can solve mysteries better than the pros. These are people who dedicate their lives to these things, and chances are they’re better at it than the random Internet denizen. Let them do their jobs.

People on Reddit are supposed to gather and share information; they don’t investigate crimes, finger suspects, or make Facebook raids. The media reports what is happening in the world, once they’re absolutely sure that it is indeed happening. If they see something worth reporting on Reddit, they are perfectly within their rights to do so, but they are obligated to make sure it’s true first. And the viewers of the news are supposed to watch it, not demand it. Demand for news leads to fabrication of news.

Everyone, please take a breather, recoup, and kindly go about your business.

Step by Step: The Incremental Approach to Crowdsourcing

Do you like electronic music? If so, listen to this new Avicii song, “X You”:

As you may have guessed, crowdsourcing produced this song. Avicii spent the last three months collecting musical samples from his fans, evaluating them, and presenting the best ones back to the crowd for them to pick their favorite. Bit by bit, from bassline to breakdown, his fans created one of the first crowdsourced pieces of music.

I’m calling this method the incremental approach to crowdsourcing; it involves a time-consuming but vigorous process of polling, discussion, and idea submission, with each cycle adding a tiny bit to the final product. It’s not the easiest or most resource-conservative way to crowdsource, but when the planets align and the controlling entity jumps into it with both feet, it can provide truly dazzling results.

Take Wizards of the Coast (WotC), for example, the company that makes the popular trading card game Magic: The Gathering. They recently started the fourth iteration of their “You Make The Card” campaign, which allows fans to help them create a card that will appear in a future release. We’ve also seen this approach utilized by Nissan, who used the help of their Facebook fans to build a limited-release performance racing vehicle. In both campaigns, the company built something great by letting their fans choose what went into it with a multi-step process that took several months.

How It Works

All three of these campaigns followed a shockingly uniform series of steps. The contest begins with the most general parts of the project and moves to the specific. For Avicii, the beginning was the song’s melody; WotC started with the card type, and Nissan first asked for an exhaust system that would do justice for the car’s engine. The crowd voices their opinion on which of several options should go onto the next round, and then the next piece is selected to be added on. More discussion, more voting, more submissions; rinse and repeat until you have a complete product.

It’s important to note here the distinction between incremental and iterative development. Both consist of several periods of discussion, voting, and designing; the main difference is what is produced at each step. Incremental development adds a new thing to the project each time; a new sound effect to a song, or a new ability on a card, or a new set of tires on a car. Iterative development, by contrast, would design the entire product at once, and then make it a little bit better better with each progressive pass. This is sometimes the method used to create Wikipedia pages, in which a heap of information is dumped onto a blank page and slowly pared down, edited, formatted, and given citations in the coming days.

What Makes It Great

The incremental approach isn’t for everyone; you need a ready-built fanbase that is not only numerous, but dedicated. Avicii is one of the most popular DJs in the world, Nissan is a multi-billion-dollar auto-industry leader, and Magic: The Gathering players have been dutifully flipping cardboard for the last 20 years. Because of this, these entities can afford to hold a contest that stretches out over months, where those less popular may find their crowds losing steam after the first few weeks.

Additionally, when designing a product where every piece of it needs to work well with every other piece, the incremental approach ensures that the crowd doesn’t get ahead of itself and the work is easy enough to swallow. If WotC had attacked this project using the iterative approach, they would have faced the tremendously difficult task of sorting through several thousand card submissions, some of which likely being completely unusable. By instead uniting its crowd on each consecutive step, WotC streamlines the conversation surrounding its project and focuses its audience on the task at hand, while not losing sight of the upcoming steps and the end goal.

Compare this to a project where each piece is designed individually. It’s been said (but never concretely attributed) that a camel is a horse designed by committee. This brings to mind a product that is designed simultaneously by several entities that have conflicting interests in the final outcome; some may want it to have functionality or features that the other groups aren’t interested in or actively oppose. In this manner, parallel design by many sub-groups of a crowd can create a product that while unique and novel, doesn’t really accomplish any one goal to a satisfactory degree.

It also helps that all three of the entities I am using for examples show great adhesion to the three tenets of successful crowdsourcing, as I’ve mentioned before with WotC and Nissan. Avicii also hits all the marks of incentive, barrier to entry, and compartmentalization; his fans are more than willing to put in the effort for the chance to hear something they created get blasted on the radio or through nightclub speakers, the submission process for samples was simple and available on social media channels Avicii’s fans frequented, and the incremental approach ensured that each new piece of music was carefully planned and fit with everything before it.

The Juice is Worth The Squeeze

I would love to see more companies use this approach in the future to replace the standard “contest” model of crowdsourcing that really only allows input from one creative mind. The incremental approach is truly the right choice for entities that want to tap into the collective knowledge of their entire crowd, with an added bonus of collecting market research as they go. Even if Nissan doesn’t end up using Exhaust System C, they will still know what their fans think of it after the contest is completed. WotC’s You Make The Card not only gave them the final product, but also inspired entire mechanics that went on to become very well-loved.

Let me know in the comments if you’ve seen a company use this approach to achieve great (or not-so-great) results, or if there’s someone who you think would benefit from changing their approach to this one.

Valve Cleans Up Steam Greenlight After Eventful Launch

Valve’s Steam Greenlight launched last week, allowing members of their popular gaming platform and community to vote on games they would like to see appear in the Steam Marketplace. Already, Valve is doing the “good guy” move and making positive changes to the system to reduce the number of sub-par submissions and make relevant games easier to find and vote on.

Change 1: Pay To Submit

• Problem: Too many bogus submissions. It’s all too easy for a jokester of a user to submit a game that’s an obvious knockoff, smutty, or fake; the only thing the submissions process requires is a few screenshots, a short video, and descriptor text.
• Solution: Valve implemented a feature in which to submit a game to Greenlight for the first time, a user must make a $100 donation to the charity Child’s Play as a show of good faith and seriousness. This is a one-time payment; any later games can be submitted freely with no additional charges.
• Reaction: If the Greenlight forums are to be believed, the reaction is positive. Many users only found out about the update when they logged on and saw that the “trash” submissions were mysteriously and conspicuously absent. The general agreement is that while a $100 charge is a little steep  (and may want to be reduced in the future), it’s not that much for an aspiring developer willing to take a risk. The fact that the money goes to charity, and not into Valve’s pockets, is another point in the company’s favor. Plus, the fee means Steam’s moderators can spend less time monitoring Greenlight. Fine by me; better they pay attention to games that are already released than try to corral the “someday” titles anyway.

Change #2: Collections

• Problem: Almost immediately, so many games flooded the marketplace that it became very difficult to rate them all. Search features were confusing and hit-or-miss at best, so it became difficult to find new games to rate, or locate ones that had been previously rated.
• Solution: The implementation of the “collections” feature, in which a user can make a playlist of sorts filled with games of a similar genre, theme, or other common thread. This list then appears on the Greenlight page, making it easy to find the “Indie Horror Bundle”, for instance, and vote on all the games therein.
• Reaction: Great idea. Curation is always an issue in systems like this, so once again the solution is to let the crowd do it for you. Friends will be able to look at the lists of other friends, which makes it easier to gain more support for games you’ve rated highly. Plus, it’s an extra angle to game discovery; you can either browse the homepage at your leisure, or check out the lists for a more linear experience.

Moving Forward: More Work to Be Done

This is an excellent start for Greenlight, especially considering how young the platform is. But there is still more to be done to make this the ultimate platform for gaming democracy. Small changes will be made to the fee, I’m sure; the ability to give to a charity besides Child’s Play will likely come sooner than later.

Additionally, there is a lot of confusion about the rating system. Users are hesitant to press the “disapprove” button, even though it’s been stated many times by Valve that only positive votes affect whether a game gets released. Users also want the ability to pass up on a game without removing it from their rating queue. For example, if a game looked intriguing but didn’t quite have the polish to earn the user’s vote yet. I feel that Valve would be wise to add a three-point rating system, with options for “I would buy this,” “We’ll see / Maybe later,” and “Not for me, next!”

At any rate, Valve once again pushed the gaming industry forward and shows us what a company can do when their primary focus is on the people who use their service. Hats off to them, and I hope that we see some truly incredible work come out of their platform in the next few months.

 

OpenStudy’s Mechanical MOOC Collaboration Teaches You Python, Is Predictably Awesome

Were you aware that I love OpenStudy? The crowdsourced study space was one of the first examples I found of a platform that really accomplishes something that could only be done through crowdsourcing, and I’ve been proudly singing its praises ever since.

So naturally, when yet another fantastic new update landed in my inbox, I expected greatness and was not disappointed. OpenStudy has teamed up with some very heavy hitters to create Mechanical MOOC, a massive online open course that will teach users how to program using the Python language.

The collaboration will take the strongest aspects of several platforms and combine them for the ultimate learning experience. OpenStudy, with their fantastic discussion and organization tools, will provide the collaboration aspects; MIT’s OpenCourseWare takes care of the subject material; Codecademy provides the tests and exercises; and the whole thing comes together on Peer 2 Peer University’s established online course space.

This melding of giants should set the standard for ambitious crowdsourcing efforts, especially because it’s so rare to see one company, let alone four, that is able to swallow its pride and admit that a competitor has certain better features. The fact that these companies would team up to provide a service that none of them could provide on their own? Well, that’s the sort of thing that just makes someone all warm and fuzzy inside.

Could you imagine if other companies took their cue from this? If we started seeing computers with Microsoft features, Apple user-friendliness, and Google applications? A fast food restaurant that had Big Macs, Checkers fries, and Starbucks coffee? Or heck, just one social network that combines the best parts of LinkedIn, Facebook, Twitter, WordPress, Pinterest, and Last.fm? How cool of a world would that be?

Sign up for the class (which starts on October 15) and see what can happen when companies put aside their differences for the greater good.