I Found Work on an Amazon Website. I Made 97 Cents an Hour.
By ANDY NEWMAN NOV. 15, 2019
Inside the weird, wild, low-wage world of Mechanical Turk.
After turking for eight hours, the author had earned $7.83. Dave Sanders for The New York Times
The computer showed a photo of what looked like a school board meeting.
My job was to rate it on a scale of 1 to 5 for 23 different qualities: “patriotic,” “elitist,” “reassuring” and so on.
I did the same for a photo of a woman wearing headphones — I gave her a 4 for “competent” and a 1 for “threatening” — and another of five smiling women flanking a smiling man in a blue windbreaker.
I submitted my answers. I checked the clock. Three minutes had passed.
I had just earned another 5 cents on a digital work marketplace run by Amazon called Mechanical Turk. At least I thought I had. Weeks later, I’m still not sure.
There are lots of ways to make a little money in this world. Amazon Mechanical Turk, which since 2005 has flourished, to varying degrees, in an obscure corner of the globe-striding behemoth’s empire, offers an uncertain, mystifying and often maddening way to make very, very little money.
On Mechanical Turk — named for a chess-playing “machine” from the 18th century that concealed a living chess master — scores of thousands of humans earn pennies or dollars doing tasks that computers cannot yet easily do.
It works like this: Employers, known as requesters, post batches of what are called Human Intelligence Tasks, or HITs, on Mechanical Turk’s website. A task could be transcribing an invoice, or taking part in a study, or labeling photographs to train an artificial intelligence program. (Occasionally a photo shows something disturbing, like a beheading.)
Freelance workers, known informally as turkers, race to grab and do the tasks, providing what Amazon’s founder, Jeff Bezos, once called “artificial artificial intelligence.”
Most tasks pay a dime or less, and there is a daily churn of tasks that pay only a penny.
And the weird thing is that workers — mostly American workers — will do them, for many different reasons.
People turk to save for a motorcycle. They turk to buy insulin. They turk to pay off debt or pass the time profitably while on the clock at a boring job.
Some do it because there are few decent-paying jobs that can be done at will. People who are confined to their homes by disability or social anxiety or who live where there are few jobs do it because, despite lousy wages, it seems like the best option.
Plenty turk full time. In a 2016 Pew Research Center survey of nearly 3,000 American turkers, a quarter said they made most or all of their earned income on the platform. More than half the turkers surveyed said they earned under $5 an hour.
Try Your Hand at Turking
Throughout this story we’ll give you the chance to do HITs (Human Intelligence Tasks) that The Times has devised based on ones that novice workers like me encounter on Mechanical Turk. Click Accept & Work on any of the tasks below to get started.
At the end of the story you can see how much you would have made and how long it took you. Please note: You will not actually be paid.
As little as turking appears to pay on paper, in practice it often pays less because MTurk, as it is known, is a sloppy, shoddy free-for-all.
Turkers spend their time fighting requesters over an unfair 10-cent rejection or a missing 60-cent payment. They waste minutes filling out bubbles on defective questionnaires that cannot be submitted. They abandon “10-minute” surveys after half an hour.
They swap horror stories and warnings on turker message boards (“rejection on a $0.50 hit,” read one recent bulletin, “reason is ‘funds were not allocated’”). They leave scathing reviews on the turker-run site Turkopticon (“unfair and wild use of the rejection button”).
Just how much turkers make is the subject of considerable scholarly debate, but one paper published last year analyzed millions of tasks done by thousands of turkers. Though they probably overrepresented novice turkers like me who do the lowest-paying tasks, the paper’s authors concluded that if you count time spent looking for tasks and working on tasks that came to nothing, the median turker’s hourly wage was $1.77.
Only 4 percent of turkers, the researchers found, made more than the federal minimum wage, $7.25 an hour.
Presiding over this production is the world’s biggest tech company, feet firmly planted on the sidelines. Amazon usually declines to get involved when turkers say requesters rip them off, even as it lets requesters hide behind aliases that can make them impossible to track down.
It has ignored turkers’ pleas to mandate higher wages, even as it takes a cut of each transaction ranging from 17 to 50 percent; a requester posting a 1-cent HIT pays one penny to the turker and another to Amazon.
Amazon even finds ways to recoup some of the pennies turkers earn, a reminder of the days when miners were paid in scrip redeemable only at the company store. While American turkers can get their wages direct-deposited, thousands of turkers overseas have only one way to get paid without incurring third-party fees: on an Amazon gift card.
While Amazon pays all of its American employees at least $15 an hour and favors raising the minimum wage, the company declined numerous requests to comment about the pay policy for turkers or anything else regarding Mechanical Turk. Minimum-wage laws generally do not apply to piecework jobs like turking.
Mechanical Turk is now one of a handful of big players in the field known as crowdwork or microwork. (One crowdwork company, Prolific, used by academic researchers, enforces a minimum wage: $6.50 an hour.)
Crowdwork’s proponents see a gleaming future — a borderless, no-overhead labor market where task-creator and task-doer meet at the intersection of supply and demand. Its critics see a throwback to something more dikkensian, where the lack of regulation and accountability keeps workers in the dark and on the defensive.
Mechanical Turk, in particular, combines the inconsistency and precariousness of gig work with Big Tech’s tendency to dodge liability for the icky things that happen on its platforms.
“This is a great little microcosm of what happens when you take away any rules and the wages fall to the bottom,” said Kristy Milland, a former turker turned labor activist who was one of the authors of the paper that analyzed turkers’ earnings.
The bottom was where I dwelled during my brief tenure as a turker.
I tagged blurry, surveillance-looking photos of construction workers on job sites according to whether they were wearing hard hats or harnesses (1 cent per photo).
I helped train a virtual paralegal by describing a hypothetical injury claim (“I suffered a serious injury because a defect in my lawn mower caused it to start”) but earned nothing because the “submit” box never popped up.
I imagined myself as an “active investor” and rated the prospect of my supporting a Kickstarter for a device that makes nut milk “extremely likely.”
Over the course of several weeks in September, I completed 221 HITs in a little over eight hours of dedicated turking, and earned a grand total of $7.83. That works out to 97 cents an hour.
But hourly wages are not everything. Jane Lamont, a 30-year-old call center worker in Louisville, quit her $7.25-an-hour second job at McDonald’s to turk.
She works from 7 to 11 on weeknights, five or six hours a day on weekends, and typically makes $5 a day.
While the pay is “really low,” she said, she prefers turking to fast food for its freedom — freedom from having to wear a uniform, freedom to spend time at home with her mother, freedom to watch videos between tasks.
Besides, she said, she would probably be online anyway — “I like being paid while just being leisurely.” On weekends, she brings her laptop to her boyfriend’s house. While he’s wasting time gaming, she’s making money.
“I love it,” Ms. Lamont said. “I feel like that money I can spend on extra things that I want,” like two trips to New York City last year.
Katie Boehm of Pittsburgh turned to turking in 2017 after her husband, who has diabetes, lost his job and insurance coverage. Her own health issues keep her from working outside the house, and turking seemed like a lifeline.
She turks at least 50 hours a week, sets herself a minimum goal of $20 a day and usually makes $30 to $50.
Her husband’s insulin costs $1,500 a month. “MTurk covers about half of what he needs to survive,” Ms. Boehm, 40, said. “Silly insulin.”
Amazon claims a turking work force of half a million, but independent researchers say the number of active turkers is smaller. An N.Y.U. data scientist who studies Mechanical Turk, Panos Ipeirotis, estimates that there are from 100,000 to 200,000 turkers, and that at any moment several thousand are doing tasks. The vast majority of turkers are believed to be in the United States — at least three-quarters, researchers say — with India a distant second.
Mechanical Turk was created to solve an in-house problem. In 2001, looking for help weeding out duplicate product listings, Amazon applied for a patent for “a hybrid machine/human computing arrangement which advantageously involves humans to assist a computer to solve particular tasks.”
The name was a homage to a contraption built by a Hungarian nobleman, featuring a bearded mannequin in a turban, that dazzled Europe with its chess-playing expertise. The moves were actually executed by a magnet-wielding human hidden beneath the board.
Mechanical Turk opened to the public in 2005 with considerable fanfare. “Market forces will define how effective it is for requesters and how lucrative it is for workers,” an Amazon executive, Peter Cohen, said at the time.
By ANDY NEWMAN NOV. 15, 2019
Inside the weird, wild, low-wage world of Mechanical Turk.
After turking for eight hours, the author had earned $7.83. Dave Sanders for The New York Times
The computer showed a photo of what looked like a school board meeting.
My job was to rate it on a scale of 1 to 5 for 23 different qualities: “patriotic,” “elitist,” “reassuring” and so on.
I did the same for a photo of a woman wearing headphones — I gave her a 4 for “competent” and a 1 for “threatening” — and another of five smiling women flanking a smiling man in a blue windbreaker.
I submitted my answers. I checked the clock. Three minutes had passed.
I had just earned another 5 cents on a digital work marketplace run by Amazon called Mechanical Turk. At least I thought I had. Weeks later, I’m still not sure.
There are lots of ways to make a little money in this world. Amazon Mechanical Turk, which since 2005 has flourished, to varying degrees, in an obscure corner of the globe-striding behemoth’s empire, offers an uncertain, mystifying and often maddening way to make very, very little money.
On Mechanical Turk — named for a chess-playing “machine” from the 18th century that concealed a living chess master — scores of thousands of humans earn pennies or dollars doing tasks that computers cannot yet easily do.
It works like this: Employers, known as requesters, post batches of what are called Human Intelligence Tasks, or HITs, on Mechanical Turk’s website. A task could be transcribing an invoice, or taking part in a study, or labeling photographs to train an artificial intelligence program. (Occasionally a photo shows something disturbing, like a beheading.)
Freelance workers, known informally as turkers, race to grab and do the tasks, providing what Amazon’s founder, Jeff Bezos, once called “artificial artificial intelligence.”
Most tasks pay a dime or less, and there is a daily churn of tasks that pay only a penny.
And the weird thing is that workers — mostly American workers — will do them, for many different reasons.
People turk to save for a motorcycle. They turk to buy insulin. They turk to pay off debt or pass the time profitably while on the clock at a boring job.
Some do it because there are few decent-paying jobs that can be done at will. People who are confined to their homes by disability or social anxiety or who live where there are few jobs do it because, despite lousy wages, it seems like the best option.
Plenty turk full time. In a 2016 Pew Research Center survey of nearly 3,000 American turkers, a quarter said they made most or all of their earned income on the platform. More than half the turkers surveyed said they earned under $5 an hour.
Try Your Hand at Turking
Throughout this story we’ll give you the chance to do HITs (Human Intelligence Tasks) that The Times has devised based on ones that novice workers like me encounter on Mechanical Turk. Click Accept & Work on any of the tasks below to get started.
At the end of the story you can see how much you would have made and how long it took you. Please note: You will not actually be paid.
- Requester
Western Manhattan School of Business - Title
Review a Crowdfunding Project - Reward
$0.01 - Action
As little as turking appears to pay on paper, in practice it often pays less because MTurk, as it is known, is a sloppy, shoddy free-for-all.
Turkers spend their time fighting requesters over an unfair 10-cent rejection or a missing 60-cent payment. They waste minutes filling out bubbles on defective questionnaires that cannot be submitted. They abandon “10-minute” surveys after half an hour.
They swap horror stories and warnings on turker message boards (“rejection on a $0.50 hit,” read one recent bulletin, “reason is ‘funds were not allocated’”). They leave scathing reviews on the turker-run site Turkopticon (“unfair and wild use of the rejection button”).
Just how much turkers make is the subject of considerable scholarly debate, but one paper published last year analyzed millions of tasks done by thousands of turkers. Though they probably overrepresented novice turkers like me who do the lowest-paying tasks, the paper’s authors concluded that if you count time spent looking for tasks and working on tasks that came to nothing, the median turker’s hourly wage was $1.77.
Only 4 percent of turkers, the researchers found, made more than the federal minimum wage, $7.25 an hour.
Presiding over this production is the world’s biggest tech company, feet firmly planted on the sidelines. Amazon usually declines to get involved when turkers say requesters rip them off, even as it lets requesters hide behind aliases that can make them impossible to track down.
It has ignored turkers’ pleas to mandate higher wages, even as it takes a cut of each transaction ranging from 17 to 50 percent; a requester posting a 1-cent HIT pays one penny to the turker and another to Amazon.
Amazon even finds ways to recoup some of the pennies turkers earn, a reminder of the days when miners were paid in scrip redeemable only at the company store. While American turkers can get their wages direct-deposited, thousands of turkers overseas have only one way to get paid without incurring third-party fees: on an Amazon gift card.
While Amazon pays all of its American employees at least $15 an hour and favors raising the minimum wage, the company declined numerous requests to comment about the pay policy for turkers or anything else regarding Mechanical Turk. Minimum-wage laws generally do not apply to piecework jobs like turking.
Mechanical Turk is now one of a handful of big players in the field known as crowdwork or microwork. (One crowdwork company, Prolific, used by academic researchers, enforces a minimum wage: $6.50 an hour.)
Crowdwork’s proponents see a gleaming future — a borderless, no-overhead labor market where task-creator and task-doer meet at the intersection of supply and demand. Its critics see a throwback to something more dikkensian, where the lack of regulation and accountability keeps workers in the dark and on the defensive.
Mechanical Turk, in particular, combines the inconsistency and precariousness of gig work with Big Tech’s tendency to dodge liability for the icky things that happen on its platforms.
“This is a great little microcosm of what happens when you take away any rules and the wages fall to the bottom,” said Kristy Milland, a former turker turned labor activist who was one of the authors of the paper that analyzed turkers’ earnings.
- Requester
Trevor - Title
Template Tagging - Reward
$0.03 - Action
The bottom was where I dwelled during my brief tenure as a turker.
I tagged blurry, surveillance-looking photos of construction workers on job sites according to whether they were wearing hard hats or harnesses (1 cent per photo).
I helped train a virtual paralegal by describing a hypothetical injury claim (“I suffered a serious injury because a defect in my lawn mower caused it to start”) but earned nothing because the “submit” box never popped up.
I imagined myself as an “active investor” and rated the prospect of my supporting a Kickstarter for a device that makes nut milk “extremely likely.”
Over the course of several weeks in September, I completed 221 HITs in a little over eight hours of dedicated turking, and earned a grand total of $7.83. That works out to 97 cents an hour.
But hourly wages are not everything. Jane Lamont, a 30-year-old call center worker in Louisville, quit her $7.25-an-hour second job at McDonald’s to turk.
She works from 7 to 11 on weeknights, five or six hours a day on weekends, and typically makes $5 a day.
While the pay is “really low,” she said, she prefers turking to fast food for its freedom — freedom from having to wear a uniform, freedom to spend time at home with her mother, freedom to watch videos between tasks.
Besides, she said, she would probably be online anyway — “I like being paid while just being leisurely.” On weekends, she brings her laptop to her boyfriend’s house. While he’s wasting time gaming, she’s making money.
“I love it,” Ms. Lamont said. “I feel like that money I can spend on extra things that I want,” like two trips to New York City last year.
Katie Boehm of Pittsburgh turned to turking in 2017 after her husband, who has diabetes, lost his job and insurance coverage. Her own health issues keep her from working outside the house, and turking seemed like a lifeline.
She turks at least 50 hours a week, sets herself a minimum goal of $20 a day and usually makes $30 to $50.
Her husband’s insulin costs $1,500 a month. “MTurk covers about half of what he needs to survive,” Ms. Boehm, 40, said. “Silly insulin.”
- Requester
Data Capture, Inc. - Title
Extract purchased items from a shopping receipt - Reward
$0.04 - Action
Amazon claims a turking work force of half a million, but independent researchers say the number of active turkers is smaller. An N.Y.U. data scientist who studies Mechanical Turk, Panos Ipeirotis, estimates that there are from 100,000 to 200,000 turkers, and that at any moment several thousand are doing tasks. The vast majority of turkers are believed to be in the United States — at least three-quarters, researchers say — with India a distant second.
Mechanical Turk was created to solve an in-house problem. In 2001, looking for help weeding out duplicate product listings, Amazon applied for a patent for “a hybrid machine/human computing arrangement which advantageously involves humans to assist a computer to solve particular tasks.”
The name was a homage to a contraption built by a Hungarian nobleman, featuring a bearded mannequin in a turban, that dazzled Europe with its chess-playing expertise. The moves were actually executed by a magnet-wielding human hidden beneath the board.
Mechanical Turk opened to the public in 2005 with considerable fanfare. “Market forces will define how effective it is for requesters and how lucrative it is for workers,” an Amazon executive, Peter Cohen, said at the time.
if this was china, i would love to see the outrage. Even if its a side hustle, that's is something nikkas in jail would be making