The person described is a "senior business analyst", reportedly making good money, but decides to work on MTurk, doing menial tasks, to help offset extra costs (diapers, etc.) of a baby that was born. This sounds insane to me. Almost anything else sounds better: reduce expenses, figure out if he can get more responsibilities and a raise, even get a low % credit to pay for those diapers for a year, etc. . Is this just me? This is an honest question.
I guess the article doesn't say it well, but this can be done in small scraps of time from wherever you currently are under circumstances where making money with a "regular job" would not be possible. If you make, say, an extra $5/day on what used to be your smoke breaks and lunch hour at work for an 8-5 job, you have an extra $180 or so a month without any additional overhead (for uniforms, whatever), scheduling conflicts from a second job, additional time taken away from family, etc.
Mechanical Turk generally sucks as a substitute for a regular job, but can make sense as supplemental income, even at nominally very low hourly pay. If you work 8 hours at $10/hour but have a 30 minute commute, you are really getting $80 for 9 hours of your time. That actually puts you below $9/hour to think of it that way.
If you do freelance work, iirc, freelancers chase their pay about 40% of the time. That also drives their real wages down. Plus, there is time involved in getting each assignment.
I have read that you can expect to do one unbillable hour for every billable hour, so you need to charge at least twice as much to make the same wage. Plus, for jobs with benefits, roughly half your compensation can be in the form of benefits.
So freelancers should charge four times as much as the hourly rate they would accept at a salaried job with benefits.
But if you already have benefits and just want to supplement your income, you can accept half as much. If you can eliminate some of the time burden of freelancing, you can halve it again.
Looked at that way, $2/hour is potentially the equivalent of $8/hour as supplemental income. And you can fit it in to otherwise useless scraps of time.
Fair enough, but there are risks there, too. Not claiming that it is necessarily the case, but when one has a fixed income job and a variable income one, effort and attention tends to skew to a variable one (you get soame $$ from the other one regardless). So your main job (promotions, etc.) can suffer.
And for a long game one will likely get much further by investing in primary job or developing new skills (teaching math to kids, home improvements, whatever) that gets them out of competing with millions of unskilled, low wage participants in MTurk.
I view MTurk as temporary fix for desperate times only. My 2c.
"Looked at that way, $2/hour is potentially the equivalent of $8/hour as supplemental income. And you can fit it in to otherwise useless scraps of time."
No. $2/hr is potentially the same as freelancing at $8/hr under your comparison.
That's completely different. Because no one freelances at $8/hr.
I don't do Mechanical Turk, but I've worked online for years and had a corporate job previously.
I made about $100/day at my corporate job and at least $20/day went towards costs involved in having the job. I don't have that kind of overhead with the work I do currently.
Freelancers don't charge $8/hour, but it's not at all hard to google up "client from hell" stories where they spent so many hours trying to get their pay that it was only like $2/hour by the time they were done or they didn't get paid at all etc.
I don't really want to argue this. Someone asked a question and I answered it based on first-hand experience making often nominally low hourly pay. And now people are nitpicking my reply, presumably because they've got relatively cushy lives and this type of assessment is alien to their experience.
If you make $100/hr at a salaried position, like a lot of programmers do, you probably don't need to think too hard about "But after my commute, etc, what's my real hourly wage?" If you make a lot less than that and want to survive, you absolutely need to think about "But what am I really making after x, y and z?" And the counterintuitive answer turns out to be that a nominally low hourly rate without a whole lot of unbillable time burden, like a commute or chasing your pay or looking for work, can be far better in real terms than a nominally higher hourly rate with a lot of hidden time burden or other costs.
If your life works just fine, good for you. But there are clearly many people willing to work for Mechanical Turk for nominally low wages. I'm sure every single one of them would love to have a higher hourly wage. I'm just trying to cast a little light on why people do this.
Everyone on HN can sneer at it all they want as stupid and not making sense etc. That doesn't change the fact that lots of people are working for nominally low pay via various online services. At best, it will just discourage me and others like me from bothering to answer questions here about it.
I've only ever freelanced as a developer but you could never charge 4x your billable rate. Good Sr. Devs make ~140k with benefits and there is no way you could charge out at 280/hr without a very sophisticated business model.
I could see how this could be different at lower hourly rates though. Clients are less likely to pay, they are less price sensitive to 16-32$/hr.
Yeah, I hear that "4x as much" figure a lot but few people seem to actually charge that. Instead, what actually happens for most people is that (for example) they are covered under their spouse's healthcare policy, so they don't need to cover the cost of benefits.
Historically, a full-time job with benefits was designed to be held by a married man who was worth that kind of compensation because he gave his all at work, went home, collapsed into a chair and said "Woman, get me a beer!" and "What's for dinner?" Then he slept like the dead.
Research shows women with children typically cannot give their all to a job like that. They get stuck in Pink Collar jobs that preserve time and energy for the "second shift" of cooking dinner, doing housework, etc. They need to be able to get up in the middle of the night if their kid is sick and so forth.
People are marrying later, having fewer kids and so on. I routinely see comments online about the downsides to that, for example how it's impossible to work full-time and cook from scratch to feed yourself properly as a single person. Cooking for one is too time consuming and labor intensive. Home cooking makes the most sense when one person is cooking for an entire family. This leaves a lot of people living on takeout and the like, which amounts to overpriced junk food in most cases.
Employers need to get enough value out of the transaction to make their business viable. They can't pay a living wage with benefits out of the goodness of their hearts. And we no longer de facto expect the wife's labor in taking care of her man to be a hidden part of the deal for ensuring that the employer gets enough value out of the deal.
I think these social changes are major driving factors in the gig economy trend. It puts the onus on the worker to figure out how to accomplish enough to pay their bills at a price that makes sense to the employer.
And part of the answer is that supplementing your income with low paid, small tasks using scraps of time that you previously could not have used to try to make money helps that equation make sense for both parties.
If you think it is fun, having a hobby (mturk) that is a small net positive can save you a lot of money compared to having a hobby which is a net loss (most hobbies).
If this was generally perceived as fun, then Amazon would charge money for you to "play" mturk. The fact mturk work pays tells me that mturk is pretty much nobody's hobby.
The "fun" may be to earn a little extra money doing menial tasks. Just the other day there was a discussion here about EVE online and how much menial management is needed in the game to set up for a small amount of fun PvP combat time. What's the difference between that and doing some mindless menial tasks for a couple extra bucks to go out on the weekend?
> The "fun" may be to earn a little extra money doing menial tasks.
That the pay is fun (or enables fun) does not make the work fun. Here mturk was named a potential "hobby", that's just ridiculous to me.
> Just the other day there was a discussion here about EVE online and how much menial management is needed in the game to set up for a small amount of fun PvP combat time. What's the difference between that and doing some mindless menial tasks for a couple extra bucks to go out on the weekend?
You could argue that EVE setup is "work" required to enable some "fun" play. I don't think anyone would consider EVE setup (just the setup) as a hobby.
Some people's "hobby" is couponing. I doubt they consider the act of digging through papers to find and cut out coupons to be fun, but the act as a whole is. But I guess if you define a hobby strictly as something you do not for pay, you're correct. If you define it more broadly as something you do during non-work time, then it fits.
I don't think you can separate the setup from the work so easily. I highly doubt if you just had the "fun" play then it'd be a successful game. There are plenty of action packed alternatives if you just want the battle part all the time.
People like grinding. Just not when they have to accept that they're grinding for the sake of grinding. There needs to be an upcoming battle or achievement or texture pack unlock or a few extra bucks in the bank in order for them to get their "I'm being productive" kick every time they complete one of their many short, simple, well defined grind tasks. Just a continuous stream of little wins.
I think I know what you're getting at. Where I work, you could almost set your watch on the almost-once-per-minute frequency of someone yelling "fuck" or "shit" because they're annoyed at something in code (I'm no exception to this), but I think it's safe to say that the 45 minutes of frustration are worth it for the 15 minutes of "I'm the coolest person ever!".
I agree. I think I have a pretty decent job, but if they stopped paying me I wouldn't show up "just for fun".
Programming on stuff I care about is fun, and for some beautiful moments at work there is overlap of "stuff I care about" and "stuff that is valuable to the business", but most of the time the work is "meh" and I mostly view it as "this is much better to me than most other jobs would be for me".
Why would they line up to work for you of all people? They can work for anyone they want, it’s somewhat of a seller’s market when it comes to programming.
I imagine just about nothing is 100% fun 100% of the time. If many parts of your work are fun, it’s fair to say your work is fun. If you’d rather be doing something else, you could try that, too. Maybe you’d like it more.
Maybe not all programmers, and maybe not all tasks...
But when I compare programming to RPGs, I feel the same way. In both of them, I spend a lot of effort supporting the really fun parts with work. I enjoy both, and I keep doing both. But RPGs have a ton of grind that I generally don't like except that it enables me to get to the fun stuff.
Most of my hobbies are like this. There are things I have to do in prep for the fun stuff, or after the fun stuff to finish the project. And I do them to get back to the fun again.
Me too, but not 30+ hours per week. And on something of my choosing. And...
Then programming usually does not come by itself: meetings, company politics, little mgmt, all usually come with it when programming professionally. Many of those tasks I consider less fun than the programming itself.
Never been more drained then after baby-induced insomnia.
A couple of low-effort, no-brain mturk forms to make extra diaper money? Great! I'll bust out the laptop while killing time at a kid's soccer practice.
We believe it's insane and dismiss it out of hand because we all realize Mechanical Turk doesn't pay a living wage in the developed world.
But I dunno, I feel OK with the premise that this guy reduced his expenses and optimized his career as much as he could.
Say you did all that and had a few hours a week you wanted to use to make your household budget a bit healthier...getting paid for some kind of remote work via an automated system that'll accommodate your inconsistent hours has an appeal!
He probably can but MTurk is superior because of liquidity and flexibility. There are very few jobs in which value is created and paid for in a span of a minute. By turning dead time into money, generating money in a way you could not for other jobs.
And a BA isn’t worth much regardless. Half the people I knew in college now work at coffee shops.
A general good rule is what when you see someone do something, there’s probably a, if not good, understandable reason for it.
> And a BA isn’t worth much regardless. Half the people I knew in college now work at coffee shops.
I agree with everything you said but just wanted to point out that in the comment you responded to, “BA” means “Business Analyst”, whereas in the sentence you wrote that I quoted it seems that you took “BA” to mean “Bachelor of Arts”.
> A general good rule is what when you see someone do something, there’s probably a, if not good, understandable reason for it.
In this case, presumably a middle-class kid who was fed a line that going to college is the way forward. They were probably decent but not outstanding students who went mostly because that was expected and got a A of Arts degree because the BS degrees are miserable.
Motivated folks can make that BA work, but your barista was probably going to end up there with or without a Bachelor's; the BA just means they're promotable, or may be able to break out of that place later.
A business analyst should be able to analyse how to get better return for his money. There’s probably not much other work that you can deliver in such small chunks, but there’s definitely work that can be performed in small chunks at a time.
That you think that you understand someone’s reasons doesn’t mean that you have to agree with or not think there’s are better ways.
As someone who regularly purchases work from MTurk, some rando's crumby bot is absolutely not what I'm looking for. We're training our own AI from MTurk responses, and we're sending to MTurk difficult edgecases that our own AI gets wrong.
Such a combination is what Unbabel does in translation. ML-driven automated translation followed by crowd-sourced human review and corrections. It's a neat idea and could be the future of translation (as opposed to full automation).
I would assume that Amazon already does this. If a task is trivial enough for an AI, they would do it themselves and get the posters money before the job gets forwarded to the Turks.
Sure, Amazon have all sorts of ml APIs. But a task platform where anyone can complete the tasks using ai, humans or a combination that f both could be substantial impressive.
Say you need to look through 20k images and determine if they contain hipsters. There may not be an off-the-shelf hipster detector. Maybe you could make one, or use humans or a combination by training an NN.. etc.
The article is more or less an advertisement for the book the "senior business analyst" has written, with a little context thrown on top to make it look legit. If you look at the authors other works, this seems to line up with their style.
That might be a little harsh, and perhaps there is no ill-intent here, but the majority of the second half of the article reads like a low-pressure sales pitch to me.
Isn't the entire point of the article that he thought it sounded like a good idea, and turned out that it wasn't? In principle, it's easier than some of the options you list, since any corporate job ("analyst") involves a lot of downtime at your computer.
Fun anecdote about some of my own dabblings in online psych experiments and MTurk.
7 or 8 years ago, my then supervisor and I experimented with running psychological (behavioural) experiments on MTurk. We had created a method that would run in browsers in native javascript and were looking to validate it. Coming from a testing-1st-year-psych-students-in-the-basement-of-the-faculty-building kind of thing, we naively took what the going wage was for that (something like 8 euros/hour), and put an experiment online that would take around 20 min to complete, and paid 2 euros.
My supervisor used his own credit card and set the spend limit to around 1000 euros; thinking that we'd never hit it. Boy how wrong were we. Apparently this 6 euro/hour wage was _much_ _much_ higher than the going rate, and we hit the spend limit in around 2 hours. Even though we had to throw out around 70% of the completions, we ended up with usable data from around 150 participants.
We went in expecting to run the experiment for a week or 2, and get maybe 50 participants, but came out with 500 in a span of 2 hours. Safe to say, we celebrated a job well done that night over one or two drinks. People were commenting on how nice of a change of pace it was compared to the then-usual MTurk tasks and that they would have done it for free. Some even left their e-mail addresses should we run another online experiment. I've since been out of the field of online behavioural experimentation, but it seems to have taken off quite a bit.
Holy smokes that is an amazing story. Do you have any more? I am always fascinated to hear how researchers design studies to prevent people from just spamming in any old answer.
I'm not sure what the state of the art is nowadays. Back in the day. The method at the time was so new, we just assumed that there would be no bots that would be able to complete the experiment successfully, or that they would stand out like a sore thumb upon analysing the data. Modern frameworks will eventually have to deal with this I'm sure. The structured way in which experiments are defined also make it easy to develop tools to automate this. Unfortunately I'm not into the field enough anymore to know how people are dealing with it.
The biggest problem we had was with people with high latency connections. The effects we were looking for were measured in 10s of milliseconds. In order to tease out these effects, we had to be very particular about the timing of when certain stimuli were presented to the participant. High latency page reloads (which were unavoidable in the system we built our method on) would mess with this high-precision presentation requirement. We measured the latency, but did not pre-emptively exclude anyone based on their latency, hence the high % of people with unusable data in our initial validation experiment.
For subsequent experiments I built a "loader" screen that would pretend to be loading the experiment. What it in fact was doing was refreshing the webpage several times (ofcourse, while progressing a progress bar) to measure the latency of the connection of the participant. High average latency + high variance latency connections were excluded. The tresholds were based on what we found in the early validation study.
Surprisingly, after throwing out the high-latency data, there were no other exclusions necessary. It seemed that for our validation study, all participants were very attentive during the experiment.
In "in-person" cases, researchers would add attention checks to their experiments with the logic that failing these attention checks by itself is no indication of "spamming", but seeing weird quirks in the data + failed attention checks would be. One that my supervisor was fond of was throwing in instruction screens, in the middle of runs of trials that required you to press a very specific button to continue. E.g., the experiment has you press 'A' and 'J' constantly, and to continue you have to press 'N'. Secretly though, A and J were also valid ways to continue with the experiment. The thought being that if you were hammering one of those buttons to quickly get through the experiment, you would also skip past the instruction screen very quickly with it.
A few years ago, a friend of mine, in her 20s, worked full-time as a technician in a university-affiliated research lab, she managed to find a steal of a studio apartment (half the price of a bottom-end one-bedroom in the neighborhood), and she bought awful cheap bulk food... but money was still so tight that, in the evening, when she was too tired to do anything else, she did these online "gigs" that paid only $1-$2 an hour.
Since she could write, she mostly did writing assignments (which I suspected were for SEO Web sites), in which they tell you a topic and how many words to write, you research and write the article, and you get paid a pittance. But $2 will buy steel-cut oats for the day.
Her time should've been worth more than $1-$2/hour, and I don't like the idea of companies arguably exploiting desperate people this way. Though it's not just companies: university researchers sometimes use Mechanical Turk workers to process data, and as research subjects.
For what it's worth, in my experience tasks from university researchers tend to be more fair. The last few studies I did, we paid an equivalent of $8-10/hour while estimating generously on time (e.g. we did $4/30min, but actual time ranged from 15-20min). People in my lab and another lab I was affiliated with seemed to pay similarly fair wages for both studies and data labeling. We also usually accept everyone's submission, even obviously junk answers. It could be that the people I've worked with are more ethical on this than other researchers, but I don't think that's true.
I don't think the IRB actually enforces a minimum wage, but the feeling of oversight seems to discourage super exploitative behavior. The downside is that high payments means higher requirements - something like 98% acceptance, 500 HITs, Master's qualification, and US location is what I generally see used, and we still get junk answers sometimes (in my last study one person somehow submitted a bunch of HTML forms with empty answers despite required fields, still not sure how they pulled that one off without intentionally doing some sketchy stuff...). This makes the fairly paid tasks inaccessible to most new Turkers, leaving them to pick up the scraps of the $1/hour HITs.
Every time I hear these stories, I don't know if they've done the math.
Let's say I live in Toronto (an expensive city to live in) and make 10$/h after taxes working at Starbucks.
40h/week, 4.5 weeks/mo = 1800$
Bedroom in apartment = 700$
Metropass = 150$
Phone = 50$
Internet = 50$
Food = 350$
Total = 1300$
That's 500$ left for things like going out, clothes, etc.
Nobody is exploiting your friends - it sounds like she complained to you, you took her words at face value and concluded that those other people are at fault.
I live in Toronto, those numbers are hilarious. You cannot live in this city on a yearly wage of 21600. Not only are your numbers wrong individually, but they're also woefully under representative because your fake individual isn't buying toiletries, isn't paying for dental, doesn't purchase clothing, doesn't have any medical consumables to purchase, only eats a meal per day or so, etc.
I've done the math. I've lived here. I've financed years of study in the city on loans and been on Ontario Works at times at the start of my career. Your model is poorly constructed.
Honestly wondering because I've fortunately not needed to budget this closely before: OP said $350 for food and "500$ left for things like going out, clothes, etc.". Does that not cover what you bring up? ~$11.60 per day for food seems enough for a healthy three meals a day for one person - a quick google shows numbers like $250/month on a "moderate budget" for food. I feel if you're eating one $11 meal every day that's probably a good place to start cutting down on the spending. Is $500 not enough for toiletries, dental, clothes, etc.? Looking at the past month of my own spending, I bought a couple rolls of toilet paper, a tube of toothpaste, body soap, no new clothes, some skincare stuff, and probably $150 worth of insurance. Maybe I forgot a ton of things but I don't think I spent close to $500 on the "extra necessities", as I would call them.
Just to be clear, I don't disagree with you that $21600 is too little to live on. But I mostly buy OP's numbers, which makes me wonder what I'm missing. Debt payments, maybe?
You are missing at a minimum: debt payments, retirement savings, regular savings (e.g. emergency fund), any sort of vacation or holiday travel budget, insurance, basic dental care or things like eyeglasses if you need them. A realistic clothes budget. Realistic meals outside the home. Any entertainment. Any travel outside the metropass. Any household stuff.
$350/food isn't bad estimate for Toronto - it would be hard to get below $250, even being very careful. So there is maybe a little to claw back there, but you'll want at least $100 for food outside the home too.
Also the rent number is too low for Toronto, assuming you don't add a commute beyond your metropass.
So yeah, $500 doesn't look that practical.
This isn't a budget for sustainable living, it's a best an emergency budget you can survive on for a bit.
It's laughably not close. I wrote a full reply with figures, but it took up a ton of space. The short version is that neither of your models are accurate as to the actual costs in this city, and you've also just flat out ignored a ton of cost verticals.
Quick sanity check: there's zero medical costs. Zero clothing costs. Zero furniture costs. Zero moving expenses. Zero allowance for costs associated with sharing your place with 2 other people. Zero education or progression related investments. Zero capital expenditures on your computer (better hope it never fails, right?).
And that's just a taste. The model is SO wrong as to be laughable.
$11 per day for meals is low, most people who live alone will spend roughly $15 a day (a single meal from McDonald's for instance is $10.95). The cheapest 'eat out' option is roughly $6 for a single meal at Costco/Ikea. Even peanut butter + jelly for every meal, and water for a drink is over $3 a day. Also, for rent, it's more like $800 for a single room in a shared unit, a bachelor is closer to $1500.
In Walmart for $11 you can buy 1 pound of chicken breast ($2/lb), 1 pound of rice ($.50/lb), 1 pound of tomato sauce ($.50), 1 pound of carrots ($1/lb), 1 pound of bread ($.88/20 oz), half pound of butter ($3/lb), half pound of cheese ($2/8 oz), and still have $4 left after tax. You can literally feed a family of four with this. Of course, most nominally poor people will spend much more, because they're not actually all that short on money.
If you take 1860s unskilled worker wage and spend it all on food, you'll get about as much as you could with SNAP benefits (~$130/month) today. The UBI is already here, it's just our standards have rose significantly.
I concur with your experience. Spouse is a great shopper. We probably spend a little more than this daily, but we frequently get 2-3 meals each out of it. Dinner for 2, 2 days lunches. We eat out 2 nights per week, one being take-out pizza and a salad. The salad always gives us enough for part of a 2nd meal. We shop carefully, cook at home. Buy soft drinks, bottled water in bulk, make iced tea, buy big bags of ground coffee, etc. We do this by choice and eat better foods than most people we know. We can afford to spend quite a bit more, but this suits our lifestyle.
This has been my experience when I was growing up, and when I was a student. Food these days is insanely cheap if you aren't going for specific tastes, or if you're good at cooking.
I'm much wealthier now, so I spend much more on food because I can, but if you're really frugal, you can keep your expenses extremely low by simply avoiding expensive ingredients. For example, a single red bell pepper costs $1.50, which is insanely expensive compared to chicken, beans, rice, or carrots, based on the nutritional value. Of course, fresh vegetables are nice, but fresh vegetables any time of the year are very modern invention, thus expensive, and you can easily do without them.
I know the rent is more expensive, in Houston you can easily get a ~800 sq ft 1 br 8 minutes from downtown in a nice neighborhood for $850. If you live in a 2 br with roommates or an efficiency you can get that down to $400-$500.
Yes, the roughly equivalent 1br in Toronto is going to be about $2500 these days.
Also keep in mind Texas has no sales tax, Ontario does. And we are talking CDN not USD (hard to directly compare some things). It's in general a bit more expensive in Toronto, across the board (except health care, obviously).
Houston is one of the cheapest "proper" metros in USA and Canada.
Just to add some context, as others are suggesting $11.60 is not enough for food:
I used to live, fairly recently, on a hard limit of $10 per day for food - I did not eat out (almost never). It was enough to bulk up on (over 4000 calories), was balanced, and even included the prices of some basic supplements I took. This was in an expensive area of the US, that per a quick search is roughly on par with Toronto.
$11.60 seems like it could cover a days food in most, if not all, North American cities. You have to be diligent about budgeting, where you shop, what you buy, etc. But it can be done.
> Nobody is exploiting your friends - it sounds like she complained to you, you took her words at face value and concluded that those other people are at fault.
You're contradicting me by stating (false) assumptions about my friend, and stating (false) conclusions predicated on those.
Your analysis also neglects American Millennial debt service and healthcare costs.
I used a personal example of my friend because it was close to home for me, and I thought that by mentioning it (as connected to an HNer), along with an age range, it would be somewhat closer to home to some other HN people. When people mention a potentially upsetting personal example, we should try to be delicate in wording things.
A somewhat interesting article which strongly turns into a puff piece of MT. I started scrolling to the bottom to find out who was paying for this content and there it was "In his book, Side Hustle From Home: How To Make Money Online With Amazon Mechanical Turk"..
I wonder just how does he manage a new baby, a side hustle that fills every minute with productive money-making and writing a book!.. The author really should be writing a book about his exceptional time management skills instead.
This seems like an almost constant feature of ways to make money online - the people who are making the money are almost uniformly making money by selling tutorials on how to make money online.
Want to make money online playing poker? Pay $800 for Upswing Poker's masterclass!
Want to make money online trading stocks? Check out the ultimate Penny Stock Playbook for only $50!
Want to make money online drop shipping? For only $1600 you can buy the ProfitableOnlineStore course and have all your questions answered!
So it makes you wonder doesn't it: If people are so good at making millions making money online, why are they offering to sell you the secret rather than actually making the millions themselves. The answer is: It's easier to sell gullible people tutorials than it is to actually make a living exploiting arbitrage opportunities in one of the most efficient markets ever known to man.
"Yo dawg, I heard you're thinking of a new way to make money...so I wrote a book about thinking of a new way to make money so you can think about a new way to make money while I make money from you thinking about my new way to think about making money."
Yeah I noticed as well. It seems like what this is really saying is that the Mechanical Turk marketplace is an interesting study in economics. Buyers are offering to buy pieces of information for too high of a price and a select few people have found out how to nab those contracts.
I halfway expected to see a secondary market emerge where you “complete” a top expensive task like “translate this text” by bundling it with five other translation tasks and posting it for three times the price.
How do people feel about the research ethics of this? Many academic studies involve the use of MTurk participants, but it seems as though they are not being paid close to a living or even fair wage ($2 an hour is abysmal!).
Should universities require that studies using MTurk pay some minimum pro-rated hourly wage? That is, if a Stanford HCI study (which seem to use MTurk often) wants workers to fill out a 15-minute survey, they would need to pay at least $8 an hour (so $2).
I’ve been to thesis defenses at Stanford where researchers have explicitly stated that they chose to use MTurk because they can pay workers far less than locally recruited participants.
I wonder if they actually get meaningful results from MTurk participants; you are getting a very skewed and unnatural demographic, and I don't see how any results based on feedback from MTurk participants can be applied to any normal population. Much more so than the traditional study group of students-who-need-to-pay-rent.
I don't see an ethical problem in how much you pay participants (after all, how much it pays or if it pays at all will be one of the variables the study needs to account for). The ethical problem I see is not caring if the results are meaningful, just publishable.
I'd think it's so skewed that it makes the results worthless. I wonder if anyone's done a study on it (e.g. get an actual random US sample vs. a MTurk sample and compare the results).
> I wonder if they actually get meaningful results from MTurk participants
I bet they do. One thing not really touched on in the article is that one of the task qualifiers is “acceptance rate”, i.e., how many of your task submissions were accepted by the person that created the tasks. And that number is usually 99.9% acceptance (of _all_ results you’ve ever submitted) or higher. There is a very strong incentive to not get any kind of rejected submission, because it can quickly tank your MTurk profile and shut off lots of opportunities.
I know that we use MTurk at the Linguistic Data Consortium (and that others in NLP use it as well). I know that those involved with it put in a lot of work to make sure we get good, useful data - there are many papers and strategies on the subject of crowdsourcing.
I'm currently paying MTurkers $2 for a study that takes 10-15 minutes. That works out to $8-$12 per hour. I would never be able to recruit people to participate locally for just $2, but I can on MTurk because mine is one of many tasks they can complete in an hour.
There is an ethic commitee (the IRB) at every university that reviews all academic research involving people. In order to conduct a user study, researchers are required to compensate participants fairly. The reason you can pay MTurkers "far less" is that a study may only take 5 minutes, and researchers typically pay $10/hour. People won't participate in a study unless the minimum payment is decent. I can't imagine many people would want to take the time out of their day to show up at a lab, work on a computer task for 10 minutes, and get paid $2.
Do the people in Thailand who make $7/day generally have reliable access to the Internet for 8 hours a day, with a cost for network access and smartphone under $8/day? And a ready source of loans to cover the initial purchase? And can they read English?
Yeah, internet in our remote village is quite excellent. I've worked from home with on mobile 4G internet, was good enough for video calling. And now have proper broadband in home of course.
English skills aren't great for most people, though if people really wanted to could learn for free using apps like Duolingo (there's English lessons for Thai people).
A lot of people have decent smartphones. Good 4G internet for a month cost me around 600 Baht per month (~15 USD). Sure could be expensive if people make 300 Baht a day, but if they could work through Mechanical Turk, they could quickly earn back the cost, since they would likely earn more compared to working on a farm.
Thai people like to lend money a lot. They're the most indebted of SEA when looking at loans compared to their incomes [0]. The use these loans to buy new cars, houses and phones as well.
I had to do some digging into crowdsourcing for my PhD thesis, and these numbers are well reported in the literature.
Our group made the calculations to ensure that our task would pay at least minimum wage. As a result, we had to throttle our participants because they would hit our servers pretty hard.
That's not to say that the experience was easy - we had to implement every possible sanity check, and even then we ended up throwing away half our data. For us it was still worth it, since twice the rate was still cheap and we knew not to trust the internet. But a less internet-savy researcher blindly trusting their data would have gotten some surprising results.
We had an interactive task (hosted in our own server) and a questionnaire afterwards. Sanity checks off the top of my head:
* Control words: our server gave them two (unique) control words (one for joining, one for winning), and we asked for them in the post-task questionnaire. Some of these words ended up being reused among several participants, even from participants that never even started the task.
* IP checks: we had an experiment that you could "win" (and earn a bonus), but you only were allowed to play once. Some people restarted the task several times, so we only used the first attempt in a sequence (as reported by their IP and timestamp).
* Data thrown away: we further removed data where we had more than one player per IP (to control for both multiple accounts per person and use of proxies), experiments that were way too fast, and experiments with unsupported browsers (which we explictly mentioned in the description).
Regarding money, we were not allowed by the TOS to withold payment to anyone that filled the questionnaire, even if we knew they did it in bad faith. We therefore implemented the "winning" bonus, and also gave bonus to people who lost but really tried.
I want to point out that a LARGE percentage of participants played honestly, and some of their data was thrown away only out of an abundance of caution. Once you keep the first bad apples out, they simply move to other, easily-exploitable tasks.
>Our group made the calculations to ensure that our task would pay at least minimum wage. As a result, we had to throttle our participants because they would hit our servers pretty hard.
What do you mean by this? Do you mean purely requests per second or what?
Yes, requests per second. Everyone would jump to our task. Compared to "regular" experiments, where getting participants requires active recruiting, this was completely unexpected.
The problem with these gigs is that the tasks are extremely generalized. So while they attract more workers, it is always going to be a sellers market. The only way a worker can scale his income, is by increasing volume - and picking the right assignments.
I've been doing side-hustles since I was a teen, and continue today, even though I have a well-paying professional job. My observations are that domain knowledge / specialized expertise always pays better.
You're much better off learning some hobby, almost any hobby, and monetizing it. How do you monetize it? Do content writing. Start flipping on Ebay / Craigslist / etc. Perform / create if you can. etc.
Now - I understand that these (MTurk) kinds of jobs are for desperate people, that want/need the money now - not 4 years down the road. Building expert knowledge will take time, and you will more likely than not get burnt a few times in the start - but the long-term payoff is much better.
Last month, I made $800 in selling two musical instruments. Time invested in it all was around 2-3 hours, mostly on picking 'em up, and shipping them.
Maybe not the answer for everyone, but it's extremely hard for me to imagine any other way now. I currently have three different hobbies which I enjoy very much (and have for years), and I'm lucky enough to have so much knowledge in that I also make a decent side-income off.
Best of all? It doesn't feel like you're working - though you need to treat it like work, if you're gonna make money.
Unless you have a clear
strategy, MTurk work is a
complete waste of time.
True. I perused Mechanical Turk tasks some months ago, and was confronted by an array of plaintive, useless demands to perform what amounts to unproductive garbage picking for listless and disinterested college grad students running unimaginative projects without a clue as to whether the requested task is even possible.
It would be a request like:
Gather phone numbers from
this queue of web pages.
And they'd seem to have paid for a "database" of "leads" which was more than likely an excel spreadsheet of "hyperlinks" categorized according to a search query of keywords from the data provider.
You get into the queue, and start pulling up each URL in series, and they're all these expired domains with parked registrar pages for GoDaddy and Tucows or whatever, Along with some Geocities, Angelfile, Tripod and AOL home pages thrown in. Quickly, you get a sense that some fool of data science masters program enrollee paid good money for a dusty, mouldering text file, didn't even look at it, and dropped it right the fuck into a template for a Mechanical Turk task.
Now, there are three immediately obvious courses of action. One, abort and never again consider Mechanical Turk as a useful platform for operating an exchange of effort for rewards. Two, plead with the task owner by reporting feedback to them and ask them to stop for a moment and consider the flaws inherent to this framing of a human activity deemed worthy of compensation. Three, obey the letter of the law, and not the spirit, hold your nose, bellow the words "you asked for it!" and proceed to fill the task with the tech support numbers for all of the domain registrars, hoping that you'll not only get paid for grifting on the task owner, but also possibly inundate all these domain squatting registrars with robocalls trawling for psyche student surveys and questionnaires that will attempt to publish similarly terrible research papers designed with the intent to ostensibly "prove" a flawed hypothesis of human behavior with results that couldn't possibly be replicated because the hypothesis itself begs its own question.
Valuing my time, I just logged out, and haven't looked back since.
They need moderators to mechanical turk the quality of each task, because it benefits no one and wastes people's time, to even propose fruitless, unredeeming tasks.
Unless things have changed since winter, from what I witnessed, there is perhaps zero review of tasks to assess whether a request fits the profile of anything even remotely possible or worth trying.
Related to this, Stanford created something so you can use Mechanical Turk without being overly exploitative (ensures you pay at least US$15/hr): https://fairwork.stanford.edu/
I have an even cleverer point suggesting that I might not be able to trust workers' self reports. Might my clever rebuttal be right?
Yes. But you are using this tool because you want to help ensure fair treatment to workers, right? Fair treatment starts with trusting workers. They will make good faith estimates.
Possibly Expensify? There was that news that came out a bit ago that their "SmartScan" actually fell back on Mechanical Turk [1] even though it was sold as a AI-type thing.
>“Most people’s first mistake is they go on there and accept any task they see,” says Naab. “They’ll do these transcriptions — $0.01 to transcribe a whole receipt for Expensify. That’s a terrible ROI.”
It's an investment because the more lucrative jobs require turks have hundreds or thousands of successfully completed tasks. So people will take one penny jobs simply to build credibility.
The whole point of MTurk is to get data you will use to train your models. Using some artificial means to make money on MTurk would be circular, and probably easily spotted - if not, your model has effectively solved the problem domain anyway!
We do language related things on MTurk (crowdsourced translation, for example). It's very easy to spot those who have used something like google translate or other artificial means - it's quite rare for a sentence to be translated exactly the same way twice, for starters.
Well, AI can't handle the edge cases yet. For some businesses it makes more economical sense to invest in Turk for transcription rather than training a ML model, which obviously requires a sizeable investment. Either way, you'll need a decent training set and Turk is great for that.
I signed up for Mechanical Turk soon after its creation. I transcribed podcasts and made about $10/hour, which was not bad money for a student, especially considering that these podcasts were on interesting subjects and it did not feel like hard work. After a few months I moved on to other things, and when I looked at MTurk again, I saw that it had quickly become a race to the bottom. No freelancing site can withstand those capitalist pressures for long.
Every time I have looked at Turk lately the HITs I've seen have been awful.
For instance there used to be a lot of HITs that involves transcribing printed receipts.
I wouldn't mind doing this so much if the receipts weren't smudged and creased or otherwise illegible, but I think these people have an OCR that can handle the easy ones and you're left with a residuum of hard cases which sometimes can't be transcribed at all, or for which you'll probably make enough mistakes to get in trouble. I find that mentally fatiguing on top of the extra time.
What would the argument be that some workers in the economy but not these people specifically deserve a minimum wage? I get that they are designated contractors but it seems pretty arbitrary. People would lose their minds if they found out Amazon was paying warehouse workers $2.50 an hour but couldn't they just arbitrarily assign prices per package and call them contractors as well?
Working as a "turker" on Mechanical Turk, or as a Tasker on Taskrabbit, or as a Lyft/Uber driver, all have one thing in common: the switching cost to/from that particular job is almost zero.
In my view, this is why you can get paid less than what you deserve, but that option is still attractive to you.
In other words, if you have 34 minutes between things, one of the few paying things you can do is MTurk, even if it doesn't pay well.
In my brief experience, bursts of actually quite a lot (My favourite being one where you got £1-£2 for "Does this sentence make sense" Yes/No) surrounded by almost nothing.
I've seen some people on Reddit who are effectively gaming it well enough to actually make consistent money, but that's clearly the exception not the rule.
I think many people do this not out of need but as a time waster that pays a bit. As the article mentions, 75% of 'Turkers' are American. The article's subject says, "I’d literally be sitting around bored half the day without it. Making money beats doing nothing."
Of course he wouldn't be doing literally nothing. Instead he'd be just mindlessly wasting time browsing Facebook, checking out the latest clickbait, posting on forums, and engaging in the sort of activities that studies are increasingly showing are destroying our collective mental health and social cohesion. Instead, he's using that time to make an extra thousand a month.
I tried mTurk after reading an article like this years ago, and it was the biggest waste of time ever. It was ungodly tedious for like pennies per task. I don't know how anyone could be doing so well at getting tasks that average out to $0.47 ($45k/95k tasks) like this guy is claiming; this is an absurd outlier.
Is there anything stopping people from just randomly filling in the surveys? It seems like the kind of thing where someone would have made a bot. How many people who are paying a couple of bucks for something like that are going to complain at you?
Survey instruments generally have a few ways to check for this, usually via attention checks which require you to answer in a specific way as stated in the question text (defeating your bot). Similarly a survey could be designed to ask the same or similar question multiple ways, and a survey response that lacks the expected correlation can also be rejected.
I wonder how many task on Mechanical Turk can actually be automated. By that I don't mean building some fancy AI but tasks that can easily be automated and have just been posted on Mechanical Turk.
I guess automating is not always the cheapest option, and for "once in a life" experiment/test/dataset it can be sometimes cheaper to hire people than develop an algorithm.
This is because it's near impossible to register unless you are from the U.S., now, i heard. Was a reply to me when i suggessted bs 'programmers' who had no idea what they were doing, to go there instead of trying to hopelessly beg for projects on Upwork - 'i'd love to, but they locked everyone outside the U.S. out'.
It is a complete waste of time in many ways but every once in a while I will log on and take a couple surveys while watching a rerun of something before bed. I used to do some survey design/analysis in academia and sometimes it is interesting to see what or how weird the surveys are. I would guess my hourly rate is about $3 and I used that very small amount to buy splurge on cheap useless stuff on Amazon.
As a person who actually lives in a 3rd world country and grew up in America, what you’re saying could not be more false. Even the poorest Americans live a life here that would be considered one of great luxury. We’re talking about living on a dirt floor in collapsing mud houses. We’re talking about babies dying from dirty water as a regular occurrence. No running water, intermittent electricity and definitely no sewage. Most people here haven’t eaten meat in months, occasionally years.
If you actually think America is a 3rd world country you need to get out more.
Americans in Flint, MI have poisonous water, and hookworm is on the rise from sewage contaminated water in Alabama.[0] In 2017, millions of Americans went without electricity or running water for months.[1]
> If you actually think America is a 3rd world country you need to get out more.
I said it's becoming a 3rd world country. There's still some legacy 20th century infrastructure that hasn't collapsed, yet.
Not sure why the downvote when it is abundantly clear that in every corner of the USA there is this general trend towards decay with very little offsetting it. All our infrastructure is falling apart. I blame the pension crisis. All available monies are going towards pension obligations. Not much is left over for anything else.
All your comments are marked [dead] and hidden. I've 'vouch'ed for this one because there seems to be nothing obviously wrong with it (and selfishly because it agrees with me).
