Last month, we wrote about a study that seems to show that over 40% of the requests on Amazon‘s labour marketplace, Mechanical Turk, were for spam. It seemed both to make sense — Mechanical Turk’s job is to automate small, repetitive tasks that humans do better than computers, which spam certainly falls under — and be damning for Amazon.Now Amazon has responded and denied the allegations, to Fast Company. But, the denial isn’t much of a denial.
An Amazon spokesman starts by saying that “our data indicates that the majority of the work [on Mechanical Turk] is valid,” which, well, if 40% of the work is spam, that means 60%, i.e. a majority, is legitimate, but it’s still pretty bad. The spokesman then notes that Mechanical Turk’s terms of service forbids this kind of behaviour, which is all well and good, but makes little difference if it’s not enforced, and that there are systems to report it, which, again, is a good thing but gives little indication as to whether these systems are working.
The spokesman goes on to say that Amazon’s own data shows the study to be off by “at least an order of magnitude.” Ok, we’ll take Amazon’s word for it. But we’d still like more details and hard data.