12/27 - Wintry Wednesday!

Should be noted that that info is needed for contacting requesters that you haven't submitted a HIT for. Otherwise, the contact form link is under the status detail page, when you click on the date and look at your HITs Completed log ->

 

@ThisPoorGuy @gurlondrums @HarryHood @Trickydude24 @NBadger fk i need to put all the people smarter than me who do anything IT related in their own group..

Spitballing an idea here:

Say I have a group of "trusted reviewers" - folks who everyone generally trusts their reviews & all that goodie two-shoes stuff.
I also have groups of users who are new to the system, they may be submitting good data or bad, but without manual intervention I have no way to tell.

Thoughts on using comparative data from "trusted" reviewers, matching it to "new" users when they've both done the same Req/HIT and bringing large outliers up for manual review when they deviate too much from trusted reviewers?

I don't want to auto-delete data, since that seems iffy, but I also don't want to have to go through every review all day by hand.. I feel like as an off-the-cuff idea this could lead to a bit of an echo-chamber of data, but OTOH that may not be a bad thing.

It kinda swings both ways.. I noticed earlier today C-SATs is randomly rated at $22/hr on TO2.. That's just as inaccurate (...and disappointing) as a Req who accidentally gets a $1/hr review lol.

I'm also thinking of setting a lower bounds threshold for # of reviews required to update Requester aggregate data if the pay is below minimum-wage. This way one person 1-bombing a Requester has literally zero effect, we'd have to have some consensus (even if its only 2 or 3 people) before throwing them in the trash heap.

 

Thanks, hadn't realized this. Also didn't realize worker removed the contact requester in the HIT details capsule.

 

6 degrees in the Falls. Don't know what everyone is complaining about, for Buffalo, that's practically swimsuit weather!

 

This is obvi open to anyone w/ a pulse.. I was just tagging people who may have an idea on the query load for doing something like this..

 

TADA

not pictured: much swearing, going from 8 ram down to 4 ram somehow, getting help from discord friends, learning about bios updates and other shit oh my

 

Title: Help us categorize this sample of online reviews ! [ $ | $ ]
Requester: Datascience Mturk (cc: B2B/ENG-DS/TECHSW) [A2TAN13KGEMTQ2]
TurkerView: [ $ / hour Unrated | Unrated | Unrated ]
Description: Do the sentences relate to the category ?
Duration: 00:16:40
Available: 43
Reward: $0.20
Qualifications:

• Qualification test for review tagging GreaterThan 12,

PREVIEW | PANDA​

[3ZAK6NWIZ30O1H7B8BNMAMWAI9H9VE]​

 

So, where are my Christmas tips, everyone?

 

You can just make it up, its what MTS does, just any string matching /[A-Z0-9]/

 

Hmm.. transcribe a pdf with 32 rows each with eight pieces of information per row, for $0.20 and no bonuses.

Duck fat.

 

It's not Christmasy but I can give you plenty of tips.

1. Never trust smiling friends with meat cleavers behind their backs.

 

What if you had some sort of a threshold so that 'new users' have to get "x" amount of "Good Review!" flags? That way you can kind of leave the system up to others looking at reviews instead of having to manually review newbies? Not that newbies aren't important, but most of us aren't going to rate a 1-bomb or $500/hr = underpaid review as a 'good review'. Then once said new user reaches 'x' amount of "Good Review!", you can do a quick manual overview (or automate it, but manual might be good to start with until TV grows) before approving them as a 'trusted user' who will have more weight in a review score.

So if trusted users have a weight and new users have a lesser weight, it won't hurt the impact of the overall score much.

Hope that makes sense lol.

 

...mturk is stupid.

 

Yeah that does make sense but FWIW I wasn't planning on having the 'trusted user' thing weight their reviews in the aggregate scoring (seems way too difficult to micro-manage that accurately given current resources), was just gonna use it so that if someone came in and started doing stuff like flagging Pinterest at $4/hr it would throw up a "hey, go look at this because its WAY different than what everyone else thinks" flag in the staff panel.

I do like incorporating ratings into who gets a trusted reviewer thing, though tbh I'm probably going to pull an Amazon and people will accrue points per review for a litany of criteria. I've been playing with it on the back end and seeing what people "score" as I adjust the rates for various pieces of a review (SPOILER: updating the approval time is worth mega points LOL) but using that to auto-grant trusted status is great.

 

I'm sorry how about you drag her ass back out of bed and throw her in front of a Stats 101 textbook for the evening. We need more girls in the math & sciences if we wanna keep up with China, really need you to do your part as a God fearing American friend.

@gurlondrums you too.

...No one question my parenting advice, clearly I have the kid's best interests at heart.

 

If you think my parenting advice is bad you should see my career advice.. Oh wait...

 

Amazon has Assassins Creed Origins for $29.99 on xbox right now.
https://www.amazon.com/dp/B072M6DHNX/?tag=thtv02-20