Saturday, April 20, 2013

Mturk reactions to profile pictures

Feeling down? Pay people on the internet to say nice things about you!

Not really... This was more about seeing what people would write when asked for their impression of a photo

This post is one part mechanical turk tutorial and one part experiment about what a profile photo says about a person and if other people can usefully describe their impressions.

Part 1: Mturk stuff

I have another blog post bitching about external hits and a super old mturk API. This time, I just used the website because my job fit one of the templates. I did, however, call upon some extra wisdom that is not obvious on the site that I will mention here.

Mturk build-a-hit within the mturk website. Adapted image transcription template.

 

Wisdom nugget 1: Mturk just wants a nice HTML form.

The mturk hit building website has a HTML-like editor for setting up a job. One might expect the WYSIWYG editor to include mturk-friendly primitives. Like places to enter text and places to show images like you saw in the templates. Alas, no.

I'm here to tell you that you just need to go into the HTML view and write your own damn HTML form elements (without the form tag). Don't forget to give your form elements names, otherwise they will be ignored. And if you have no named form elements, mturk will get mad at you. I made a textarea named impression.

<textarea rows="4" cols="50" name="impression"></textarea>

That's pretty much it. Mturk will display your html chunk inside itself. Then workers will enter stuff into your form fields, and mturk will save it in a very reasonable way that you can view later and download as a csv file.

Wisdom nugget 2: Mturk template variable things.

You might wonder, how do I customize a job, eh? What if I have a list of photos I want feedback on? To do that, define your own variables inside your job layout. I had this line, with ${image_url} being the variable in question.

<p><img src="${image_url}" style="margin-right: 10px;" alt="image_url" /></p>

Then once I saved my job layout thing, (sorry, I don't care to figure out the exact vocabulary... the website is pretty straight forward, except when it's not) I made actual jobs that actual people could do by uploading a file that defined my variables. It looked like this, with the variable name(s) at the top. I know you can have multiple variables by using tabs (or maybe commas) to separate them.

image_url
http://homes.cs.washington.edu/~ktuite/profilereactions/fb.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/cse.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/flickr1.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/flickr2.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/github.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/gplus.jpg
http://homes.cs.washington.edu/~ktuite/profilereactions/linkedin.jpg


Viewing Mturk results

I guess they intended people to spend a small amount of time making jobs and a very large amount of time actually dealing with the results/finished work. So the job results viewer is pretty slick.

Watching other people get work done is oddly satisfying.

View of the results. See my image_url field and a textarea field I had named "impression".

Great, now that you know about html forms and csv files defining the variables of your jobs.... you can go off and make simple mturk tasks super easily! If you don't want to do that right now and instead want to hear about the experiment I just ran, continue reading.

Part 2: Profile picture reaction experiment

CSE homepage, Facebook, Flickr 1, Flickr 2, Github, Google+, Linkedin

Whether we like it or not, the images we display on our online profiles affect peoples' impressions of us. I realized I had five different online profile images across five sites, and while I had vague reasons for picking those particular images, I've never actually validated what impressions other people get from looking at those images. For example, I use a hand-drawn picture for github, because it's where I craft other things, like code. I used those five images here, and two other recent ones that I don't think I'd ever use as a profile picture, but that I kind of like.

Prompt

I paid people $0.11 per impresion. I didn't figure out how to limit one worker to one picture so they could do multiple hits and see multiple pictures of me. Here's the text they saw:

This is a public photo of me on the internet and I want to know what it says about me.

  • What impression of me does this photo give you?
  • What could I change about the photo to have it give a more positive impression of me?

Honestly, this worked better than I expected. I thought people might be too harsh or too nice or too creepy, but I think the responses are reasonable. The main suggestions were to smile more and maybe remove my glasses.

very wonderful and very attractive photo.
photo clarity was very bad
hidden thoughts, lonely, thoughtful, shy, healthy, self-conscious, camper, adventurer, enjoys hiking, nice scenery
This is my current facebook profile. I like it because Adam took it, it's got a nostalgic feeling, and I think I look pretty. The internet requested that the background be changed to the sea. I could be up for that, the sea is nice. There was also this kind of cheesy suggestion: "The only thing I would change is to have a photo with the sun shining bright and with you having a radiant smile on your face." One radiant smile coming up.

mountain, desert, morning, happy, adventurer, explorer, free, interesting, intelligent, dark, silent, calm, nervous, gloomy, docile
I think this is a cool picture. Taken in the grand canyon last week, at a location we could only be at at sunset because we were camping down there. I wouldn't use it as a profile picture because the internet is right, it's too dark and too gloomy. My favorite comment, though, is that it's "a little strange, in a good way."

playful, bold, tourist, tracker, nerdy, curious, cute, sweet, sexy, happy, calm, relaxed
Here's another grand canyon picture that I would not use as a profile picture because I know how long I'd been without a shower when the picture was taken. This is more of a "I was here, I looked like this as I was hiking out" picture. One piece of feedback was, "could tone down a little bit." Maybe that means the brightness/shadows. Another comment was to look more directly into the camera.

nice, pretty, cool, popular, attractive, curious, not smiling, judging
A drawing I traced of myself as my github/gravatar picture. Someone said, "I could feel clear sense of judgement in your eyes and it is not positive." Other suggestions were to smile more and face the camera. This one I might actually change, if I can find a different picture of myself that I think still conveys hacker/nerd/artist. 

anxious, sympathetic, eyes are beautiful, silent, speak no evil, funny, cool, wild, weird
I didn't want to give Google+ too much attention, so I uploaded this artsy-farsty blank face picture. I think it's kind of cool, but I know it's a bit weird. Mturk people wanted a nose and mouth to be present, and someone said, "weird and a little disturbing, maybe you're a goth or a zombie lover." Maybe! Honestly, that's a fine thing for viewers of my Google+ profile to think.

natural, cute, sweet, calm, gentle, charming, smart, intelligent, redhead, happy, pleasant, sophisticated, alive and confident
Taken by the department photographer, this became my "professional" portrait on Linkedin. It feels a bit awkward to me, and I wish my chin was more defined than it is. The internet seems to think it's good, though. It's definitely giving the right impression for its purpose, even if it doesn't feel like "me" to me. Someone said, "This picture gives quite positive impression about you. It may be because of the mild smile and the happy eyes. That specs make you look like an office lady." Someone also suggested moving the glasses up a bit and smiling more. Those sound like good suggestions to me.

serious, quiet, dreamy, reserved, friendly, reasonable, cute, far away thoughts, wonder what you're thinking about, inquisitive, deep
Finally, here's the picture on my academic homepage. It feels more "me" than the previous one. Yup, just hangin' out in a tree, smiling a little bit and looking thoughtful... People had suggestions for improving it, though: smile more, lose the glasses, wear more makeup (although it might be okay to not have much makeup if you're outside, especially in a tree), face the camera. I could do those things, but then I might look less thoughtful and inquisitive! Something to consider, though.


Wrap up

So it seems that my prompt worked pretty well and mturk users provided reasonable feedback and even tried to provide a reading of my emotional state. People's impressions jive pretty well with what I intended, though it did get me thinking.  I might even update one or two of those pictures based on their suggestions.

1 comment:

  1. This is a pretty cool experiment! I like that people actually analyzed it on the emotional side. It'd be kind of fun to see this sort of experiment on a wide scale to see what the most ideal profile pictures are for different situations, haha.

    ReplyDelete