Crowdsourcing made better

765140960_735722ddf8_zRead an article the other day in MIT News (Better wisdom from crowds) about a new approach to drawing out better information from crowdsourced surveys. It’s based on something the researchers have named the “surprising popularity” algorithm.

Normally, when someone performs a crowdsourced survey, the results of the survey are typically some statistically based (simple or confidence weighted) average of all the responses. But this may not be correct because, if the majority are ill-informed then any average of their responses will most likely be incorrect.

Surprisingly popular?

10955401155_89f0f3f05a_zWhat surprising popularity does, is it asks respondents what they believe will be the most popular answer to a question and then asks what the respondent believes the correct answer to the question. It’s these two answers that they then use to choose the most surprisingly popular answer.

For example, lets say the answer the surveyors are looking for is the capital of Pennsylvania (PA, a state in the eastern USA) Philadelphia or not. They ask everyone what answer would be the most popular answer. In this case yes, because Philadelphia is large and well known and historically important. But they then ask for a yes or no on whether Philadelphia is the capital of PA. Of course the answer they get back from the crowd here is also yes.

But, a sizable contingent would answer that the capital of PA is  Philadelphia wrong (it is actually Harisburg). And because there’s a (knowledgeable) group that all answers the same (no) this becomes the “surprisingly popular” answer and this is the answer the surprisingly popular algorithm would choose.

What it means

The MIT researchers indicated that their approach reduced errors by 21.3% over a simple majority and 24.2% over a confidence weighted average.

What the researchers have found, is that surprisingly popular algorithm can be used to identify a knowledgeable subset of individuals in the respondents that knows the correct answer.  By knowing the most popular answer, the algorithm can discount this and then identify the surprisingly popular (next most frequent) answer and use this as the result of the survey.

Where might this be useful?

In our (USA) last election there were quite a few false news stories that were sent out via social media (Facebook and Twitter). If there were a mechanism to survey the readers of these stories that asked both whether this story was false/made up or not and asked what the most popular answer would be, perhaps the new story truthfulness could be completely established by the crowd.

In the past, there were a number of crowdsourced markets that were being used to predict stock movements, commodity production and other securities market values. Crowd sourcing using surprisingly popular methods might be used to better identify the correct answer from the crowd.

Problems with surprisingly popular methods

The one issue is that this approach could be gamed. If a group wanted some answer (lets say that a news story was true), they could easily indicate that the most popular answer would be false and then the method would fail. But it would fail in any case if the group could command a majority of responses, so it’s no worse than any other crowdsourced approach.

Comments?

Photo Credit(s): Crowd shot by Andrew WestLost in the crowd by Eric Sonstroem

 

To iPad or not to iPad – part 3

Apple iPad (wi-fi) (from apple.com)
Apple iPad (wi-fi) (from apple.com)

Well I did take the iPad and BlueTooth (BT) keypad to a short conference a couple of weeks ago and it was a disaster unlike what I envisioned in Parts 1 & 2 of this saga.  It turns out that some WiFi logins don’t work with the iPad (not sure if this is “Flash” issue or not).  In any event, the iPad was rendered WiFi-less during the whole conference which made for an unconnected experience to say the least (recall that I don’t own a 3G version).

The hotel used T-Mobile for their WiFi connection.  I must have created my account at least 3 times and tried to log-in afterward at least 5 times (persistance occasionally pays but not this time). Each time the login screen hung and I never got in.  The conference had a different WiFi supplier but it had the same problem only this time all I had to do was to sign into the service with a conference supplied SSID&password.  No such luck.  The hotel gave me two free WiFi card keys for T-Mobile but I can’t use them.

I even tried some of the tricks that are on the web to get around this problem but none worked. Nuts!

The blog post from hell

Of course, I didn’t plan to write a blog post at the conference but I had the time and the muse struck.  So I whipped out my trusty iPhone, paired the BT keypad with the iPhone, used Notes and WordPress App (WP, available free) to create a new blog post.  I power typed it into the iPhone Notes app and copied and pasted into WP’s new post window.

I was always curious how to add media to posts via the WP app but anything on the iPhone including the photo library and camera photos were accessible as new media to be added to any post.  I had used my iPhone to earlier take some pictures from the conference and easily added these to the post.  The WordPress app uses the more primitive editing window (not WYSIWYG) but that was ok as I didn’t have a lot of fancy text layout.  What’s funny is that saving on the WP app was not the same as uploading it to my blog.  And once uploaded you had to change the post status to Published to get it externally visible.

Another option would have been to use the web and update the blog post through WordPress on Safari. I  can’t recall but last time when I used Safari & WordPress there were some scrolling incompatibilities (inability to scroll down into the post – flash maybe) and there were other nuisances, so I decided to try the WP app this time.

The only problem with using the iPhone & WP app to enter the post was that it was hard to check spellings and see the whole post to edit it properly.  Only really got to see a couple of (short) lines at a time in the iPhone WP app window and the WP app preview was not all that useful.

Needless to say, the post was published with numerous typos, mis-spellings, grammatical faux pas, etc. (so what’s different Ray?).   A few readers caught the issues and DMed me on Twitter which I picked up later that night.  I tried my best to fix them but it still had problems a day later when I got to my desktop.  For some unknown reason, it became my most  popular post – go figure.

Using the iPhone at the conference

Of course the iPhone 4 worked fine for emails, twitter, facebook and other social media given its screen and soft keypad limitations during the conference.  And I was still able to take notes with the iPad I just couldn’t send them anyplace and would have liked to insert them into the post as an outline but couldn’t be done.

There is just no way to get data out of an iPad without WiFi or 3G access.  Maybe if I could take a screen shot with the iPhone and then use an OCR app to interpret it into a Notes item and then I could get the text into iPhone – but I didn’t have an OCR app at the time. Also, it smacks of a Rube Goldberg contraption.

—-

I would say the WP app on the iPad looks a lot better than the one on the iPhone but much of that is due to the increased screen space.  If everything was working fine I probably wouldn’t have had as many problems using iPad WP app to enter in the post.  Of course I would have had to mail the photos from the iPhone to the iPad to enter them into the post but this is standard practice with the iPad…

There’s another conference coming up (it’s conference season here in the US) and I am NOT taking the iPad. Too bad, my back hurts already just thinking about it.  I foresee either a 3G iPad or the Mac Air laptop sometime in my near future but for now on it’s lugging laptops.

Just not sure if I shouldn’t take the BT keypad to take notes on the iPhone!?

PS. Saw Rob Peglar and he had a Verizon Dongle that provided a local WiFi for his iPad and 4 other “close” friends.  Maybe that’s what I should invest in?