Read an article in Stanford Research, Crowdsourced research gives experience to global participants that discussed an activity in Stanford and other top tier research institutions to try to get global participation in academic research. The process is discussed more fully in a scientific paper (PDF here) by researchers from Stanford, MIT Media Lab, Cornell Tech and UC Santa Cruz.
They chose three projects:
- A HCI (human computer interaction) project to design, engineer and build a new paid crowd sourcing marketplace (like Amazon’s Mechanical Turk).
- A visual image recognition project to improve on current visual classification techniques/algorithms.
- A data science project to design and build the world’s largest wisdom of the crowds experiment.
Why crowdsource academic research?
The intent of crowdsourced research is to provide top tier academic research experience to persons which have no access to top research organizations.
Participating universities obtain more technically diverse researchers, larger research teams, larger research projects, and a geographically dispersed research community.
Collaborators win valuable academic research experience, research community contacts, and potential authorship of research papers as well as potential recommendation letters (for future work or academic placement),
How does crowdresearch work?
It’s almost an open source and agile development applied to academic research. The work week starts with the principal investigator (PI) and research assistants (RAs) going over last week’s milestone deliveries to see which to pursue further next week. The crowdresearch uses a REDDIT like posting and up/down voting to determine which milestone deliverables are most important. The PI and RAs review this prioritized list to select a few to continue to investigate over the next week.
The PI holds an hour long video conference (using Google Hangouts On Air Youtube live stream service). On the conference call all collaborators can view the stream but only a select few are on camera. The PI and the researchers responsible for the important milestone research of the past week discuss their findings and the rest of the collaborators on the team can participate over Slack. The video conference is archived and available to be watched offline.
At the end of the meeting, the PI identifies next weeks milestones and potentially directly responsible investigators (DRIs) to work on them.
The DRIs and other collaborators choose how to apportion the work for the next week and work commences. Collaboration can be fostered and monitored via Slack and if necessary, more Google live stream meetings.
If collaborators need help understanding some technology, technique, or too, the PI, RAs or DRIs can provide a mini video course on the topic or can point to other information used to get the researchers up to speed. Collaborators can ask questions and receive answers through Slack.
When it’s time to write the paper, they used Google Docs with change tracking to manage the writing process.
The team also maintained a Wiki on the overall project to help new and current members get up to speed on what’s going on. The Wiki would also list the week’s milestones, video archives, project history/information, milestone deliverables, etc.
At the end of the week, researchers and DRIs would supply a mini post to describe their work and link to their milestone deliverables so that everyone could review their results.
Who gets credit for crowdresearch?
Each week, everyone on the project is allocated 100 credits and apportions these credits to other participants the weeks activities. The credits are used to drive a page-rank credit assignment algorithm to determine an aggregate credit score for each researcher on the project.
Check out the paper linked above for more information on the credit algorithm. They tried to defeat (credit) link rings and other obvious approaches to stealing credit.
At the end of the project, the PI, DRIs and RAs determine a credit clip level for paper authorship. Paper authors are listed in credit order and the remaining, non-author collaborators are listed in an acknowledgements section of the paper.
The PIs can also use the credit level to determine how much of a recommendation letter to provide for researchers
Tools for crowdresearch
The tools needed to collaborate on crowdresearch are cheap and readily available to anyone.
- Google Docs, Hangouts, Gmail are all freely available, although you may need to purchase more Drive space to host the work on the project.
- Wiki software is freely available as well from multiple sources including Wikipedia (MediaWiki).
- Slack is readily available for a low cost, but other open source alternatives exist, if that’s a problem.
- Github code repository is also readily available for a reasonable cost but there may be alternatives that use Google Drive storage for the repo.
- Web hosting is needed to host the online Wiki, media and other assets.
Initial projects were chosen in computer science, so outside of the above tools, they could depend on open source. Other projects will need to consider how much experimental apparatus, how to fund these apparatus purchases, and how a global researchers can best make use of these.
My crowdresearch projects
Some potential commercial crowdresearch projects where we could use aggregate credit score and perhaps other measures of participation to apportion revenue, if any.
- NVMe storage system using a light weight storage server supporting NVMe over fabric access to hybrid NVMe SSD – capacity disk storage.
- Proof of Stake (PoS) Ethereum pooling software using Linux servers to create a pool for PoS ETH mining.
- Bipedal, dual armed, dual handed, five-fingered assisted care robot to supply assistance and care to elders and disabled people throughout the world.
Non-commercial projects, where we would use aggregate credit score to apportion attribution and any potential remuneration.
- A fully (100%?) mechanical rover able to survive, rove around, perform scientific analysis, receive/transmit data and possibly, effect repairs from within extreme environments such as the surface of Venus, Jupiter and Chernoble/Fukishima Daiichi reactor cores.
- Zero propellent interplanetary tug able to rapidly transport rovers, satellites, probes, etc. to any place within the solar system and deploy theme properly.
- A Venusian manned base habitat including the design, build process and ongoing support for the initial habitat and any expansion over time, such that the habitat can last 25 years.
Any collaborators across the world, interested in collaborating on any of these projects, do let me know, here via comments. Please supply some way to contact you and any skills you’re interested in developing or already have that can help the project(s).
I would be glad to take on PI role for the most popular project(s), if I get sufficient response (no idea what this would be). And I’d be happy to purchase the Drive, GitHub, Slack and web hosting accounts needed to startup and continue to fruition the most popular project(s). And if there’s any, more domain experienced PIs interested in taking any of these projects do let me know.
Comments?
Picture Credit(s): Crowd by Espen Sundve;
Videoblogger Video Conference by Markus Sandy;
Researchers Night 2014 by Department of Computer Science, NTNU;
