Project for Computational and cognitive neuroscience summer school, Shanghai, 2014.

Most of the files should be adequately annotated and straight-forward. Example question banks and answer formats are also supplied.

• Can you improve mediocre scores of individuals by combining them with responses of other individuals (who, in turn, also had mediocre scores)?

• Can response time be used as a proxy for confidence?

• In general, combining answers from multiple individuals (and selecting the most common answer) imporves scores, a phenomenon known as "wisdom of the crowd". The theory is that wrong answers (especially in a forced-choice test) will be random and noisy, whereas right answers will not be noisy. Thus, with enough samples even a small number of correct answers separates itself from the noise. This can cause confounds in observational data (where we call it p-hacking due to not correcting for multiple comparisons).