Amateurs and Experts: Netflix Challenge

Andrew Keen’s slim volume, The Cult of the Amateur, was an interesting* read. One question he asked came to mind as I read this month’s Wired (16.03): “Can a social worker in Des Moines really be considered credible in arguing with a trained physicist over string theory?”

What’s an amateur? I originally considered the guy from the Wired article an amateur: “This Psychologist Might Outsmart the Math Brains Competing for the Netflix Prize”.

Here’s my executive summary of the Wired article:

  • Netflix is offering a $1M prize to the individual or team that can improve their recommendation engine by 10% or more.
  • Heavy hitters in the math world, including a team from AT&T and Princeton alums, are competing.
  • So is “Just a guy in a garage”.
  • Gavin Potter, a 48 year-old psychologist in London is currently in 6th place (that was as of press time, he’s down to 8th currently).

I don’t think Gavin Potter is an amateur, though on the surface he may appear to be one. Certainly, he’s no programmer or mathematician. The other teams approaching the prize are heavy on professional numbers people. Potter, on the other hand, has asked his daughter to solve calculus for his Netflix Prize work.

The Des Moines social worker may have something to contribute to something outside his/her obvious expertise. The trick is in figuring out how to apply existing knowledge in other fields of study.
* “interesting” in the Chinese proverb/curse way: “May you live in interesting times.”