Spring 2006 DLF Forum: Melvyl Recommender Project, 28
3 April, 2006
Preliminary Results
●In general, all 3 content ranking methods beat unranked in returning “Very Useful” items
●All 3 content ranking methods put more “Very Useful” items in top quartiles
●Preferences differed by expertise
●No clear-cut advantage to a single ranked method
●More queries per task using unranked method
Caveat: still in the midst of analysis

Some questions to be settled before we can make strong statements.  E.g.:
•How to account for very large and very small result sets?
•Sys_id, used for generating “unranked” sets, may not be completely random.