Users' empirical evaluation of results

Quality of pages varies widely

• Relevance is not enough
• Other desirable qualities
  • Content: Trustworthy, new info, non-duplicates, well maintained,
  • Web readability: display correctly & fast
  • No annoyances: pop-ups, etc

Precision vs. recall

• Precision = proportion of retrieval relevant
  Recall = proportion of all revelant found
• On the web, recall seldom matters

What matters

• Precision at 1? Precision at 1 for first page?
• Comprehensiveness - must be able to deal with obscure queries
• Recall matters when the number of matches is very small

User perceptions may be unscientific, but are significant over a large aggregate