Semantic Search Measurement
Today's search engines are still very sensitive
to
the way queries are constructed. In some occasions, equivalent but
slightly different forms of a query lead to completely different
results. However, popular queries with only one right answer seem to be
generally well served by search engines, which generally return the
correct answer among their top 10 search results. Internet's redundancy
of information and the recent proliferation of user generated content
helps search engines to remain almost entirely keyword oriented and
still robustly handle equivalent versions of queries. We propose a
family of metrics to evaluate the semantical invariance of a given
search engine, and we report experimental results for well-known
engines such as Google, Yahoo!, Live and Ask.com, as well as for new
semantic search engines like Hakia and Cuil.