Evaluate our new search

Wednesday 20 December 2006 3:07:44 am

Hi,

in the last months we have done research on relevance-based ranking of search results. The result of that research is a test implementation to compare different ranking approaches and configurations. (An article about that will follow soon.)

Because you cannot define relevance in a mathematical formula, your feedback is very important for us, as it allows us to test whether our approaches are successful or not. Thus we encourage you to evaluate our new ranking approach.

The search and evaluation interface can be found at http://eval.ez.no/content/searchevaluation

After you submitted the search, the result will be shown in the bottom of the page (most browsers should jump to the point where the results are listed). If you click on a link the page will be shown in the preview frame above the result list. Then you can rate the result by dragging and dropping the results into the right box (relevant, partially relevant, not relevant) below the result list.

Some more information are listed on the site itself.

You can use this thread if you have problems or suggestions.

Thanks

Thomas

Wednesday 20 December 2006 5:30:05 am

Hi Kristof,

I answered this to your comment in the blog. This is fixed now.

If you want to know in detail: it was due to the indexing of two databases in one search index. So I added the sql string for the permission check of the content objects in the documentation manually. But the permission management in the documentation was changed afterwards: new sections where introduced which where not present in the hard coded permission string.

Have a nice day

Thomas

Thursday 21 December 2006 12:16:59 am

Hi Thomas

I actually meant something else than the issue I described in the blog. Some results I would expect (if I search for "tree", I really expect to find an entry about the content/tree fetch function) are not in the result list. If a search doesn't return all expected results, then ranking also won't matter.

By the way, in the article some characters are replaced by question marks (probably the charset wasn't converted properly).

independent eZ Publish developer and service provider | http://blog.coomanskristof.be | http://ezpedia.org

Friday 05 January 2007 10:25:18 am

Thomas,

I think it's really great that work is being done to improve searching and search results.
I did rate several searches before coming to a conclusion.

I'm not certain how hard it would be to add multi-selection of items in the results list ... yet if this is a control set / ui to be reused later. I would suggest strongly to see if multi selection could be supported.

I found I could only do so many one <i>at a time</i> labeling of each of the search result items before it becoming a bit tedious. I hope my continued ratings are of use.

Cheers,
//kracker

Member since: 2001.07.13 || http://ezpedia.se7enx.com/

Sunday 07 January 2007 2:12:09 am

No worries :)

It's just not very useful for large scale ratings without a more selectable gui containers in the results selection ui.

When rating hundreds of objects via one at a time 'click, drag, release, drag, repeat' one can <i>quickly</i> become tedious.

When an intuitive would expect making a selection of items from the results for each 'click, drag, release, drag, repeat' filing into ratings.

This might affect the relevance collected either shallow in depth of results rated and accuracy.
Still, I seek to continue to support your means to an end, regardless.

I wrote up a variation the <i>opensearch</i> plugins contributed recently for this effort. The search plugin once installed, can be used to connect user's browser search input to your search evaluation site. It is limited to searching all sections with no limitations atm.

Plugin, <i>http://pubsvn.ez.no/community/tools/browser_opensearch_plugins/ez_no__search-eval.xml</i>.
Installation, <i>http://ezpedia.org/wiki/en/ez/opensearch#eztoc36541_2</i>

Perhaps you can use it will increase the frequency of or number of people using the search evaluation.

hth,
//kracker

<i>Anime: Big O - Roger the Negotiator</i>

Member since: 2001.07.13 || http://ezpedia.se7enx.com/