Evaluate our new search

Author Message

Thomas Nunninger

Wednesday 20 December 2006 3:07:44 am

Hi,

in the last months we have done research on relevance-based ranking of search results. The result of that research is a test implementation to compare different ranking approaches and configurations. (An article about that will follow soon.)

Because you cannot define relevance in a mathematical formula, your feedback is very important for us, as it allows us to test whether our approaches are successful or not. Thus we encourage you to evaluate our new ranking approach.

The search and evaluation interface can be found at http://eval.ez.no/content/searchevaluation

After you submitted the search, the result will be shown in the bottom of the page (most browsers should jump to the point where the results are listed). If you click on a link the page will be shown in the preview frame above the result list. Then you can rate the result by dragging and dropping the results into the right box (relevant, partially relevant, not relevant) below the result list.

Some more information are listed on the site itself.

You can use this thread if you have problems or suggestions.

Thanks

Thomas

Kristof Coomans

Wednesday 20 December 2006 4:55:54 am

Hi Thomas

What about missing results?

An example: I searched for tree (no other limitations). There were no search results directly leading me to the content/tree fetch function documentation.

independent eZ Publish developer and service provider | http://blog.coomanskristof.be | http://ezpedia.org

Thomas Nunninger

Wednesday 20 December 2006 5:30:05 am

Hi Kristof,

I answered this to your comment in the blog. This is fixed now.

If you want to know in detail: it was due to the indexing of two databases in one search index. So I added the sql string for the permission check of the content objects in the documentation manually. But the permission management in the documentation was changed afterwards: new sections where introduced which where not present in the hard coded permission string.

Have a nice day

Thomas

Jennifer Zickerman

Wednesday 20 December 2006 12:54:18 pm

We have just published an article written by Thomas about his work on the search engine. In addition to details about the implementation, it includes some interesting background information about search concepts and considerations.

http://ez.no/community/articles/creating_a_search_engine

Kristof Coomans

Thursday 21 December 2006 12:16:59 am

Hi Thomas

I actually meant something else than the issue I described in the blog. Some results I would expect (if I search for "tree", I really expect to find an entry about the content/tree fetch function) are not in the result list. If a search doesn't return all expected results, then ranking also won't matter.

By the way, in the article some characters are replaced by question marks (probably the charset wasn't converted properly).

independent eZ Publish developer and service provider | http://blog.coomanskristof.be | http://ezpedia.org

Thomas Nunninger

Thursday 21 December 2006 1:52:45 am

Hi Kristof,

ok, I see your point. And obviously you are right. I looked into the documentation of the fetch( content, tree, ...) function. The term "tree" does occur only three times in the page. Seems not to be enough for the actual configuration :-)

Unfortunately I can't change anything on this at the moment. But I will add those pages to your ratings of that query manually. As I will reconfigure the search index later and re-run the queries, this is usefull information.

<b>@all:</b>

<b>If you miss any important content in the result of your query, feel free to post it here.</b>

I need those information:

- What words did you search for?
- Which subtree and class limitations did you use?
- What content do you miss (URLs if possible)

Thanks

Thomas

kracker (the)

Friday 05 January 2007 10:25:18 am

Thomas,

I think it's really great that work is being done to improve searching and search results.
I did rate several searches before coming to a conclusion.

I'm not certain how hard it would be to add multi-selection of items in the results list ... yet if this is a control set / ui to be reused later. I would suggest strongly to see if multi selection could be supported.

I found I could only do so many one <i>at a time</i> labeling of each of the search result items before it becoming a bit tedious. I hope my continued ratings are of use.

Cheers,
//kracker

Member since: 2001.07.13 || http://ezpedia.se7enx.com/

Thomas Nunninger

Saturday 06 January 2007 5:48:18 am

Hi Kracker,

<i> I'm not certain how hard it would be to add multi-selection of items in the results list ... yet if this is a control set / ui to be reused later. I would suggest strongly to see if multi selection could be supported.</i>

I'm not sure if I got you right. I think you are talking about multiselecting items for rating... The interface is only for evaluation. Our idea of drag and drop was, that you can see easily, what objects you rated how. We thought that people perhaps rate results related to others.

<i>I hope my continued ratings are of use.</i>

Of course. The more ratings we get, the better it the results will be.

Thank you very much

Thomas

kracker (the)

Sunday 07 January 2007 2:12:09 am

No worries :)

It's just not very useful for large scale ratings without a more selectable gui containers in the results selection ui.

When rating hundreds of objects via one at a time 'click, drag, release, drag, repeat' one can <i>quickly</i> become tedious.

When an intuitive would expect making a selection of items from the results for each 'click, drag, release, drag, repeat' filing into ratings.

This might affect the relevance collected either shallow in depth of results rated and accuracy.
Still, I seek to continue to support your means to an end, regardless.

I wrote up a variation the <i>opensearch</i> plugins contributed recently for this effort. The search plugin once installed, can be used to connect user's browser search input to your search evaluation site. It is limited to searching all sections with no limitations atm.

Plugin, <i>http://pubsvn.ez.no/community/tools/browser_opensearch_plugins/ez_no__search-eval.xml</i>.
Installation, <i>http://ezpedia.org/wiki/en/ez/opensearch#eztoc36541_2</i>

Perhaps you can use it will increase the frequency of or number of people using the search evaluation.

hth,
//kracker

<i>Anime: Big O - Roger the Negotiator</i>

Member since: 2001.07.13 || http://ezpedia.se7enx.com/

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.