eZFind facets problem with ezselection datatype indexed as text

Wednesday 13 January 2010 7:34:13 am

Mapping ezselection to string is the right solution with eZ Find 2.1

Declaring it as text means it is subjected to transformations like stemming (which removes the "s" at the end) and tokenizing

Paul

eZ Publish, eZ Find, Solr expert consulting and training
http://twitter.com/paulborgermans

Wednesday 27 January 2010 5:32:08 am

Hi Martin,

I thought that we won't need multiple select in our project, but finally we need and we've got exactly the same problem. No solution for now.

Baobaz
http://www.baobaz.com

Wednesday 27 January 2010 10:31:33 am

Ok Paul thank you a lot.

Until a fix i will loop class attribute values and won't display number of results.

EZP is Great

Tuesday 14 June 2011 12:13:13 pm

Any updates on this?

eZ Publish certified developer
http://auth.ez.no/certification/verify/392313

Wednesday 15 June 2011 1:22:24 am

Great!

eZ Publish certified developer
http://auth.ez.no/certification/verify/392313

Thursday 14 July 2011 11:50:27 am

We have a solution!  My talented co-worker David Sayre did the majority of the work on this, and we're happy to share it with everyone.

Basically we created a new class that handles the indexing of ezselection datatype attributes, and passes their values as an array to Solr rather than a string with spaces in it.

Wasn't sure of the best way to put this out there other than pasting it here, so please let me know if there's a better one.

NB:  I had to completely blow away the contents of /usr/local/solr/java/solr/data/index/ before this would take affect, but YMMV.

To implement:

1. Add the following to your ezfind.ini override file:
2. Add this file to your custom extension's "classes" folder as ezfsolrdocumentfieldselection.php

 <?php
//
//
// ## BEGIN COPYRIGHT, LICENSE AND WARRANTY NOTICE ##
// SOFTWARE NAME: eZ Find
// SOFTWARE RELEASE: 2.3.0
// COPYRIGHT NOTICE: Copyright (C) 1999-2010 eZ Systems AS
// SOFTWARE LICENSE: GNU General Public License v2.0
// NOTICE: >
//   This program is free software; you can redistribute it and/or
//   modify it under the terms of version 2.0  of the GNU General
//   Public License as published by the Free Software Foundation.
//
//   This program is distributed in the hope that it will be useful,
//   but WITHOUT ANY WARRANTY; without even the implied warranty of
//   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
//   GNU General Public License for more details.
//
//   You should have received a copy of version 2.0 of the GNU General
//   Public License along with this program; if not, write to the Free
//   Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
//   MA 02110-1301, USA.
//
//
// ## END COPYRIGHT, LICENSE AND WARRANTY NOTICE ##
//

/**
 * Based off of the ezfSolrDocumentFieldDummyExample class.
 *
 * This class addresses an issue with indexing the ezselection datatype
 * for multi-select fields.
 *
 * By default, ezselection fields are stored as "text" fields in Solr.  However, this leads to undesired transformation
 * of the values, e.g. "Europe" gets stored in Solr as "europ".
 *
 * A workaround for this is to tell Solr to store ezselection data in "string" fields.  This works fine for single-select
 * ezselection fields.  However, an issue arises with multi-select ezselection fields, namely the values get strung together,
 * e.g. "Europe USA" would get stored for an item with "Europe" and "USA" both selected.
 * This is a result of the implementation of the "metaData" function in kernel/classes/datatypes/ezselection/ezselectiontype.php.
 *
 * This class gets around that problem by passing an array() of values to Solr rather than a space-joined string for multiple selections.
 *
 * ======================
 * To tell eZ Find to use this class:
 *
 * 1)  Place this file in extension/[your_extension]/classes/
 * 2)  Add the following to your ezfind.ini override:
 *        [SolrFieldMapSettings]
 *        CustomMap[ezselection]=ezfSolrDocumentFieldSelection
 *
 * ======================
 *
 * More information:  http://share.ez.no/forums/extensions/ez-find/ezfind-facets-problem-with-ezselection-datatype-indexed-as-text/
 * ======================
 * @author David Sayre, Jordan Hirsch via Beaconfire - http://www.beaconfire.com
**/

class ezfSolrDocumentFieldSelection extends ezfSolrDocumentFieldBase
{
    /**
     * Contains the definition of subattributes for this given datatype.
     * This associative array takes as key the name of the field, and as value
     * the type. The type must be picked amongst the value present as keys in the
     * following array :
     * ezfSolrDocumentFieldName::$FieldTypeMap
     *
     * WARNING : this definition *must* contain the default attribute's one as well.
     *
     * @see ezfSolrDocumentFieldName::$FieldTypeMap
     * @var array
     */
    public static $subattributesDefinition = array( self::DEFAULT_SUBATTRIBUTE => 'string' );
    // public static $subattributesDefinition = array( self::DEFAULT_SUBATTRIBUTE => 'text',
                                                    //self::DEFAULT_SUBATTRIBUTE => 'string' );

    /**
     * The name of the default subattribute. It will be used when
     * this field is requested with no subfield refinement.
     *
     * @see ezfSolrDocumentFieldSelection::$subattributesDefinition
     * @var string
     */
    const DEFAULT_SUBATTRIBUTE = 'data_type_string';

    /**
     * @see ezfSolrDocumentFieldBase::__construct()
     */
    function __construct( eZContentObjectAttribute $attribute )
    {
        parent::__construct( $attribute );
    }

    /**
     * @see ezfSolrDocumentFieldBase::getData()
     */
    public function getData()
    {
        // Extract data from the attribute, and format it as described in the doc link above.
        $data = array();
        $contentClassAttribute = $this->ContentObjectAttribute->attribute( 'contentclass_attribute' ); //get class attribute
        $metaData = array();
        $classAttributeContent = self::getSelectClassContent($contentClassAttribute); //parse selection options into array
        $selectedValue = $idString = explode( '-', $this->ContentObjectAttribute->attribute( 'data_text' ) );

       if ( count( $selectedValue ) > 0)
           {
            $count = 0;
               $optionArray = $classAttributeContent['options'];
               foreach ( $selectedValue as $id )
               {
                   foreach ( $optionArray as $option )
                   {
                       $optionID = $option['id'];
                       if ( $optionID == $id )
                           $metaData[] = $option['name'];
                   }
               }
        }

        //failsafe
        if( !is_array( $metaData ) ) $metaData = array( $metaData );

        $fieldName = self::getFieldName( $contentClassAttribute, self::DEFAULT_SUBATTRIBUTE );
        $data[$fieldName] = $metaData;
        ob_flush();
        return $data;
    }

    /**
     * @see ezfSolrDocumentFieldBase::getFieldName()
     */
    public static function getFieldName( eZContentClassAttribute $classAttribute, $subAttribute = null )
    {
        if ( $subAttribute and
             $subAttribute !== '' and
             array_key_exists( $subAttribute, self::$subattributesDefinition ) and
             $subAttribute != self::DEFAULT_SUBATTRIBUTE )
        {
            // A subattribute was passed
            return parent::generateSubattributeFieldName( $classAttribute,
                                                          $subAttribute,
                                                          self::$subattributesDefinition[$subAttribute] );
        }
        else
        {
            // return the default field name here.
            return parent::generateAttributeFieldName( $classAttribute,
                                                       self::$subattributesDefinition[self::DEFAULT_SUBATTRIBUTE] );
        }
    }

    /**
     * @see ezfSolrDocumentFieldBase::getFieldNameList()
     */
    public static function getFieldNameList( eZContentClassAttribute $classAttribute, $exclusiveTypeFilter = array() )
    {
        // Generate the list of subfield names.
        $subfields = array();

        //   Handle first the default subattribute
        $subattributesDefinition = self::$subattributesDefinition;
        if ( !in_array( $subattributesDefinition[self::DEFAULT_SUBATTRIBUTE], $exclusiveTypeFilter ) )
        {
            $subfields[] = parent::generateAttributeFieldName( $classAttribute, $subattributesDefinition[self::DEFAULT_SUBATTRIBUTE] );
        }
        unset( $subattributesDefinition[self::DEFAULT_SUBATTRIBUTE] );

        //   Then hanlde all other subattributes
        foreach ( $subattributesDefinition as $name => $type )
        {
            if ( empty( $exclusiveTypeFilter ) or !in_array( $type, $exclusiveTypeFilter ) )
            {
                $subfields[] = parent::generateSubattributeFieldName( $classAttribute, $name, $type );
            }
        }
        return $subfields;
    }

     /*!
      * Copy from kernel/datatype/ezselection/ezselection.php
     * Returns the content data for the given content class attribute.
     * //do NOT collide function or variables with existing
    */
    static function getSelectClassContent(eZContentClassAttribute $selectClassAttrib )
    {
        $dom = new DOMDocument( '1.0', 'utf-8' );
        $xmlString = $selectClassAttrib->attribute( 'data_text5' );
        $optionArray = array();
        if ( $xmlString != '' )
        {
            $success = $dom->loadXML( $xmlString );
            if ( $success )
            {
                $options = $dom->getElementsByTagName( 'option' );

                foreach ( $options as $optionNode )
                {
                    $optionArray[] = array( 'id' => $optionNode->getAttribute( 'id' ),
                                            'name' => $optionNode->getAttribute( 'name' ) );
                }
            }
        }

        if ( count( $optionArray ) == 0 )
        {
            $optionArray[] = array( 'id' => 0,
                                    'name' => '' );
        }
        $attrValue = array( 'options' => $optionArray,
                            'is_multiselect' => $selectClassAttrib->attribute( 'data_int1' ) );
        return $attrValue;
    }


    /**
     * @see ezfSolrDocumentFieldBase::getClassAttributeType()
     */
    static function getClassAttributeType( eZContentClassAttribute $classAttribute, $subAttribute = null )
    {
        if ( $subAttribute and
             $subAttribute !== '' and
             array_key_exists( $subAttribute, self::$subattributesDefinition ) )
        {
            // If a subattribute's type is being explicitly requested :
            return self::$subattributesDefinition[$subAttribute];
        }
        else
        {
            // If no subattribute is passed, return the default subattribute's type :
            return self::$subattributesDefinition[self::DEFAULT_SUBATTRIBUTE];
        }
    }
}
?>

Me: http://jordan.teamhirsch.com
My blog: http://wiredformusic.blogspot.com
My other company: http://thinkimprov.com
eZ Certification: http://auth.ez.no/certification/verify/402488
eZ Award: http://ez.no/company/news/ez_awards_2007_prize_winners