Ezfind/solr optimization

Author Message

Ali Nebi

Wednesday 11 March 2009 8:32:47 am

Hi,

we have an issue with high cpu usage from java using ezfind with ez 4.

The system is with this hardware:

2 x Quad-Core AMD Opteron™ 2352
2 x 4 x 2,1 GHz
16GB RAM
RAID 5
OS: Centos 5.2

java version "1.5.0_13"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_13-b05, mixed mode)

On of the servers we have set java options for solr with these parameters:

-Dezfind -Xms768M -Xmx768M

On this server we experience difficulties with high cpu usage from java. There is enough memory and server does not swap.

On another server with this hardware:
Intel(R) Xeon(R) CPU 5150
2 x 2.66GHz
4GB RAM
OS: Centos 5.2.
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b22, mixed mode)

Java settings for solr on this server are following:

-Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC -XX:PermSize=128m -XX:MaxPermSize=128m

We experience the following difficulties on this server:

1. When we leave java options these: -Dezfind -Xms512M -Xmx512M, then after some hours/days work, solr stop to work. Java process is available, but search on the site does not work. We get Http code 500 in logs. When i try to connect to solr administration i get following message:

GC overhead limit exceeded
RequestURI=/solr/admin
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

2. When i set java options this way:
-Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC
Then solr works fine for some days and then it stop to work again. Java process is still available. Search does not work on the site. In the log we get HTTP Code 500. When i try to login to solr administration i get following:

java.Lan.OutOfMemoryError: Java heap space

3. I tried to solve the problem "java.Lan.OutOfMemoryError: Java heap space" with putting additional settings "-XX:PermSize=128m -XX:MaxPermSize=128m", but it didn't help so much. I still experience this problem.

The question is how to optimize solr/ezfind to decrease cpu usage and to solve OutOfMemory problems?

Thanks in advanced!

Iguana Information Technologies, SL - http://www.iguanait.com

Gaetano Giunta

Thursday 12 March 2009 3:51:10 am

Not sure it will help, but you might try switching to a 32 bit version of Java. IIRC the 64 bit version was not the best choice on redhat os...

Principal Consultant International Business
Member of the Community Project Board

Ali Nebi

Saturday 14 March 2009 2:16:22 am

Thanks for the reply.

I will be happy if someone post other comments too, related with ezsolr/ezfind optimization.

Regards, Ali Nebi!

Iguana Information Technologies, SL - http://www.iguanait.com

Paul Borgermans

Monday 16 March 2009 2:41:43 pm

It is probably the way to start the backend (solr), so try the alternate startup parameters below

If you don't have the 64 bit JVM (and OS)

java -server -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar

Otherwise (enable the 64 bit datamodel)

java -server -d64 -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar

The Xm.... parameters are memory allocation, if you have a lot and large indexes (many objects, many files), then you may even try to increase that

Can you tell me if that helped?

hth
Paul

eZ Publish, eZ Find, Solr expert consulting and training
http://twitter.com/paulborgermans

Ali Nebi

Sunday 22 March 2009 3:11:55 pm

Hi,

Thanks for the reply and trying to help me to solve this problem.

I still expirience the problem on the second server where solr stop to work after some time. I have set there :

java -server -d64 -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar

When i see that search is not working and try to login to the admin page of solr i get connection refused message. All requests to the solr are with code HTTP 1.1 500 in the logs during this time.

It is probably because the memory, but if i increase the memory then server start to swap more than usual.

Where can i enable the debug mode if necessary? This way i can provide more info for the problem probably.

Iguana Information Technologies, SL - http://www.iguanait.com

Gaetano Giunta

Thursday 28 July 2011 11:29:20 pm

Two more bits of insight:

1. Java 64bit is now the recommended version - problems with the 64 bit version only applied to old versions of rhel (circa 2007)

2. according to the solr wiki (http://lucene.apache.org/solr/ ), using -XX:+AggressiveOpts is dangerous with a version 6 jre

Principal Consultant International Business
Member of the Community Project Board

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.