Ezfind/solr optimization

Wednesday 11 March 2009 8:32:47 am

Hi,

we have an issue with high cpu usage from java using ezfind with ez 4.

The system is with this hardware:

2 x Quad-Core AMD Opteron™ 2352
2 x 4 x 2,1 GHz
16GB RAM
RAID 5
OS: Centos 5.2

java version "1.5.0_13"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_13-b05, mixed mode)

On of the servers we have set java options for solr with these parameters:

-Dezfind -Xms768M -Xmx768M

On this server we experience difficulties with high cpu usage from java. There is enough memory and server does not swap.

On another server with this hardware:
Intel(R) Xeon(R) CPU 5150
2 x 2.66GHz
4GB RAM
OS: Centos 5.2.
java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b22, mixed mode)

Java settings for solr on this server are following:

-Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC -XX:PermSize=128m -XX:MaxPermSize=128m

We experience the following difficulties on this server:

1. When we leave java options these: -Dezfind -Xms512M -Xmx512M, then after some hours/days work, solr stop to work. Java process is available, but search on the site does not work. We get Http code 500 in logs. When i try to connect to solr administration i get following message:

GC overhead limit exceeded
RequestURI=/solr/admin
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded

2. When i set java options this way:
-Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC
Then solr works fine for some days and then it stop to work again. Java process is still available. Search does not work on the site. In the log we get HTTP Code 500. When i try to login to solr administration i get following:

java.Lan.OutOfMemoryError: Java heap space

3. I tried to solve the problem "java.Lan.OutOfMemoryError: Java heap space" with putting additional settings "-XX:PermSize=128m -XX:MaxPermSize=128m", but it didn't help so much. I still experience this problem.

The question is how to optimize solr/ezfind to decrease cpu usage and to solve OutOfMemory problems?

Thanks in advanced!

Iguana Information Technologies, SL - http://www.iguanait.com

Saturday 14 March 2009 2:16:22 am

Thanks for the reply.

I will be happy if someone post other comments too, related with ezsolr/ezfind optimization.

Regards, Ali Nebi!

Iguana Information Technologies, SL - http://www.iguanait.com

Sunday 22 March 2009 3:11:55 pm

Hi,

Thanks for the reply and trying to help me to solve this problem.

I still expirience the problem on the second server where solr stop to work after some time. I have set there :

java -server -d64 -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar

When i see that search is not working and try to login to the admin page of solr i get connection refused message. All requests to the solr are with code HTTP 1.1 500 in the logs during this time.

It is probably because the memory, but if i increase the memory then server start to swap more than usual.

Where can i enable the debug mode if necessary? This way i can provide more info for the problem probably.

Iguana Information Technologies, SL - http://www.iguanait.com