Ezfind/solr optimization

Next topic

Author	Message
Ali Nebi	Wednesday 11 March 2009 8:32:47 am Hi, we have an issue with high cpu usage from java using ezfind with ez 4. The system is with this hardware: 2 x Quad-Core AMD Opteron™ 2352 2 x 4 x 2,1 GHz 16GB RAM RAID 5 OS: Centos 5.2 java version "1.5.0_13" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_13-b05) Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_13-b05, mixed mode) On of the servers we have set java options for solr with these parameters: -Dezfind -Xms768M -Xmx768M On this server we experience difficulties with high cpu usage from java. There is enough memory and server does not swap. On another server with this hardware: Intel(R) Xeon(R) CPU 5150 2 x 2.66GHz 4GB RAM OS: Centos 5.2. java version "1.6.0_06" Java(TM) SE Runtime Environment (build 1.6.0_06-b02) Java HotSpot(TM) 64-Bit Server VM (build 10.0-b22, mixed mode) Java settings for solr on this server are following: -Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC -XX:PermSize=128m -XX:MaxPermSize=128m We experience the following difficulties on this server: 1. When we leave java options these: -Dezfind -Xms512M -Xmx512M, then after some hours/days work, solr stop to work. Java process is available, but search on the site does not work. We get Http code 500 in logs. When i try to connect to solr administration i get following message: GC overhead limit exceeded RequestURI=/solr/admin Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 2. When i set java options this way: -Dezfind -Xms512M -Xmx512M -XX:+UseConcMarkSweepGC Then solr works fine for some days and then it stop to work again. Java process is still available. Search does not work on the site. In the log we get HTTP Code 500. When i try to login to solr administration i get following: java.Lan.OutOfMemoryError: Java heap space 3. I tried to solve the problem "java.Lan.OutOfMemoryError: Java heap space" with putting additional settings "-XX:PermSize=128m -XX:MaxPermSize=128m", but it didn't help so much. I still experience this problem. The question is how to optimize solr/ezfind to decrease cpu usage and to solve OutOfMemory problems? Thanks in advanced! Iguana Information Technologies, SL - http://www.iguanait.com
Gaetano Giunta	Thursday 12 March 2009 3:51:10 am Not sure it will help, but you might try switching to a 32 bit version of Java. IIRC the 64 bit version was not the best choice on redhat os... Principal Consultant International Business Member of the Community Project Board
Ali Nebi	Saturday 14 March 2009 2:16:22 am Thanks for the reply. I will be happy if someone post other comments too, related with ezsolr/ezfind optimization. Regards, Ali Nebi! Iguana Information Technologies, SL - http://www.iguanait.com
Paul Borgermans	Monday 16 March 2009 2:41:43 pm It is probably the way to start the backend (solr), so try the alternate startup parameters below If you don't have the 64 bit JVM (and OS) java -server -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar Otherwise (enable the 64 bit datamodel) java -server -d64 -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar The Xm.... parameters are memory allocation, if you have a lot and large indexes (many objects, many files), then you may even try to increase that Can you tell me if that helped? hth Paul eZ Publish, eZ Find, Solr expert consulting and training http://twitter.com/paulborgermans
Ali Nebi	Sunday 22 March 2009 3:11:55 pm Hi, Thanks for the reply and trying to help me to solve this problem. I still expirience the problem on the second server where solr stop to work after some time. I have set there : java -server -d64 -Xmx600m -Xms600m -XX:+UseParallelGC -XX:+AggressiveOpts -XX:NewRatio=5 -jar start.jar When i see that search is not working and try to login to the admin page of solr i get connection refused message. All requests to the solr are with code HTTP 1.1 500 in the logs during this time. It is probably because the memory, but if i increase the memory then server start to swap more than usual. Where can i enable the debug mode if necessary? This way i can provide more info for the problem probably. Iguana Information Technologies, SL - http://www.iguanait.com
Gaetano Giunta	Thursday 28 July 2011 11:29:20 pm Two more bits of insight: 1. Java 64bit is now the recommended version - problems with the 64 bit version only applied to old versions of rhel (circa 2007) 2. according to the solr wiki (http://lucene.apache.org/solr/ ), using -XX:+AggressiveOpts is dangerous with a version 6 jre Principal Consultant International Business Member of the Community Project Board