Forums / Extensions / Ez Find Update Index Clean taking too much server resources
Julian Mancera
Wednesday 13 January 2010 7:43:26 am
Hi all,
I'm having problem to make a clean indexing of my production site. The updatesearchindexsolr.php --clean script takes too much resuorces of the server. I check the server processes and it seems that java and mysql are draining all memory and processor and as this is my production machine the web server serves the pages really slow. I tried to put a "nice" linux command to see if the process drains less resources but as the script calls mysql and java that have unlimited use of resources as they are used in the web server, is there a way of limiting the resources usage so I can run a clean indexing?
Thank you all for your time and take care
Julian
Paul Borgermans
Wednesday 13 January 2010 9:38:48 am
Hi Julian,
re-indexing indeed needs a lot of cpu/disk resources, mainly on the ez publish side (pulling data and assembling the ezp objects)
You could insert usleep() or sleep() in the update script, but then re-indexing may take too long
Paul
eZ Publish, eZ Find, Solr expert consulting and training http://twitter.com/paulborgermans
Robin Muilwijk
Saturday 16 January 2010 12:30:13 am
Hi Paul,
Out of curiosity, would it help to tune Java as explained on http://ez.no/doc/extensions/ez_find/2_1/basic_configuration/solr_backend ?
And what about updating a staging area with production files and database, running the re-index on staging, and then move the solr index file over to production?
Thanks Robin
Board member, eZ Publish Community Project Board - Member of the share.ez.no team - Key values: Openness and Innovation. LinkedIn: http://nl.linkedin.com/in/robinmuilwijk // Twitter: http://twitter.com/i_robin // Skype: robin.muilwijk
Saturday 16 January 2010 12:38:00 pm
Hi Robin
Tuning helps yes, and also using a staging index helps (you can simply copy over the index files). For more intensive sites with lots of search traffic, a master slave config for Solr may be an even better alternative.
But again, even when re-indexing with a tuned eZ Find/Solr, its still the eZ Publish side that is slow
Saturday 16 January 2010 11:52:52 pm
@Paul, thanks for elaborating on this.
@Julian, hopefully this shows you some directions in improving the performance of re-indexing your site.
-- Robin