Slow site - help/direction troubleshooting?

Next topic

Author	Message
Dave Holowiski	Saturday 30 April 2011 10:02:26 pm I have a site that is running quite slow, and I'm trying to figure out why. It's a quad core 2ghz dell server and at it's peak it's serving about 2 visits per minute (not sure how many requests that calculates out to). We're talking 6-12 second load times. The templates are all extremely customized. I've turned debugging on and noticed a few things: about 10% of the time is spent in "Loading Cache", and 60-70% is spent "processing template". The other values are fairly low, although on a few pages MySQL queries are through the roof 2000+ As far as I can tell caching is turned on and working properly. I'm relativley new to eZ Publish, and not sure where to go from here. Do I start investigating the code in my template? Or do I need a PHP Accelerator? Is this just how eZ publish works? Do I need to put a caching server like squid or nginx in front of my eZ publish server? I have no trouble doing the hard work myself, but I'm looking for some direction - what would you do next in troubleshooting this issue? Thanks in advance for your advice.
André R.	Sunday 01 May 2011 2:14:53 am You should focus on that sql call count, best practice is between 0 and 5 on fully cached pages... So: look for any "fetch" functions within loops look into using cache-block's Be sure there are only 2-3 at the "root" (usually pagelayout), but you can have several nested of course, just make sure the outer most cache-block has cache keys that cover keys on all inner cache blocks as well Make sure to tweak the cache keys on them But first and foremost check that TemplateCache is enabled Look for code that inefficiently use the api, "count( $node.children )" would be one such thing as it causes the system to load all child's of the node and create objects for them before the count is performed, instead of more appropriate count fetch functions or methods that uses a SQL count behind the scenes. Other than that, search for "performance" and "optimization" on these (forum) pages, there are some in-depth threads as well as a few articles that might help you. eZ Online Editor 5: http://projects.ez.no/ezoe \|\| eZJSCore (Ajax): http://projects.ez.no/ezjscore \|\| eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription @: http://twitter.com/andrerom
Marko Žmak	Sunday 01 May 2011 2:38:44 pm Just a few tips... Try to identify which sql queries are slowest and which are repeated a lot of times. Use SQLOutput and SlowQueriesOutput settngs: http://doc.ez.no/eZ-Publish/Technical-manual/4.x/Reference/Configuration-files/site.ini/DatabaseSettings This one is a wild guess but sometimes having wrong permisions on files can degrade the site's performance. Check that the web server user has rw permissions for your cache folders, and read permisions for all templates. Additionally, you could check if the cache files are really generated. Also check if you are doing attribute filtering, this can produce some pretty heavy sql queries. -- Nothing is impossible. Not if you can imagine it! Hubert Farnsworth
Steven E. Bailey	Monday 02 May 2011 1:36:03 am 2 visits per minute? 6-12 seconds of load time? Is that the load time for the first time a page is loaded (and cache is being populated) or for every time the page is being populated? If it is every time, then your cache isn't set up correctly. A php accelerator would probably translate to a 10% gain but with that little amount of traffic it's not going to matter much. A visit every 30 seconds isn't very much traffic at all. I assume you have more than 1GB of RAM on the machine? I assume that the CPU load on the machine is next to nothing? Is this a virtual server and it shares a network card with other, much busier, sites? Is the machine slow to log into (assuming you have command line access)? Do you have DebugTransactions or SlowQueriesOutput enabled? I think the DebugTransactions really slows down a site. Are there specific pages that are really slow? Is the backend slow too or just specific pages on the frontend? If it is just specific pages on the frontend, then it's probably a template problem - if it's not, then it's some problem with the setup (such as the above-mentioned permissions problem) or cache isn't being generated for the templates with the heavy sql load or a debug setting that shouldn't be on. Other places to look is something badly misconfigured in the my.cnf settings. Or whatever database configuration file if you're not using mysql. Check the logging level there too. Certified eZPublish developer http://ez.no/certification/verify/396111 Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com
Dave Holowiski	Monday 02 May 2011 8:29:27 am Steven: That's the load time for the cached version, non cached is much worse. The machine has 4gb of ram, and CPU load is super high... during 'peak time' of 2 visits per minute all 4 cores are running at 100%. It's on a dedicated server, files are being served off a RAID 5 array physically located inside of the server. Our hosting provider checked it out and confirmed the hardware (raid etc) is all healthy. Everything is slow, all pages front and back end. Some pages are slower than others but the home page is the worst. TemplateCache is enabled DebugTransactions is disabled SlowQueriesOutput is set to 0 DebugOutput is Enabled, but only for certian ID's - could this be a problem? I'm leaning towards a template issue? The non-cached version of the homepage makes 780 mysql queries and the cached version makes 17. I didn't write any of the code, but I'm thinking I'm going to have to go through it and clean it up...
Marko Žmak	Monday 02 May 2011 9:47:37 am It sounds like it could be one of this two issues: bad template problem (a lot of looping or poor caching) some problem with reading cache or template files (like the permission problem) But if you say that the backend is also slow, then it's more probable that the second one is the case. Did you check the permissions? Also check with your admin if he can detect if there's an unusually high occurence of stat() calls on the server. And another idea, do you have Stale Cache enabled? There are some situations where using stale cache can produce severe performance degradation (I had experience with such a situation). And two more toughts: where does the 100% of the CPU times go? Is it in IO wait or in CPU processing? If it's IO wait, that probably means a lot of reading/writing on the disk. which process consumes the most CPU? Is it httpd or mysqld? -- Nothing is impossible. Not if you can imagine it! Hubert Farnsworth
Steven E. Bailey	Monday 02 May 2011 2:19:53 pm 4GB of ram and 100% for all cores at 2 request a minute - something is way out of wack. A 2 GB 2 core machine should be handling 2 requests a second without hitting a CPU of over .999. O.k. maybe 1.999. 780 mysql requests for one page is a lot... but 17 is nothing - so, with that machine and cache enabled you should be getting response times (as long as everything is set up right) of between .7 and 1.2 seconds - in my experience. Your uncached page could take something like 20-40 seconds to load... The question is why isn't this happening. Marko has good points. If you figure out what is using up all that CPU then you're closer to figuring out what is going on. "Everything is slow, all pages front and back end" - what about ssh? Do you have ssh access? Because if that's slow then you probably have a bad network card - or it's whatever else is causing the high CPU load. When I've had badly cached or badly set up sites, normally the backend is still pretty fast... since normally they aren't heavily customized. If they are the stock admin templates, then most likely the problems that you are having are not (really) related to your templates. Probably they aren't related to ezpublish at all. Certified eZPublish developer http://ez.no/certification/verify/396111 Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com
Steven E. Bailey	Monday 02 May 2011 2:24:54 pm Also, do you have this on a development server? Does it run o.k. on the development server? If you don't have a development server then can you set it up on another machine to test if it works there? If you don't have this capability then contact me offline. Also, never trust what an ISP says... they may be right but you are one of 100s or 1000s of customers... so, they really (in my experience) don't really care about you. Certified eZPublish developer http://ez.no/certification/verify/396111 Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com