Dedicated server crashed with few sites

Author Message

John Smith

Monday 04 January 2010 8:39:48 am

Hi guys,

Hope someone can help me. We got few sites (around 10) running on a dedicated server with the following specification and it crashed on the day when they were needed. At that time there were around 800 users and around 22,195 hits.

Intel(R) Core(TM)2 Duo CPU E4600 @ 2.40GHz

Speed: 2394.103

Cache Size: 2048 KB

Total Memory 2064780kB

Total Swap 497944 kB

Is there anyway to check whether it is something wrong with ezPublish or the server. I have already checked the debug for all sites there are no warnings or errors? I am using ezPublish 4.0.1, Please help.

Pål Messenlien

Monday 04 January 2010 1:31:37 pm

I have seen the same happend with just 1 eZ site. We released a portal for registrations. I noticed something was wrong the minute i released the site. As soon the users started logging in and registering the server went nuts. It all crashed within an hour. We fixed this with some code changes on cache blocks etc. But the main fix was adding a php accelerator and tuning both mysql and php. After a couple hours we had the site back up and running again. I would first look into the php accelerator for your server. We use APC. Also check if php got enough memory, and check for tuning on mysql. If possible move mysql to another server, it will take some of the load off. Logged in users doing lots of stuff in eZ seems to be very power hungry. There should be lots of posts here in the forums about tuning php and servers.

These are what i found that might be usefull:

Other things you should consider:

  • Upgrading eZ to a later version. 4.2 seems to run allot easier than 4.0.1 was. (just upgraded a large site)
  • Adding Varnish to the sites

Hope some of this help.

--------------------------------------------
Høgskolen i Lillehammer
Lillehammer University College
http://www.hil.no
--------------------------------------------
Messenlien IT
http://messenlien.com
-------------------------------------------

John Smith

Tuesday 05 January 2010 9:44:44 am

Thanks for you kind feedback.

Tuning MySQL means allocation of memory towards database. I did try few settings (max connections, key buffer, table cache) as discussed in the article this afternoon, but now apache web server started hanging.

Still stuck quite frustrating...I know the setting would be different for different server specifications but is there any general set of check points which needs to be checked before releasing the site to the public so that it would not crash the server. Any help would be appreciated.

Jérôme Vieilledent

Wednesday 06 January 2010 2:13:02 pm

Hi John

What about Apache max_clients ? Try to tune it (here's a tiny tutorial ;)).

Can you be more precise on your server configuration please ? PHP/Apache/MySQL/eZ Publish versions, PHP configuration (memory_size, APC config if you have it...). Is Apache in prefork mode ? Is PHP compiled as a module ? Do you run your server on Linux (which distro) ?

Also, you should try to install monitoring tools such as Cacti or Nagios.

You can monitor your eZ Publish websites with jvMonitoring (simple, you can develop new handlers) or eZSNMPd.

First things to check for performance :

  • Cache configuration (is view cache activated, did you disable INI File mtime check in config.php, are your cache blocks in pagelayout efficient, ...)
  • Opcode cache such as APC (highly recommended, with 128MB at least)
  • A reverse proxy cache eventually, such as Varnish

Hope this helps :)

John Smith

Friday 08 January 2010 12:51:47 am

Cheers mate,

I will revert back with all the question you asked for. By the way checking few first things as suggested by you:

View cached is activated with the following settings. Please have a look.

[ContentSettings]
ViewCaching=enabled
PreViewCache=enabled

[TemplateSettings]
TemplateCache=enabled
TemplateCompile=enabled
TemplateOptimization=enabled

I did not get the point regarding INI file mtime check in config.php, Please explain.

Cache blocks seems fine in pagelayout.tpl. Is there any tutorial where I can get full information about Cache Blocks.

Cheers

Jérôme Vieilledent

Friday 08 January 2010 1:35:30 am

Hi John

OK everything seems fine with your cache config. Also check if DevelopmentMode is disabled ([TemplateSettings]).

About config.php stuff, it's from eZ Publish 4.1. You can consider upgrading to 4.2 (huge performance improvements) ;)

John Smith

Friday 08 January 2010 9:21:16 am

cheers,

It would be favour if you can suggest me how to cache the following.

I have written some code to display the latest events and random images from gallery on home page. I have to use the following:

{set-block scope=root variable=cache_ttl}0{/set-block}

The reason I am using set-block is to exclude it from cache. If I do not write it then the system would not update events when added from the administration interface. But it also increases the number of mysql queries and load on the server.

What is the best way to do it?

Gaetano Giunta

Friday 08 January 2010 9:44:20 am

A couple of hints:

- mysql queries might be killing your app. You can find out how many queries are executed per-page by enabling the debug output (it can be done without disabling caches) and looking at the bottom of the page for the table line that reads "Mysql_queries" (or Mysqli_queries). Anything above 100 is plain suicide. Below 10, you'r gold.

- disabling view cache on hp is a bad idea, really.

You can set it to eg 10 minutes if you want latest news appearing, just not in real time:

<span>{</span><a href="http://doc.ez.no/set-block" mce_href="http://doc.ez.no/set-block" target="ez_no_documentation"><span>set-block</span></a> <span>scope</span><span>=</span>root <span>variable</span><span>=</span>cache_ttl<span>}36</span>0<span>{</span><span>/</span><a href="http://doc.ez.no/set-block" mce_href="http://doc.ez.no/set-block" target="ez_no_documentation"><span>set-block</span></a><span>}</span>

You can use a cache block where you put in the "keys" param a random value calculated between eg 1 and 10 if what you want is some random pictures - you will have the cache block executed about 10 times before it is fully warmed, and successive hits will not execute it anymore

If your news do not update very often, even better than a periodic expiry is one based on actual linking of content editing action with cached expiration. look up the docs about the "smart view cache" for that

- a lot of developers always fetch the current node in the pagelayout, because it is not there and they need just a bit of info from it. Do not do it, use the persistent_variable instead (look it up in the forums/docs)

- make sure your cache blocks are properly tuned: if they do not depend on content expiration, use "ignore_content_expiry", if they depend on expiration on a particular node, set it in "subtree_expiry", and do not forget plain "expiry" timeout (http://ez.no/doc/ez_publish/technical_manual/4_x/reference/template_functions/miscellaneous/cache_block)

Principal Consultant International Business
Member of the Community Project Board

John Smith

Friday 15 January 2010 1:15:28 am

Hi Gaetano,

Thanks for your suggestions, I tried all the points you suggested and it has reduce the load dramatically.

One thing which needs to be confirmed with you. I did link content editing action with cached expiration to display latest events on the home page. It worked every well with smart view cache enabled, which means if somebody add/edit event - the home page cache get cleared and updates the list.

If the content editing action is not triggered means nobody add or edit any event, then it does not update the list of events automatically on homepage which has already held yesterday or before as I am display all latest event today onwards. Is there any trick to do that as well.

Cheers,

Gaetano Giunta

Friday 15 January 2010 1:42:42 am

Sorry, I do not understand fully your request.

If you want the homepage expired every day event if no editing action has been happening, you can set up a cronjob that expires its cache. The php call to expire the cache of an object is:

 eZContentCacheManager::clearContentCache( $contentObject->attribute('id') );

Principal Consultant International Business
Member of the Community Project Board

John Smith

Friday 15 January 2010 2:03:25 am

Yes, I want the homepage cache to be expired everyday in the event if no editing action has been happening. Is it possible to explain in more detail how to implement and setup as cronjob.

Much appreciated.

Cheers,

John Smith

Tuesday 19 January 2010 7:06:58 am

Anyone ??

Nicolas Pastorino

Monday 01 February 2010 8:39:44 am

"

Anyone ??

"

Hi John,

Late answer on this, hopefully helpful anyways. Check these concise resources in order to understand how to setup a custom cronjob:

  • http://ez.no/doc/ez_publish/technical_manual/4_x/features/cronjobs
  • http://ez.no/doc/ez_publish/technical_manual/4_x/features/cronjobs/the_cronjob_scripts
  • http://ez.no/doc/ez_publish/technical_manual/4_x/features/cronjobs/configuring_cronjobs
  • http://ez.no/doc/ez_publish/technical_manual/4_x/features/cronjobs/running_cronjobs

The code contained in the cronjob you apparently want to setup would be the line Gaetano gave :

eZContentCacheManager::clearContentCache( $homePageObjectId );

where $homePageObjectId can be fetched from an INI file.

How is your website going now ?
Do you have uptodate stats (pv, hits, unique visitors, etc) ?
Cheers,

--
Nicolas Pastorino
Director Community - eZ
Member of the Community Project Board

eZ Publish Community on twitter: http://twitter.com/ezcommunity

t : http://twitter.com/jeanvoye
G+ : http://plus.tl/jeanvoye

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.