|
Andre Felipe Machado
|
Monday 31 October 2005 2:41:37 am
Hello,
The runcronjob is depleting resources at my machine.
It exahusts ram and maximum open files limits.
The site syndicates some rss external feeds and then links are included. So, the site "grows".
At the next cronjob, the additional links and pages are checked and memory is filled, maximum open files are reached, site crashes with Ezpublish fatal error or error 403, forbidden access(!), database acess fails (unacessible mysql table files).
Is there a way to optimize the cronjobs?
Today, its code imposes a practical limit to the site size, number of pages and number of links.
What can I do?
Regards.
Andre Felipe
---
A Debian user never dies. Issues a last command:
shutdown -h now
http://www.techforce.com.br
|
|
Andre Felipe Machado
|
Wednesday 02 November 2005 7:30:20 am
Hello
Found the culprit and an ugly workaround.
The cronjob does not scale to bigger sites.
It starts linkchecking and this process is slow, because each link must connect or time out.
At my site, each instance of linkcheck use almost 6 hours to complete.
As each instance is already a resource hog on its own, 6 instances running simultaneausly exhausts system resources.
I had to reconfigure cronjob from hourly to at each 6 hours...
The rss export import are now very bad. Workflow....
A more reasonable solution could be using independent link check cron job from the other tasks.
The link check could run once a day.
Other solution could be check many links in parallel mode.
Regards.
Andre Felipe
---
A Debian user never dies. Issues a last command:
shutdown -h now
http://www.techforce.com.br
|
