Forums / Developer / 1000000 (one milion) versioned multi-located objects
Piotrek Karaś
Thursday 23 October 2008 7:05:56 am
Hi,
Has anyone got and would be willing to share (on public or privately) experiences of using eZ Publish for managing around one milion (1000000) of content objects with several object versions and on average 20-50 object locations?
I'm interested in any aspects of such implementation: hardware, practical problems (if any), and so on.
Or maybe there are some known case studies available for such projects?
Much appreciated,Piotrek
-- Company: mediaSELF Sp. z o.o., http://www.mediaself.pl eZ references: http://ez.no/partners/worldwide_partners/mediaself eZ certified developer: http://ez.no/certification/verify/272585 eZ blog: http://ez.ryba.eu
Ivo Lukac
Thursday 23 October 2008 8:45:49 am
Hello Piotrek,
Did some testing with 300000 objects with several versions each placed randomly in 20 folders. Hardware was nothing special, just development machine. Main purpose was to see how would such implementation work.
There was one big issue which was slowing done publishing process (publish time was increasing as total number of objects). The problem was in view cache clearing keyword related objects because I had keywords in all objects (eZContentCacheManager::appendKeywordNodeIDs() is very slow).
There is no setting option to disable this so kernel hack is the saver :)
http://www.linkedin.com/in/ivolukac http://www.netgen.hr/eng/blog http://twitter.com/ilukac
André R.
Thursday 23 October 2008 9:40:58 am
>There is no setting option to disable this so kernel hack is the saver :)
No but there is a patch that are looking for feedback:http://issues.ez.no/IssueView.php?Id=13429&activeItem=4
Adds setting to limit keyword relations to clear view cache for, and setting to let you have total control over this with viewcache.ini rules(instead of just adding rules to the hardcoded defaults).
eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription @: http://twitter.com/andrerom
Thursday 23 October 2008 10:16:12 am
Great :)
When will it be in trunk and not like patch?
Thursday 23 October 2008 11:01:04 am
Where there is some feedback on it =)For instance should the keyword limit be set by default to for example 500? or 100? or..
Thursday 23 October 2008 11:42:00 pm
I think the most important is that you can disable this because you don't need it always. For example if you have keywords and show them in full view linking to content/keyword/something you don't need this. You need this only when you have listed keyword related objects.
Right?
Friday 24 October 2008 12:47:58 am
yes. Witch reminds me that there should be possible to control this by class :)
Tuesday 18 November 2008 8:45:06 pm
Thanks for your feedback, Ivo, and sorry for the delay.
I'm justing finishing a script for running similar tests in a flexible manner, I'll share that as soon as it's ready. Unfortunately, I will also need to buy a PC for running those tests to reach numbers we're talking about before christmas ;)
I will use a real-life thesaurus to feed objects with content, and will definitely include XMLBlock and Keywords datatypes testing, as well.
I will get back on this issue.