Big big fecth..??

Author Message

Jorge estévez

Saturday 24 October 2009 8:45:39 pm

I have to allow my client to select a node that has many products, once selected I must display a list of all the products within and some of its attributes.

The problem arises when this kind of “fetch” has to deal with so many nodes that the memory gets compromised (I have whole control of apache at the server , but I would like to keep memory used for running scripts not to big) .

I need to know how to deal with a really big fetch even if it takes some seconds or even minutes but in some way I can free memory withing the fetching of groups of nodes or else.

Any Ideas?

Diseño Web Cuba
Web Design Cuba
www.elfosdesign.com

Gabriel Finkelstein

Monday 26 October 2009 10:43:17 am

If you're displaying all items on the same page, you should page them.

Piotrek Karaś

Monday 26 October 2009 11:28:51 pm

Can you illustrate your example with some numbers? How many nodes do you have to display? Also, won't caching layer take care of the problem (generate once and server from cache later on)?

--
Company: mediaSELF Sp. z o.o., http://www.mediaself.pl
eZ references: http://ez.no/partners/worldwide_partners/mediaself
eZ certified developer: http://ez.no/certification/verify/272585
eZ blog: http://ez.ryba.eu

André R.

Tuesday 27 October 2009 6:53:19 am

I'll add another: Why?

But ignoring the fact that this sounds like a big design mistake, here is how it can be done:

A. Fetch some of the items at once (50 or 100 nodes), if your on 4.1 and lower then make sure you add "load_data_map, false()" parameter (was added in 3.10). And create a template operator that clears object + data_map + attribute cache (search for "cronjob memory" or something to find out how you do that).

B. Same as above, except fetch all nodes at once instead of some using "as_object, false()" parameter, and create a template operator that creates the objects using "eZContentObject::fillNodeListAttributes( $listOfNodes );".

Page will probably be slower, but you'll avoid eating memory.

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Jorge estévez

Wednesday 28 October 2009 4:46:53 am

Hello,

My client, at different offices around the world is populating the site with products, lots of them, at the head office there is a need to know the list of codes of products that are already inserted in a folder-node (and its folder-childs) to be able to compare with the products that they have in their database, reasonable need indeed).

Taking into consideration usability I built a tree of nodes (similar to the ez admin page) that has checkmarks (all nodes with its corresponding checkmark), So the head office admin clicks the checkmark and a button that makes a call to an url using jAVASCRIPT like this one

http://www.thesite.com/index.php/esl/content/view/getdupcode/181

the 181 is added to the url based on the selection of the node in the tree, and the getupcode is a view mode that get's all the product nodes (within the node's tree) and displays them.

Getting all the products takes too much time and memory, thousands of nodes (and still more to come) lies beneath different branches of the selected node.

I tried to page the output as to display 200 products codes from page to page, by doing so

http://www.thesite.com/index.php/esl/content/view/getdupcode/181/(offset)/50

by some reason the pages goes to it's previous state (before selecting and clicking the button) that is

http://www.thesite.com/index.php/esl/content/view/full/181

And displays nothing (not even the information that should be there before the selection and click to the button)

Thanks.

Diseño Web Cuba
Web Design Cuba
www.elfosdesign.com

Romain Petit

Monday 02 November 2009 2:07:30 pm

Hi,

If you are already using javascript to load your content you can fetch nodes one after another with several consecutive ajax calls, and insert them in your page ( or 10 by 10... ).

Is the memory limit reached ? If so, is that during the fetch or during the template processing ?

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.