Content Import Questions

Tuesday 20 January 2004 11:53:04 am

After spending a lot of time combing through the forums, documentation and contributions I have managed to thoroughly confuse myself as to the best way to importat data. So, I hope some of you can help me determine the best path to follow as I work on a new site.

[The Challenge]
I need to import data from a text file (I'm not sure if it will be comma or tab delimited as of yet) into eZ publish 3.3. This data will be updated at least once every 24 hours. This is a brand new site, so I do not have to worry about legacy data when importing for the first time.

[My Questions]
1. Does anyone have a good foundation script written to import text into eZ publish? I will have quite a few fields to import per entry (item name, item number, item comments, item price, various specifications). The vast majority of imported fields will be simple text lines, though I will need one text field. I should not need XML Text field import, though I am quite happy to use it, if that is what is available. I do not need to worry about binary data.

2. What pitfalls do I need to be prepared to avoid?

3. The information at http://ez.no/developer/ez_publish_3/documentation/development/importing_attribute_data looks promising, but I am unable to download the the testimport.php script that is linked on that page. Does anyone have that script available so I could look at it?

As always I appreciate your help!

Alex

Alex
[ bald_technologist on the IRC channel (irc.freenode.net): #eZpublish ]

<i>When in doubt, clear the cache.</i>

Tuesday 20 January 2004 3:03:13 pm

Hi Alex

I've worked on a number of sites that have been done using eZ publish and utilise external data. I've come to the conclusion that unless the data is going to be managed via eZ publish (i.e. you are doing an initial import and the data will be edited) it is much more efficient to store the data in a its own table(s) and to write a module to display it.

Cheers
Bruce
http://www.designit.com.au

My Blog: http://www.stuffandcontent.com/
Follow me on twitter: http://twitter.com/brucemorrison
Consolidated eZ Publish Feed : http://friendfeed.com/rooms/ez-publish

Wednesday 21 January 2004 3:21:12 am

Hi Alex,

I've had the nightmare job of consolodating and importing data from multiple mysql database tables along with various filemaker databases into ez. Previously all the data was syncd manually, (which means that there was data inconsistancy which is whole other problem altogether)

I havn't done anything with regards to text files though. Although my code gets all the data into a consistant associative array format:

$array[*ezcontentclassidentifer*][*value*]

where ezcontentclassidentifier is the string id (not the numerical id) of the content object attribute.

So working from there it takes care of pretty much everything.

Your first stage would be to get the text file data into an associatve array format (which will be a piece of cake). Then you should create ez objects from there.

Once I have my code finished (it still has many hickups) I'll release it.

The interesting problem I have had to face is keeping ez publish and filemaker synchronized. An update in filemaker must appear on the website an update on the website must appear in filemaker. Its messy I know but filemaker is entrenched in their system and moving from it would have been impossible. I keep them syncd by:

ez -> filemaker: workflow publish events
filemaker -> ez: cronjob run every x minutes.
Comparing versions, timestamps and values.

Could you tell us more on the nature of your problem. Is the text file simply feeding ez, or will ez also feed the text file?

IE is data *appended* to the text file (so that you'll have objects that need to be *updated* and also you'll have to create *new* objects each time the script is run) or is the data file being overwritten each time so that the only task of the script will be to *create* new ez objects and publish them?

willie dot seabrook at levity dot co dot nz

Monday 26 January 2004 3:12:37 am

Hi!

I have done something similar to what you're after. Data are created and updated in an internal product database and exported to a text file, which is then imported into eZ. Products are never created in eZ.

The import script has to check whether the product exist or not, and create or update it. I used the examples in contributions as a base (Import 2.2 to 3.x), and I used the remote_id field in ezcontentobject as an identifier. I made the remote_id out of class name and ID from the product database, just as it was done in the 2.2 -> 3.x example.

The greatest challenge was to find the correct parent node to create the product under. The category tree was created manually in eZ, and I made a map table with categories from the product database and node numbers from eZ. Not an elegant solution, but it was the only way for me. You may use eZContentObjectTreeNode::childrenByName instead if you have a simple category tree, e.g:

$nodeFolder =& eZContentObjectTreeNode::fetch( $productFolderId );
$existsNode =& $nodeFolder->childrenByName( $category );
if ( count( $existsNode ) > 0 ) {
$parentNodeID = $existsNode[0]->attribute( 'main_node_id' );
}

But then you also have to create the category node it it don't exists.

This import system has been in use for 4 months now, and working great. Take a look at http://www.damm.no/content/view/full/10896 for an example of the result (in Norwegian).

Roy

Monday 26 January 2004 6:57:52 am

Hi Alex,

For XML data import you might want to check out the text2xml class at http://www.ez.no/ez_publish/documentation/development/importing_attribute_data if you haven't already. It worked like a charm for me.

Sincerely,

Eirik Johansen

Sincerely,

Eirik Alfstad Johansen
http://www.netmaking.no/

Tuesday 01 June 2004 6:31:14 am

Alex,

Have you had any joy on this? I have started to look into the issue too, see post at:

http://ez.no/community/forum/setup_design/import_product_data_from_xl_spreadsheet_customisation_of_administration_interface

Seems quite a tricky conundrum...

Steve.

http://www.oneworldmarket.co.uk

Tuesday 01 June 2004 7:13:37 am

Alex,

Thanks for the reply.

Currently I'm trying to just assess the feasability so I can create a quote for a potential client. The answer seems to be 'Yes', but its clearly going to be a bit of a learning curve for me to implement.

Have a good trip!

Steve.

http://www.oneworldmarket.co.uk

Wednesday 09 June 2004 7:54:13 am

Alex,

Cheers for the note. I have built in a good amount of time to develope this and am waiting to hear back from the client. Fingers crossed as it would be an interesting project to build out.

Steve.

http://www.oneworldmarket.co.uk