Importing and Exporting Content Objects in eZ

Wednesday 31 March 2010 11:38:47 pm

Hi

I have the same recurrent problem... Packages import/export often fail with content objects.

Thursday 01 April 2010 1:31:57 am

Hello,

This is a really painful (so really interesting) topic here.

Exporting/Importing ezpackages of content objects is not reliable, we often experience issues with object relations or translations loss.

On this topic, we could also add the import of content classes definitions modifications (at least attributes addition) which is impossible and requires human actions through the web UI.

These issues make really difficult the process of developing an eZ Publish application and applying changes on different staging environments.

On our main project, we have to deal with dev / user acceptance tests / preproduction / production environments which are managed by different teams which are not all trained to eZ Publish.

Without such reliable tools, eZ Publish can not be considered as a fully corporate-friendly solution.

We did not have time to test extensively the different extensions available yet as we could not identify an "all-in-one" solution without having to plan a full project only fot this "basic" (at least for corporate management) subject.

Thanks a lot for your experiences on this topic !

Twitter: @kevingaudin

Thursday 01 April 2010 2:00:02 am

Well, eZXMLExport exports in XML and data_import has an XML handler so that should work without any problem

:)

Thursday 01 April 2010 2:13:54 am

Well there is an even simpler solution.

eZXMLExport allows you to use an XSLT file which is used during the export process, so with each XML file created you can apply a smal and specific XSLT that will be used to generate the final result, if you already have a ready to go XML handler in data_import that would be an interesting solution :)

Thursday 01 April 2010 2:25:20 am

The good point though is that one can version the content class definitions in any version control system (SVN/CVS/git..).

"The best would be a possibility to export a content class to a xml understandable by ezxmlinstaller."
Yes, that would help. Alternatively, making sure ezxmlinstaller understand an ezpkg could do the deal.

Cheers,

--
Nicolas Pastorino
Director Community - eZ
Member of the Community Project Board

eZ Publish Community on twitter: http://twitter.com/ezcommunity

t : http://twitter.com/jeanvoye
G+ : http://plus.tl/jeanvoye

Thursday 01 April 2010 4:10:41 am

Actually, ezcreateclass handler does handle class modification (see handler code).

@ Matthieu : eZXMLInstaller provides an export module for classes and roles (/export/classes & /export/roles) :)

Thursday 01 April 2010 7:31:03 am

Second that for import. We used data_import few times already and is very good extension.

For example we once wrote mysql import/update function with it pretty fast and easy....

http://www.linkedin.com/in/ivolukac
http://www.netgen.hr/eng/blog
http://twitter.com/ilukac

Monday 05 April 2010 7:05:57 pm

Hi again,

OK, so I can see how I might hook data_import into ezxmlexport, and match ezxmlexport's XML format in a data_export custom source-handler. But just using data_import's basic examples (e.g. the XMLFolders.php example) I cannot see the imported content in any GUI - although the data is definitely there in the ezcontentobject MySQL table.

Can someone who's used data_import before show me what I'm missing or doing wrong? The default example should "just work" shouldn't it? - I should be able to use XMLFolders.php as an example and create my own content-class handler from it and use to parse the XML output from ezxmlexport - right?

Any pointers would be most welcome. I'd be just as happy to post this on data_import's projects.ez.no page, but that forum prob doesn't get the kind of traffic this forum gets...

Thanks again :-)
Russell

Russell Michell, Wellington, New Zealand.
We're building! http://www.theruss.com/blog/
I'm on Twitter: http://twitter.com/therussdotcom

Believe nothing, consider everything.

Tuesday 06 April 2010 1:08:53 pm

Hi Nicolas and thanks for your offer of help :-)

Yes I cleared the cache - always do :-)

I'm trying to use the data_import extension. There is no entry for it (run.php) in my settings/override/cronjobs.ini.append.php and there is no copy of cronjob.ini that is bundled with the data_import extension. There is no mention of using a cronjob in the data_import README and no 'cronjobs' folder in the data_import extension.

What am I missing?

I can see the imported content-objects in the ezcontentobject MySQL table, so the import itself does work, but there is something missing or not set correctly somewhere in the DB becuase these items do not show up at all in the admin siteaccess.

This is what I used to set it off:

#> php extension/data_import/scripts/run.php -i ImportOperator -d  XMLFolders -s my_site_access

Starting import with "Folders  Handler" handler
Importing remote object (xmlfolder_10) updating object ID ( 6218 ).
Importing remote object (xmlfolder_20) updating object ID ( 6219 ).
Importing remote object (xmlfolder_30) updating object ID ( 6220 ).Finished
  #> mysql
 Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6969 to server version: 5.0.22
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 mysql> use my_DB;
 Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 Database changed
 mysql> SELECT * FROM ezcontentobject WHERE id IN (6218,6219,6220);
 +-----------------+-----------------+------+---------------------+--------------+---------------+----------+------------+----------+-----------+--------------+------------+--------+
 | contentclass_id | current_version | id   | initial_language_id |  is_published | language_mask | modified | name       | owner_id |  published | remote_id    | section_id | status |
 +-----------------+-----------------+------+---------------------+--------------+---------------+----------+------------+----------+-----------+--------------+------------+--------+
 |               1 |               1 | 6217 |                   2  |            0 |             3 |        0 | New Folder |       10  |         0 | xmlfolder_10 |          1 |      0 |
 |               1 |               1 | 6218 |                   2  |            0 |             3 |        0 | New Folder |       10  |         0 | xmlfolder_20 |          1 |      0 |
 |               1 |               1 | 6219 |                   2  |            0 |             3 |        0 | New Folder |       10  |         0 | xmlfolder_30 |          1 |      0 |
 +-----------------+-----------------+------+---------------------+--------------+---------------+----------+------------+----------+-----------+--------------+------------+--------+
 3 rows in set (0.00 sec)

The only differences I can see are that the remote_id isn't an MD5 hash - but I guess it becomes one at some point? And also there is no entry for these object ID's in the ezcontentobject_link table where there entries for other content-objects I created manually earlier.

I noticed the owner_id for these objects was set to a non-existent user, so I updated these to reflect the admin user which made no difference.

Any guidance would be very much appreciated :-)

[UPDATE]
I just installed a fresh version of eZ 4.1.3 and installed data_import. Ran the same instruction as above and it worked. There must be some issue with my DB - its schema has changed several times due to at least 3 upgrades and extensions - maybe an issue lies there? Still - no errors in error.log which is odd... I'll keep looking

More...

• Dumped working DB (With data_import's pre-imported example content-objects), imported into faulty DB: The data_import example data showed-up OK
• Dumped working DB (Without data_import's pre-imported example content-objects), imported into faulty DB: Re-ran the data_import procedure on faulty DB, and the data_import example data showed-up OK
• This means:

1. There's nothing wrong with eZ's codebase in the faulty eZ install
2. There's something wrong (different) between the working and faulty DBs

In the faulty DB:

• There are additional extension's tables (ezxmlexport, ezgmaplocation and my own customisations - additional columns - to ezrss_export) which should have no impact - right?
• Some tables have a slightly different column definition order in their CREATE TABLE declarations: ezpending_actions, ezsession, ezworkflow_event, ezuservisit & ezurlalias_ml have slightly different column definition ordering.

So might there be an issue with some existing data? Is there an admin script or extension you know of that can "clean" the database of "bad" objects? I have already run eZ's cron scripts: internal_drafts_cleanup.php, subtreeexpirycleanup.php and staticcache_cleanup.php which made no difference..
[/UPDATE]

Regards,
Russell

Russell Michell, Wellington, New Zealand.
We're building! http://www.theruss.com/blog/
I'm on Twitter: http://twitter.com/therussdotcom

Believe nothing, consider everything.

Monday 12 April 2010 2:46:29 am

Hi Russell,

Good to hear you made it, with a little help from us :)

Note for later : never use any form of 4.0.x anymore :)

Cheers !

--
Nicolas Pastorino
Director Community - eZ
Member of the Community Project Board

eZ Publish Community on twitter: http://twitter.com/ezcommunity

t : http://twitter.com/jeanvoye
G+ : http://plus.tl/jeanvoye