Upgrading ezpublish from 3.6.1 to 4.0 - Pressure

Author Message

John Smith

Tuesday 30 October 2007 5:08:03 am

Hi Guys,

Hope someone can help me.

I am planing to upgrade few sites already running in ezpublish 3.6.1 to ezpublish 4.0 because of php5 pressure.

I checked the upgradation process in the documentation and the proper way of doing is to follow the following version upgrade steps:

1. 3.6.1 to 3.8.0
2. 3.8.0 to 3.9.0
3. 3.9.0 to 3.10.0

It will a nightmare for me to do three step upgrade for too many sites.

Just confirming with you guys, do I need to follow all the above steps or is there any single method directly from 3.6.1 to 4.0(when released).

As I know for testing purpose I need three php versions 4.3(for ez3.6.1), 4.4(for ez3.8.0) and 5.2(for ez4.0). What about Mysql. At the moment I got mysql 4.0. Will that do the job for the latest ezpublish versions or need upgradaton to mysql 5.0 as well.

Please Help.

Cheers,
Smith

André R.

Tuesday 30 October 2007 5:50:17 am

At the moment It should be something like this:

1. 3.6.1 to 3.8.9
2. 3.8.9 to 3.9.4
3. 3.9.4 to 4.0beta1

But 4.0beta1 won't be out before later this week, so if your in a hurry you should upgrade to 3.10.0 before you go to 4.0.

Note: The reason why you should upgrade to latest in a series is that they tend to fix a lot of bugs that where introduced in the blank (x.y.0) versions, even in the upgrade scripts..

When upgrading from 3.6.1 run the following sql updates:
# dbupdate-3.6.0-to-3.8.0.sql
# dbupdate-3.8.0-to-3.8.1.sql
# dbupdate-3.8.1-to-3.8.2.sql
# dbupdate-3.8.2-to-3.8.3.sql
# dbupdate-3.8.3-to-3.8.4.sql
# dbupdate-3.8.4-to-3.8.5.sql
# upgrade php to 4.4.7
# run upgrade scripts for 3.8

Then when upgrading to 3.9.4:
# dbupdate-3.8.0-to-3.9.0.sql remove the 3.8.1 and 3.8.5 parts as described in the upgrade doc
# dbupdate-3.9.0-to-3.9.1.sql
# dbupdate-3.9.1-to-3.9.2.sql
# dbupdate-3.9.2-to-3.9.3.sql
(skip dbupdate-3.9.3-to-3.9.4.sql since it's covered by the 3.10 upgrade)
# run upgrade scripts for 3.9

then when upgrading to 3.10:
# dbupdate-3.9.0-to-3.10.0.sql remove the parts for 3.9.1 and 3.9.3 as described in the upgrade doc
# run upgrade scripts for 3.10

then when upgrading to 4.0:
# upgrade php to 5.2.4
#run upgrade scripts for 4.0

But I do see the need for a upgrade script directly from latest 3.6 version to 4.0, so I have created a enhancement for it:
http://issues.ez.no/IssueView.php?Id=11787&activeItem=1

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

Xavier Dutoit

Tuesday 30 October 2007 5:53:22 am

Hi,

You don't have to install the intermediate versions, "just" run the mysql scripts and the php scripts that might be required.

As for mysql, the transition between 4.0 to 4.1 (and the utf8 joy) did seriously mess the datas in my case.

Let us know and if you are into a mood of posting the details, it will be greatly appreciated (at least by myself, that will have to follow the same path sooner or later)

X+

http://www.sydesy.com

André R.

Tuesday 30 October 2007 6:06:29 am

Ahh, yes X+ is correct, the current fast path would be:

1. upgrade the eZ Publish files to 3.10
2. upgrade php to 4.4.7
3. dbupdate-3.6.0-to-3.8.0.sql
4. run upgrade scripts for 3.8
5. dbupdate-3.8.0-to-3.9.0.sql
6. run upgrade scripts for 3.9
7. dbupdate-3.9.0-to-3.10.0.sq
8. run upgrade scripts for 3.10
9. upgrade the eZ Publish files to 4.0
10. upgrade php to 5.2.4
11. run the 4.0 upgrade scripts

The enhancement would save step 1 and 2 (and make step 9 -> 1 and step 10 -> 2).

eZ Online Editor 5: http://projects.ez.no/ezoe || eZJSCore (Ajax): http://projects.ez.no/ezjscore || eZ Publish EE http://ez.no/eZPublish/eZ-Publish-Enterprise-Subscription
@: http://twitter.com/andrerom

John Smith

Tuesday 30 October 2007 7:35:37 am

Thanks both of you for your swift replies.

Thanks for creating the issue as an enhancement. Really Appreciated. Do I wait for that issue to get resolved and then do some action or just try the last 11 steps posted by Andre?

My only problem is to update too many websites thats why I do not want to create a mess or break all the live sites.

X+: Few months ago I tried to upgrade 3.61 to 3.8.3 on my development pc, it worked well but needed to update siteaccess settings in the the roles and permissions manually from the administration interface, which I reported as a bug and put me off from upgrading. In otherwords broke few sites. Hope you have not met with that sort of bug now.

As all the websites which needs upgradation are on the linux server but my development pc is running windowsxp. Is it good idea to test all upgradation steps on the windows environment instead of linux?

What you recommend regarding MySQL version for 3.10 and 4.0. Should I keep it 4.0 or 4.1?

Cheers
Smith

Paul Borgermans

Tuesday 30 October 2007 8:14:54 am

Hi

MySQL version: 4.1 at least, and UTF-8 is really required for eZ Publish 4.x.

Regards
Paul

eZ Publish, eZ Find, Solr expert consulting and training
http://twitter.com/paulborgermans

John Smith

Tuesday 30 October 2007 9:34:17 am

Hi Paul,

I am worried about, what X+ Stated in his eariler post. It is not 3-4 sites which I am going to deal with, quite a few in number.

X+ Stated:

<b>As for mysql, the transition between 4.0 to 4.1 (and the utf8 joy) did seriously mess the datas in my case.</b>

Do I need to upgrade mysql 4.0 to mysql4.1 for ezpublish 3.10 as well because you said at least 4.1 for ezpublish4.0.

Cheers.

Xavier Dutoit

Tuesday 30 October 2007 10:24:28 am

Hi,

Don't remember the details, but the problem is that in 4.0, every varchar is a binary blob.

From 4.1 it takes into account the charset. If you have datas already in utf8 in 4.0, it might try to convert them to utf8 again in the update/import.

And double utf encoding are not good for your karma.

I'd suggest to take a db you know that contains something else than english ascii, and do the update locally to see how it behave.

X+

http://www.sydesy.com

John Smith

Tuesday 30 October 2007 11:16:51 am

Cheers for kind reply. I will go for 3.10 for the time being and Mysql 4.0.

One more thing bit confusing, In the upgrading steps, it is written

Step: 1 Make sure that you copy the following directories:

design/example
design/example_admin
var
settings/siteaccess
settings/override

But in the directories, there is nothing like example_admin.

All sites are with

design/news
design/admin
design/base
design/standard

Hopefully it is regarding "news" and "admin" folders.

Please confirm.

Xavier Dutoit

Wednesday 31 October 2007 5:19:34 am

example is just an... example

Save the design(s) you have created, that's it.

In the upgrade, there is also the extension not to forget, of course
X+

http://www.sydesy.com

Steven E. Bailey

Wednesday 31 October 2007 7:00:16 am

I use this script to go from 3.5 to 3.10. It may or may not help. But, you really should know what you are doing and of course this needs linux and it's pretty much works in my environment and may or may not work in any other.

You'll also probably have to run it multiple times to get it right so make sure the database is backed up. It's a good idea to make a backup after the drafts are removed in the upto38 - I constantly have problems with this and always have to write hacks around the failures.

To use it, you have to put your site accesses in the sites function and your admin siteaccess in the adminsites function and the dbname dbuser and dbpassword in the dbinfo function... then for the upto310 updatetipafriendpolicy.php you need a password too. I normally comment that out and run it from the command line.

I've set up an ez360 directory, an ez380 directory and then the ez310 directory and made a symbolic link to the script in each directory. That way I can run the thing from each appropriate directory and then just move to the next one.

To do another site, simply update the siteaccess and the dbinfo...

#!/bin/sh
sites ()
{
        echo "en de fr es pl"
}
adminsites ()
{
        echo "plain_admin"
}
dbinfo ()
{
cat <<-EOF
<dbname> <dbuser> <dbpassword>
EOF
}
newtype ()
{
echo "Converting all tables to InnoDB"
dbinfo|while read db user password
do
        /usr/bin/php4 bin/php/ezconvertmysqltabletype.php --host=localhost  --user=$user --database=$db --password=$password --newtype=InnoDB 2>/dev/null
done
}
upto352 ()
{
echo "Starting 3.5.2 upgrade"
dbinfo|while read db user password
do
        mysql -u $user --password=$password $db < update/database/mysql/3.5/dbupdate-3.5.1-to-3.5.2.sql|egrep -v "Warning|Notice"
done
newtype
}
upto36 ()
{
echo "Starting 3.6 upgrade"
dbinfo|while read db user password
do
        mysql -f -u $user --password=$password $db < update/database/mysql/3.6/dbupdate-3.5.2-to-3.6.0.sql
done
newtype
echo "Starting scripts"
for siteaccess in `adminsites`
do
        echo "Updating Top level"
        /usr/bin/php4 update/common/scripts/updatetoplevel.php -s $siteaccess
        echo "Updating eztimetype"
        /usr/bin/php4 update/common/scripts/updateeztimetype.php -s $siteaccess
        echo "Updating xmllinks"
        /usr/bin/php4 update/common/scripts/convertxmllinks.php -s $siteaccess
done
for siteaccess in `sites`
do #this has to be run on ALL site accesses
        echo "Updating eztimetype"
        /usr/bin/php4 update/common/scripts/updateeztimetype.php -s $siteaccess
done
dbinfo|while read db user password
do
        echo "Updating crc"
        /usr/bin/php4 update/common/scripts/updatecrc32.php $user:$password:$db:mysql
done
}
upto38 ()
{
echo "Starting 3.8 upgrade"
dbinfo|while read db user password
do
        mysql -f -u $user --password=$password $db < update/database/mysql/3.8/dbupdate-3.6.0-to-3.8.0.sql
done
newtype
echo "Starting scripts"
for siteaccess in `adminsites`
do
        echo "Updating multilingual"
        /usr/bin/php4 update/common/scripts/updatemultilingual.php -s $siteaccess
        echo "Updating rss import"
        /usr/bin/php4 update/common/scripts/updaterssimport.php -s $siteaccess
done
}
upto39 ()
{
echo "Starting 3.9 upgrade"
dbinfo|while read db user password
do
        for sql in update/database/mysql/3.9/*.sql
        do
                echo "running $sql"
                mysql -f -u $user --password=$password $db < $sql
                if [ $? -ne 0 ]
                then
                        echo $sql failed!!!
                        exit
                fi
        done
done
echo "Done with DB conversions."
newtype
echo "Starting scripts"
for siteaccess in `adminsites`
do
        #This should be the main language for each site
        echo "Updating class translations"
        /usr/bin/php4 update/common/scripts/3.9/updateclasstranslations.php -s $siteaccess --language=eng-GB
        if [ $? -ne 0 ]
        then
                echo translation failed!!!
                exit
        fi
        echo "Updating xmltext"
        /usr/bin/php4 update/common/scripts/3.9/correctxmltext.php -s $siteaccess
        echo "Updating typerelation"
        /usr/bin/php4 update/common/scripts/3.9/updatetypedrelation.php -s $siteaccess

done
}
upto310 ()
{
echo "Starting 3.10 upgrade"
dbinfo|while read db user password
do
        for sql in update/database/mysql/3.10/*.sql
        do
                echo "running $sql"
                mysql -f -u $user --password=$password $db < $sql
                if [ $? -ne 0 ]
                then
                        echo $sql failed!!!
                        exit
                fi
        done
done
echo "Done with DB conversions."
newtype
echo "Starting scripts"
for siteaccess in `adminsites`
do
        echo "Updating nice urls"
        /usr/bin/php4 bin/php/updateniceurls.php -s $siteaccess
        echo "Importing dba file"
        /usr/bin/php4 bin/php/ezimportdbafile.php --datatype=ezisbn -s $siteaccess
        echo "Converting isbn13 file"
        /usr/bin/php4 bin/php/ezconvert2isbn13.php -s $siteaccess
        echo "Updating multioption datatype"
        /usr/bin/php4 update/common/scripts/3.10/updatemultioption.php -s $siteaccess
        echo "Updating tipafriend"
        /usr/bin/php4 update/common/scripts/3.10/updatetipafriendpolicy.php -s $siteaccess -l admin -p <password>
        echo "Updating vatcountries"
        php4 update/common/scripts/3.10/updatevatcountries.php -s $siteaccess
done
}
########
#      #
# MAIN #
#      #
########
case $1 in
        1) upto352;;
        2) upto36;;
        3) upto38;;
        4) upto39;;
        5) upto310;;
        *) cat <<EOF
input needed:
        1 upto352 you should be in the 3.6 directory
        2 upto36 you should be in the 3.6 directory
        3 upto38 you should be in the 3.8 directory
        4 upto39 you should be in the 3.10 directory
        5 upto310 you should be in the 3.10 directory
EOF
           ;;
esac


And, of course, the code above wraps badly...

Certified eZPublish developer
http://ez.no/certification/verify/396111

Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com

John Smith

Wednesday 31 October 2007 7:31:47 am

Nice script. Atleast I understand what is happening. One thing how you switching in between different php versions for testing purposes????

I would like to tell you guys my experience of upgrading (3.6.0 to 3.10.0) steps this morning.

Step: 2 Upgrading the database

Ran update/database/mysql/3/8/dpupate-3.6.0-to-3.8.0.sql worked well, no errors from command line, but when I checked the database the tables were the mixture of innodb and MyISAM. Changed the create statement in the sql so that the new tables are of Innodb type. Worked well. No problems.

Dont know how to set default storage engine for a database instead of whole server. Any comments welcomed????

Step: 3 Running the system upgrade scripts

Ran update/common/scripts/updatemutilingual.php -s <siteaccess>

Got the following error:

Step 1/6: Removing the drafts:
Fatal error: A database transaction in eZ Publish failed.

The current execution was stopped to prevent further problems.
You should contact the System Administrator (<name>) of this site with the information on this page.
The current transaction ID is <id number> and has been logged.

Please include the transaction ID and the current URL when contacting the system
administrator.

Any comments if anybody knows why I am getting this.

Steven E. Bailey

Wednesday 31 October 2007 8:54:11 am

php4/php5 - I'm doing it with two machines. I haven't found an easy good way to get both running on a debian machine. Which is also why my script goes to 3.10 not 4.0.

There is this, but I haven't done anything with it yet:
www.apachefriends.org/en/xampp.html

The failure during the remove drafts has happened to me many times - especially when I've had 1000+ drafts.

I use this: http://ez.no/developer/contribs/hacks/remove_drafts

It may still fail for some other reason (ezimagealiashandler comes to mind for some reason) but you should have a better idea of where it fails.

Certified eZPublish developer
http://ez.no/certification/verify/396111

Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com

John Smith

Wednesday 31 October 2007 9:36:55 am

Steven,

Thanks you very much. I figured out why I am getting "Fatal error: A database transaction in eZ Publish failed."

Reason:

Guide by Andre R and X+

1. upgrade the eZ Publish files to 3.10
2. upgrade php to 4.4.7
3. dbupdate-3.6.0-to-3.8.0.sql
4. run upgrade scripts for 3.8
5. dbupdate-3.8.0-to-3.9.0.sql
6. run upgrade scripts for 3.9
7. dbupdate-3.9.0-to-3.10.0.sq
8. run upgrade scripts for 3.10
9. upgrade the eZ Publish files to 4.0
10. upgrade php to 5.2.4
11. run the 4.0 upgrade scripts

As guided by Andre and X+ in their earlier replies to my posts, I have upgraded the files to 3.10 to by pass some steps and was running the upgrade scripts for 3.8.

Now I ran the same script after upgrading the files to 3.8.10 and it worked very well. No Problem so far.

But the only problem is the number of steps I have to follow as I mentioned, I am not doing only one site, it is quite few sites.

Results after upgrading 3.6.1 to 3.8.10

1. Following policy in the "Roles and Policies" section created by the administrator
user login SiteAccess( )

is coming blank.

If the policy is there by default means at the time of setup when you install the software they are ok, it is coming with the name.

user login SiteAccess(<Sitename>)

So this means after upgrading the site, I have to manually check all the policies created by my client and edit them with the proper name otherwise they won't be able to use the roles and policies created by them.

This is for one site.

Lets suppose if I am upgrading 30 sites and there are 1-15 custom policies for each site, how will I deal with that. Somebody else has reported it as buy but nobody borthered so far....

Check the Comments on the following URL.

http://ez.no/doc/ez_publish/upgrading/upgrading_to_3_8/from_3_6_x_or_3_7_x_to_3_8_0

Have you meet with this problem? Please clarify.

Steven E. Bailey

Saturday 03 November 2007 7:34:38 am

Yes, this is a (really annoying) problem that I have had - the stripped siteaccesses from the roles. It seems to happen randomly (and of course makes no sense to me). I haven't come up with a solution other than eyeballing the roles afterward.

Actually if you (or anyone else) can come up with a solution to this I'd be real happy to know.

Certified eZPublish developer
http://ez.no/certification/verify/396111

Available for ezpublish troubleshooting, hosting and custom extension development: http://www.leidentech.com

John Smith

Friday 07 March 2008 1:23:21 am

Just thought to start this conversation again. I welcome anybody to discuss their experiences about upgrading ezpublish from 3.6.1 to 4.0.

In the upgradation process is Roles and Permission the issue as disscussed by Steven earlier.

Has anybody written some sort of shell script to automate the process. It will be quite useful if you are upgrading 20-30 sites.

Hopefully someone can help the community who is going though upgradation process from 3.61. to 4.0

Cheers

Regards

Smith

John Smith

Monday 31 March 2008 6:10:12 am

Hi there,

Few things.

Successfully upgraded 3.6.1 to 4.0 with no error. "Successfully" only means no command level errors while upgradation.

Now Checked the site, some of the strange things happened.

1. In the roles and permission, the siteaccess is coming empty, no value.

2. All the urls are now changed to numbers.

Before:

http://www.sitename.com/site_access/news/new_website

After upgrading

http://www.sitename.com/site_access/31/34

All the urls in the aritcles which were linked as "/news/new_website" are not working.

Have I done something wrong.

Please help..

Cheers...

John Smith

Tuesday 01 April 2008 7:29:30 am

Hi there,

Is there any simple script to fill the siteaccess.

After upgrading ezpublish from 3.6.1 to 3.8.0 I am getting this

user login SiteAccess( )

Where as it should be

user login SiteAccess( sitename_admin )

In my site I got 40 different "Roles" in which I am using above mentioned policy.

Please help...

Cheers

Brian Baxter

Wednesday 02 April 2008 11:44:51 am

I too am having this issue, and found out that it is a nightmare, I have dozens of websites, some are using other cms's and other forum program scripts i.e. vbulletin, and none of them had a problem switching over to the new company server running php 5.x. now my main site, the ezpublish one is night mare. I am currently running ezpublish 3.92, and need to get to 4.0 in order to allow it to run on the new server. somehow I am unable to successfully install 3.10 on the old server in an attempt to upgrade from 3.92 to that, and then from there move to 4.0 and run the upgrade scripts. any hints can I just go from 3.92 to 4.0, and do the necessary upgrades there?

Behind every great fortune lies a crime.
http://www.3cwired.com - Web Design/SEO/Repair/Sales/Upgrades
http://www.galants.org - The Home for Galant Enthusiasts
http://www.locatemyip.com - More than just free IP displays

John Smith

Thursday 03 April 2008 3:56:17 am

Are you getting empty Siteacess() in roles and permission after upgrading.

Has anybody test this??

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.