eZ Publish with subversion

Author Message

Alain Sahli

Monday 01 December 2008 9:20:05 am

Hi all,

I created a repository for a eZ Publish project. Each developper has its own local httpd server but the database is shared on a server.

The problem is that this solutin is VERY SLOW ! Which is the best way to manage an eZ Publish project with subversion ?

Thanks for help.

Alain

http://www.wess.ch
eZ Publish Certified developer : http://auth.ez.no/certification/verify/272583

Arnaud Lafon

Tuesday 02 December 2008 1:08:49 am

Hi Alain,

we, at SQLI, are working the same way you are. Only one development database for all of our developper... but we don't have any performance issue, so there are several things to check : network connection, MySQL server configuration, write permissions on each apache instance and ezpublish var/ directory...
I think the problem isn't due to subversion itself.

Regards

Looking for information about SQLI ?
Looking for a new job in Paris ?
Please contact me at alafon [at] sqli [dot] com

Gaetano Giunta

Tuesday 02 December 2008 1:13:54 am

The fact that subversion is used has most likely nothing to do with the perceived slowness of the solution. In fact, your setup is very common, being welll balanced in terms of developer's isolation and need to work on the same set of content.

Factors of slowness are generally:

- not enough ram on the developers workstations for running at the same time eZP/php/apache + ide (eclipse?)

- the db shared by developers is not tuned at all. use one of the available free perl scripts to get basic hints about tuning a mysql database

- disable any antivirus running on developers machines. Antivirus plain kills eZPublish caching systems

- never (ever) store the contents of the var dir in subversion. In fact, you will have to set it to svn:ignore

- eclipse svn integration is still slow and flaky. I'd suggest tortoisesvn on windows or any other svn client on *nix

Principal Consultant International Business
Member of the Community Project Board

Alain Sahli

Tuesday 02 December 2008 5:36:04 am

Thanks for replies.

the db shared by developers is not tuned at all. use one of the available free perl scripts to get basic hints about tuning a mysql database

This is probably my problem. Can you get me a link to this perl script ?

http://www.wess.ch
eZ Publish Certified developer : http://auth.ez.no/certification/verify/272583

Felipe Jaramillo

Tuesday 02 December 2008 8:36:07 am

We've been looking for a perfect Subversion setup for quite a while. I agree the speed issues you face should not be related to SVN.

Our developers have different OS's and sometimes they work away from the office, so having a server environment for each developer was not very attractive.

A solution shared by our friends in Commonplaces was this:

Use a single development machine. Each developer has their home directory accessible through Samba. They check in/out accordingly from SVN.

The dev server has virtual hosts that respond to each developer. eg. developer1.project.company.com.

This way you only have one server, one db, but multiple code working areas.
When code is ready, developer check in, and the SVN admin approves the code to move into the official dev environment and eventually into production.

The VirtualHost and siteaccess take a little time but can be reused.

Just thought I'd share this.

Regards,

Felipe

Felipe Jaramillo
eZ Certified Extension Developer
http://www.aplyca.com | Bogotá, Colombia

Gaetano Giunta

Tuesday 02 December 2008 3:39:44 pm

- an easy to use perl script for mysql: http://wiki.mysqltuner.com/MySQLTuner (I can provide a patched version that works on windows if needed)
- mk-audit could be another one iirc

Principal Consultant International Business
Member of the Community Project Board

Alain Sahli

Monday 08 December 2008 7:31:42 am

I tried to optimize my database server but the queries takes always to long time. I give you my debug output :

Accumulator Elapsed Percent Count Average
ini_load
Load cache 0.0159 sec 0.1441% 14 0.0011 sec
FindInputFiles 0.0079 sec 0.0717% 14 0.0006 sec
Mysql Total
Mysql_queries 0.0655 sec 0.5950% 32 0.0020 sec
Looping result 0.0012 sec 0.0110% 28 0.0000 sec
TS translator
TS init 0.0178 sec 0.1613% 5 0.0036 sec
TS cache load 0.0064 sec 0.0577% 5 0.0013 sec
TS context load 0.0042 sec 0.0384% 5 0.0008 sec
Template Total 0.7441 sec 6.8% 3 0.2480 sec
Template load 0.3077 sec 2.7934% 3 0.1026 sec
String conversion in template resource 0.0034 sec 0.0306% 29 0.0001 sec
Template parser: create text elements 0.0095 sec 0.0860% 6 0.0016 sec
Template parser: remove whitespace 0.0018 sec 0.0164% 6 0.0003 sec
Template parser: construct tree 0.0550 sec 0.4995% 6 0.0092 sec
Template load and register function 0.0132 sec 0.1197% 6 0.0022 sec
Template processing 0.4362 sec 3.9591% 3 0.1454 sec
override
Cache load 0.0124 sec 0.1130% 16 0.0008 sec
Sytem overhead
Fetch class attribute name 0.0002 sec 0.0021% 2 0.0001 sec
Fetch class attribute can translate value 0.0031 sec 0.0286% 1 0.0031 sec
class_abstraction
Instantiating content class attribute 0.0000 sec 0.0003% 2 0.0000 sec
XML
Image XML parsing 0.0004 sec 0.0037% 1 0.0004 sec
General
String conversion 0.0026 sec 0.0237% 30 0.0001 sec
dbfile 0.0014 sec 0.0127% 13 0.0001 sec
String conversion w/ mbstring 0.0013 sec 0.0116% 29 0.0000 sec

Total script time: <b>11.0166 sec</b>

11 sec. !!!! This is really to much :-(

http://www.wess.ch
eZ Publish Certified developer : http://auth.ez.no/certification/verify/272583

Gaetano Giunta

Monday 08 December 2008 2:38:55 pm

Something is definitely weird in your setup, but afaict it is not the db. Look at the following two lines from debug output:

Mysql_queries 0.0655 sec 0.5950% 32 0.0020 sec
Looping result 0.0012 sec 0.0110% 28 0.0000 sec

This means that you spent 0.002 secs executing 32 queries, and 0.0 secs fetching data obtained from those queries.

If you sum up all the times listed in the debug output, I feel (without getting my calculator out) that it would amount to 2,3 secs.

Are you sure you are not having some external resource pulled in causing the slowdown, such as an ldap or webservice connection? Other than that, slow access to the filesystem might play a role... Is there anything strange in the logs?

You could also try the ezenhanceddebug extension, that will give you better insight into the templates you are executing.

Principal Consultant International Business
Member of the Community Project Board

Alain Sahli

Tuesday 09 December 2008 12:10:36 am

I also saw that the sql query only takes 0.002 sec... But if I use my localhost mysql database the website works perfectly... So I think the shard db is the problem !

I will try the ezenhanceddebug an give you a feedback. Thanks for your help !

http://www.wess.ch
eZ Publish Certified developer : http://auth.ez.no/certification/verify/272583

Alain Sahli

Thursday 12 February 2009 7:04:50 am

Hi !

I finally found the problem ! The MySQL user is user@% and with this kind of user the server's loosing a lot of time with the name resolving. There's two solutions, either add root@user-ip or modify the my.cnf file and add

skip-name-resolve

I choose the second solution and now it works fine !

http://www.wess.ch
eZ Publish Certified developer : http://auth.ez.no/certification/verify/272583

Gaetano Giunta

Thursday 12 February 2009 9:23:22 am

Wow!
No wonder it took so long to find out - it would have never occurred to me to look into that direction.
A good trick to keep in your ninja spells book. :-)

Principal Consultant International Business
Member of the Community Project Board

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.