eZDFSFileHandler

Author Message

Alexandre Henriet

Tuesday 10 November 2009 12:15:01 am

eZDFSFileHandler
Hi all,
I'm trying to setup an eZ Publish cluster following the guide found at
http://pubsvn.ez.no/nextgen/trunk/doc/specifications/trunk/db_nfs_cluster_handler/dbnfsclusterhandler.txt
I migrated a 4.1.3 site to 4.2 stable.
I copied it onto 2 servers, I adapted the database credentials and host information.
I have a nfs share mounted on both servers at /mnt/nfs.
I created the second database required for handling locks.
I modified the file.ini to use eZDFSFileHandler, the nfs share and the second database.
For the moment, I'm only trying to make work ONE of the eZ Cluster node, and
I encounter a problem that I cannot really identify, there are many error messages
in the logs, mostly of type :
eZDFSFileHandlerMySQLBackend::_fetchContents:
An error occured while reading contents of DFS://var/plain_site/cache/ezcontentlanguage_cache.php
---
eZContentObjectTreeNode::pathWithNames() failed to fetch path of node 198,
falling back to generated url entries. Run updateniceurls.php to fix the problem.
I tried to run the script updateniceurls, here's what I received :
phpcc-lab htdocs42 # ./bin/php/updateniceurls.php
Note: any errors encountered will be logged to urlalias_error.log
Using fetch limit: 200
Warning: Invalid argument supplied for foreach() in /var/www/ezcluster_castor/htdocs42/kernel/classes/ezcontentlanguage.php on line 213
Call Stack:
0.0044 386568 1. {main}() /var/www/ezcluster_castor/htdocs42/bin/php/updateniceurls.php:0
0.0837 5694912 2. eZContentObjectTreeNode::fetch() /var/www/ezcluster_castor/htdocs42/bin/php/updateniceurls.php:1079
0.0837 5696048 3. eZContentLanguage::sqlFilter() /var/www/ezcluster_castor/htdocs42/kernel/classes/ezcontentobjecttreenode.php:3028
0.0837 5696476 4. eZContentLanguage::prioritizedLanguages() /var/www/ezcluster_castor/htdocs42/kernel/classes/ezcontentlanguage.php:718
0.0838 5697300 5. eZContentLanguage::fetchByLocale() /var/www/ezcluster_castor/htdocs42/kernel/classes/ezcontentlanguage.php:344
0.0838 5697436 6. eZContentLanguage::fetchList() /var/www/ezcluster_castor/htdocs42/kernel/classes/ezcontentlanguage.php:284
...
Total update 0/49
Node time taken: 0s
Total time taken: 0s
---
Does anyone have experience of that kind of setup ?
Can someone provide me some help ?
Thanks in advance,
Alexandre

Bertrand Dunogier

Tuesday 10 November 2009 1:19:37 am

Hi.

Besides the eZDFSFileHandlerMySQLBackend::_fetchContents error, the other messages are URL alias migration issues. My suggestion is that you should first get the site working without cluster mode enabled. Once URLs do work correctly, configure your cluster, and clusterize your binary files.

Could you also show us your file.ini override ?

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Alexandre Henriet

Tuesday 10 November 2009 1:20:21 am

Sure.

In fact I modified the file.ini directly (I know it's a bad practice, I wanted to reduce the complexity of my problem).

As I said, I switched the file handler in [ClusteringSettings]

[ClusteringSettings]
# Cluster file handler.
# Since 4.1 name of the filehandlers have changed
# you may choose between :
# - eZFSFileHandler
# standard file handler
# - eZFS2FileHandler
# enhanced standard file handler, with better concurrency handling
# requires linux or PHP 5.3 on windows
# - eZDBFileHandler
# database file handler
# - eZDFSFileHandler
# distributed filesystem handler with a DB overlay. Required for NFS based
# architectures
# and it is case sensitive
#FileHandler=eZFSFileHandler
FileHandler=eZDFSFileHandler

And I configured eZDFSClustering

# DFS Cluster Handler settings
[eZDFSClusteringSettings]
# Path to the NFS mount point
# Can be relative to the eZ publish root, or absolute
MountPointPath=/mnt/nfs
# Database backend
DBBackend=eZDFSFileHandlerMySQLBackend
DBHost=10.2.xx.xxx # ip is valid in my conf
DBPort=3306
DBSocket=
DBName=ezcluster_dbnfs
DBUser=ezcluster
DBPassword=xxxxxx # password is different in my conf
DBConnectRetries=3
DBExecuteRetries=20

That's all I've modified ..

Database exists and can be connected by the specified user,
and /mnt/nfs is writable by the apache user ...

Alex

Alexandre Henriet

Tuesday 10 November 2009 1:23:24 am

The site works perfectly with right URLS in every language and without any error in the debug when I use the default file handler

FileHandler=eZFSFileHandler

in file.ini ...

Bertrand Dunogier

Tuesday 10 November 2009 2:20:48 am

Is anything written to /mnt/nfs once you've tried loading a page ? And is there something in the ezdfsfile table of your cluster DB ?

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Alexandre Henriet

Tuesday 10 November 2009 3:32:25 am

Here is where I am now :
1. I cleared the ezdfstable in my ezcluster_dbnfs schema
2. I deleted the var/ folder which has been created in my /mnt/nfs
3. I cleared the cache

./bin/php/ezcache.php --clear-all --purge
Purging : Cache d'affichage de contenu, Cache Global INI, Cache INI, Cache codepage, Cache des identifiants de classes, Cache des clés de classement, Cache des alias d'URL, Cacher de transformation des caractères, Alias d'image, Cache des templates, Cache des template block, Cache des templates override, Cache des textes convertis en image, Cache RSS, Cache de informations personnelles des utilisateurs, Menu de l'arborescence de contenu., Cache des l'imitations d'états.
4. When I browsed the site again, page generation was successful.
In my database I got entries for files located in var/plain_site/cache/...
with different scopes (user-info-cache, template-block, viewcache, wildcard-cache-index,
classidentifiers, statelimitations, content)
And in nfs
ls -l /mnt/nfs/var/plain_site/cache/
total 32
-rw-r--r-- 1 apache apache 841 2009-11-10 12:01 classidentifiers_4dcc7e97b05af059deb0c224815e5853.php
drwxrwxrwx 3 apache apache 4096 2009-11-10 12:01 content
-rw-r--r-- 1 apache apache 122 2009-11-10 12:01 expiry.php
-rw-r--r-- 1 apache apache 498 2009-11-10 12:01 ezcontentlanguage_cache.php
-rw-r--r-- 1 apache apache 39 2009-11-10 12:02 statelimitations_4dcc7e97b05af059deb0c224815e5853.php
drwxrwxrwx 6 apache apache 4096 2009-11-10 12:02 template-block
drwxrwxrwx 3 apache apache 4096 2009-11-10 12:01 user-info
drwxrwxrwx 2 apache apache 4096 2009-11-10 12:02 wildcard
*Happy*
4. Problem occurred after, when I ran
./bin/php/ezcache.php --clear-all
Clearing : Cache d'affichage de contenu, Cache Global INI, Cache INI, Cache codepage,
Cache des identifiants de classes, Cache des clés de classement, Cache des alias d'URL,
Cacher de transformation des caractères, Alias d'image, Cache des templates, Cache des template
block, Cache des templates override, Cache des textes convertis en image, Cache RSS,
Cache de informations personnelles des utilisateurs, Menu de l'arborescence de contenu.,
Cache des l'imitations d'états.
Pages with only text content kept working perfectly, but those with image files were not displaying images
anymore, and the debug said :
Error: eZImageManager::createImageAlias Nov 10 2009 12:19:46
The reference alias original file var/plain_site/storage/images/members/yyy/1453-52-eng-GB/yyy.jpe does not exist
Error: Nov 10 2009 12:19:46
Original alias does not exist, cannot create other aliases without it
Error: eZImageManager::createImageAlias Nov 10 2009 12:19:46
Failed creating the referenced alias reference, cannot create alias subslogo
Error: eZImageManager::createImageAlias Nov 10 2009 12:19:46
The reference alias original file var/plain_site/storage/images/members/yyy/zzz/1997-25-eng-GB/zzz.jpe does not exist
Error: Nov 10 2009 12:19:46
Original alias does not exist, cannot create other aliases without it
And many more .. (one couple of errors per image displayed on the page)
5. I don't have that problem on that page the first time I load that page after clearing the /mnt/nfs and the cluster db,
and I never encounter it when I'm using the default file handler ...
Any idea ?
Thanks a lot already for reading me !
Alex

Bertrand Dunogier

Tuesday 10 November 2009 4:03:44 am

Hum... we're getting somewhere, but I'm a bit confused by the images thing.

Have you created the index_cluster.php file along with the rewrite rules so that images are served from DFS ?

Could you run the following query on your DFS cluster table:

 SELECT * FROM ezdfsfile WHERE name LIKE 'var/plain_site/storage/images/members/yyy/1453-52-eng-GB/yyy.jpe'

And please also do a

 ls /mnt/nfs/var/plain_site/storage/images/members/yyy/1453-52-eng-GB/yyy.jpe

And does this happen with every image file ?

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Alexandre Henriet

Tuesday 10 November 2009 6:40:15 am

I don't have the index_cluster.php file, I was following that documentation :
http://pubsvn.ez.no/nextgen/trunk/doc/specifications/trunk/db_nfs_cluster_handler/dbnfsclusterhandler.txt
and it doesn't appear in it :)
I've seen that the index_cluster.php appears in
http://ez.no/doc/ez_publish/technical_manual/4_x/features/clustering/setting_it_up
but that page describes a setup using the ezdb file handler, is there an equivalent
documentation for the ezdfs (nfs) way of doing ?
I don't have a storage/ folder in the var/ of /mnt/nfs ..
and in the ezdfsfile table, I only have entries with names like var/plain_site/cache/.*
It's the only page that serves image files uploaded through the backend,
and it happens with all of them yes ..

Bertrand Dunogier

Tuesday 10 November 2009 8:11:55 am

Okay, I can understand the confusion now.

I have no idea _ yet _ why the real documentation was skipped, but I'll add it here, as special footage currently exclusive to the community.

HOW-TO CONFIGURE eZDFS
About
eZDFS stands for Distributed File System.
This handler's principle can be summarized as follows: cluster files are mainly stored on NFS, while their metadata (size, mtime, expiry status) are maitained in a DB table similar to the one used by eZDB. NFS is used to read and write the reference copy of clusterized files. Cache files will be copied locally when used by a frontend. Images & binary files (when accessed directly via the browser) will be streamed directly from NFS.
eZDFS: Global architecture configuration
The most important aspects of architecture are the NFS mount point and the cluster database.
Each instance of eZ publish sharing the same relational database has to use the same cluster database, and each should have a local mount point to the same NFS export, configured in the exact same way.
Example:

 NFS-server:/exports/myvardir
eZpublish-server1:/var/www/var/nfsmount => mount of NFS-server:/exports/myvardir
eZpublish-server2:/var/www/var/nfsmount => mount of NFS-server:/exports/myvardir
eZpublish-server3:/var/www/var/nfsmount => mount of NFS-server:/exports/myvardir

The var directories should in NO CASE be shared among instances, since they will automatically be synchronized. This is valid for both eZDB and eZDFS. The cluster handlers take care of synchronizing data from/to the centralized repository.

Installing
Creating the database structure
The database structure required to hold clusterized files informations has to be created. This table can be created either on the same MySQL server as the one used for the relational database, or on a different one. For large scale websites, a dedicated MySQL server can really improve performances.
The definition of this table can be found in the eZDFS MySQL driver class file (kernel/private/classes/clusterfilehandlers/dfsbackends/mysql.php). It is exactly the same as the one you can find below:

 CREATE TABLE ezdfsfile (
    `name` text NOT NULL,
    name_trunk text NOT NULL,
    name_hash varchar(34) NOT NULL DEFAULT '',
    datatype varchar(60) NOT NULL DEFAULT 'application/octet-stream',
    scope varchar(25) NOT NULL DEFAULT '',
    size bigint(20) unsigned NOT NULL DEFAULT '0',
    mtime int(11) NOT NULL DEFAULT '0',
    expired tinyint(1) NOT NULL DEFAULT '0',
    `status` tinyint(1) NOT NULL DEFAULT '0',
    PRIMARY KEY (name_hash),
    KEY ezdfsfile_name (`name`(250)),
    KEY ezdfsfile_name_trunk (name_trunk(250)),
    KEY ezdfsfile_mtime (mtime),
    KEY ezdfsfile_expired_name (expired,`name`(250))
) ENGINE=InnoDB;

NFS

Since eZDFS is based on NFS, a local mount point to a NFS server has to be available & writeable by the webserver's user on each server that will use eZ Publish.

How the mount points are configured depends on the software vendor of the NFS solution.

Configuring the cluster handler

Each eZ Publish instance that shares the same var directory has to be configured correctly in order to get the cluster handler working. Everything is configured in file.ini (you should of course create a global override for this file).
Setting the cluster handler to eZDFS

This is done by setting in file.ini the ClusteringSettings.FileHandler directive to eZDFSFileHandler:

# in settings/override/file.ini.append.php
[ClusteringSettings]
FileHandler=eZDFSFileHandler

Configuring the handler itself is done in the same file, in the INI block eZDFSClusteringSettings. Most settings are self explanatory, but more details are however available below:

# in settings/override/file.ini.append.php

[eZDFSClusteringSettings]
# Path to the NFS mount point
# Can be relative to the eZ publish root, or absolute.
# The example below will work for a mount of NFS to the subdirectory
# var/nfsmount in the eZ Publish root. Any location is possible   MountPointPath=var/nfsmount

# Database backend (only one available at the moment)   DBBackend=eZDFSFileHandlerMySQLBackend
# Hostname of the MySQL server where the ezdfsfile table lies
DBHost=localhost
# MySQL port
DBPort=3306
# Database name
DBName=ezpublishcluster
# Database user
DBUser=ezpublish
# Password for the user above
DBPassword=ezpublish

Configuring the index_* files for clustering

In order to make cluster files available via direct HTTP requests (images,binary files, etc), it is required to use a configuration script and rewrite rules.
The configuration scripts are included in the release. The one used by apache to serve binary files request is index_cluster.php, and it will be referenced in the rewrite configuration.

This file contains configuration constants, and will include the file index_image.php (part of the eZ Publish distribution) when executed. This file will then include `index_image_dfscluster.php`, the final script that will read and stream the files to the browser.

You will find below an example for index_cluster.php, matching the INI configuration above (settings will usually match between INI and this file):

<?php
// DFS parameters

// the cluster handler name
define( 'STORAGE_BACKEND', 'dfsmysql' );
// database host (eZDFSClusteringSettings.DBHost)
define( 'STORAGE_HOST', 'localhost' );
// database user (eZDFSClusteringSettings.DBUser)
define( 'STORAGE_USER', 'ezpublish' );
// database user password (eZDFSClusteringSettings.DBPassword)    define( 'STORAGE_PASS', 'ezpublish' );
define( 'STORAGE_DB', 'ezpublishcluster' );

define( 'MOUNT_POINT_PATH', 'var/nfsmount' );

include_once( 'index_image.php' );
?>

Cluster rewrite rules

mod_rewrite is mandatory to get the above index_cluster.php script to deliver binary files & images. These rules are actually quite simple: they rewrite every request for a content image / binary file to index_cluster.php, which will then deliver the files directly through HTTP from the NFS server.

These rules are the same than the ones used for eZDB ( http://ez.no/doc/ez_publish/technical_manual/4_0/features/clustering/setting_it_up ). These rules can however be found below:

RewriteEngine On
Rewriterule ^/var/([^/]+/)?storage/images-versioned/.*  /index_cluster.php  [L]
Rewriterule ^/var/([^/]+/)?storage/images/.*            /index_cluster.php  [L]

# Other eZ Publish rewrite rules

These rules have to be found before the standard eZ Publish rewrite rules.
Note: As for the standard cluster, it is more than strongly recommended to use some sort of reverse proxy to cache the binary files request, since these will directly query DB (eZDB) / NFS+DB (eZDFS) and may lead to performance issues.
Clusterizing the files

Once everything is correctly configured, binary data (image & files) have to be clusterized.
This can be achieved using the clusterize.php script:

php bin/php/clusterize.php -v

The script will push all the images found in ezimage and the binary files found in ezbinaryfile to the ezdfsfile table (metadata), and the file themselves to the configured NFS mount point.

Note:
this process can take a long time depending on the amount of filesthe eZ Publish instance contains. If the process is interrupted for any reason, it can be safely restarted.

Clear the eZ Publish cache

Use the command line ezcache.php script:

$ezpublishroot: php bin/php/ezcache.php --clear-all

If you are configuring multiple servers, execute this command on each server.
Troubleshooting
Once everything has been configured and binary content has been clusterized, testing can sometimes be complicated.
First, override the debug.ini file (to settings/override/debug.ini.append.php), and enable the kernel-clustering debug key:

[GeneralCondition]
kernel-clustering=enabled

It will present your with much more debug information regarding the cluster. Refresh any page of your website. The page itself should work correctly.
If images are not displayed, it is most likely that something is wrong witheither your rewrite rules, or your index_cluster.php file. The best way to locate the error is to load an image directly in the browser (usually using the contextual menu on an image, and choosing "Open image in a new tab" or similar). If a "Module not found" error is shown instead of the image, this means that your rewrite rule is not correctly configured. If a PHP error is shown, it means that your index_cluster.php configuration might be wrong.

End of documentation ;)

Now, what you're missing, I think:

  • Once you are done configuring the cluster handler in file.ini.append.php, you need to clusterize your content. What this process does is simple: it copies files from the local var/ folder to your cluster. For eZDFS, it will create the metadata entries in the ezdfsfile table, and copy the binary files to NFS. This will make your images/binary files available.
  • The last step will be to make these binary files available to HTTP requests, and this requires the creation of the index_cluster.php file along with the rewrite rules you will find explained in the doc above.

I hope it helps. And I think it does ;-)

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Alexandre Henriet

Friday 13 November 2009 3:58:04 am

Bertrand,

thanks a lot for your precious help !

It works just fine ;)

Alex

Bertrand Dunogier

Friday 13 November 2009 7:04:15 am

Glad I could help :-)

Bertrand Dunogier
eZ Systems Engineering, Lyon
http://twitter.com/bdunogier
http://gplus.to/BertrandDunogier

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.