robots.txt

Author Message

Tony Wood

Monday 20 October 2003 8:27:03 am

Add ez kernel handling for robots.txt. This would enable well behaved search engines to be controlled.

This could pick the file from the root project directory.

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Paul Borgermans

Monday 20 October 2003 9:24:25 am

Hi Tony

I managed to do that for virtual hosts with the apache rewrite rules (extra rule before the ezp ones which stops further redirect processing).

regards

-paul

eZ Publish, eZ Find, Solr expert consulting and training
http://twitter.com/paulborgermans

Tony Wood

Monday 20 October 2003 10:22:02 am

I know, but it should be a kernel function for folks who do not have access to httpd.conf

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Jan Borsodi

Wednesday 22 October 2003 4:33:07 am

So you mean that index.php should serve the robots.txt file?

If so how about having a generic way of serving files directly trough index.php?
The files could be defined in an array in a .ini file which the index.php checks before doing eZ publish kernel functionality.

--
Amos

Documentation: http://ez.no/ez_publish/documentation
FAQ: http://ez.no/ez_publish/documentation/faq

Tony Wood

Wednesday 22 October 2003 5:35:50 am

Yes, we need a way of serving up system files like robots.txt and also to choose to have a 404 page give a 404 header so spiders will know to stop when they hit a 404.

This needs to be done via index.php. Paul just suggested to me that maybe we use the error module to sort through files like this?

Tony

Tony Wood : twitter.com/tonywood
Vision with Technology
Experts in eZ Publish consulting & development

Power to the Editor!

Free eZ Training : http://www.VisionWT.com/training
eZ Future Podcast : http://www.VisionWT.com/eZ-Future

Powered by eZ Publish™ CMS Open Source Web Content Management. Copyright © 1999-2014 eZ Systems AS (except where otherwise noted). All rights reserved.