Frans van Dunné

Copying a Plone website with SVN

Publicado el viernes, 16 de diciembre de 2011

For a new project I quickly setup  a website with webshop in Joomla. But then the specifics of the requirements came in:

  1. Be as secure as secure can be
  2. Allow for specific user rights to content and content-within-content
  3. Allow for customizations, all related to user rights management
  4. Offer a secure document publication workflow

And this I was not able to do with Joomla, and there was too much bad press for PHP on the security side at that moment, whether that should have been a realistic concern or not. So after digging into the CMS options at hand in the summer of 2011 we choose Plone.

Plone is not for the faint at heart when it comes to the technicalities for setting it up. My level of expertise was basically not being afraid of the command line or of tweaking existing code. And with only that in my pocket I embarked on the Plone adventure.

It has been very enjoyable, but the learning curve is as steep as they say. Some of the very basic parts of setting up a website, or even installing add-ons is written in the documentation as something simple and straightforward. But he writers often seem to forget how much they already know and how much knowledge they take for granted.

So as I’m going along I’ll post some of the basics that I have encountered as being tricky – sometimes taking me days to figure out. And yes, when you have gone through the motions once, it seems very intuitive. This is a reminder to myself that it was not when I started.

One of the problems I found – and this may be simply due to my lack of experience with Plone – is that I had strange discrepancies in the Plone was acting on ./bin/buildout with an installation on my local machine. The solution was to create two virtual machines using the same Ubuntu 10.04LTS image I have running as my production server. This works like a charm. However, keeping them in sync is less straightforward, because I work using SSH to all three servers, with no graphical interface.

Recipe number one:

I want to move all my Plone website content to a staging server, make changes on staging and after acceptance bring them back to my production server.

Seems easy? Well yes it is after you get some of the basics going. For some reason I have not been able to do this using repozo (the backup mechanism of Plone), because it refuses to restore any snapshots I make on the production server back on staging server. If you read this and know what switch to repozo I am missing, please let me know.

The options below assume however that all the same functionality is available. So if you have added any functionality by adding it to buildout.cfg and then running ./bin/buildout, you must do exactly the same on your staging server first.

Option 1 – Works like a charm
However, if you are going to do this regularly it is easier to be able to push a button and have the latest version of your site contents on your development or staging server. I worked with SVN first (see option 2) but this is too time consuming, and there is no point to pollute your version control with so much data.

Instead a little bashscript worked wonders. Be careful with this – test it somewhere safe with your own settings, because bash has the power to destroy things if you copy paste without thinking.

Here it goes:
echo "HOTCOPY SCRIPT"
echo "Will fetch latest versions of filestorage and blobstorage from the HotCopy folder on the Production server"
echo "Now copying filestorage"
scp -r root@xxx.xxx.xxx.xx:/home/PloneBackup/HotCopy/filestorage /usr/local/Plone/zeocluster/var/
echo "Copied the latest data.fs files from production"
echo "Now copying blobstorage"
scp -r root@xxx.xxx.xxx.xx:/home/PloneBackup/HotCopy/blobstorage /usr/local/Plone/zeocluster/var/
echo "Copied the latest blobstorage from production"
echo "This script has finished successfully"

It will ask for your password twice, but this is a benefit because it offers a natural breakpoint to not copy the blobstorage again. Usually it is the ZODB database itself you want to keep updated.

Also please note that I copy the files to a safe location outside the production folders first.

Option 2 – cumbersome, but it works
You could bring your Data.fs files and blobstorage folder under version control. But this is not the best option. It is great to keep track of your src and buildout configurations though.

If you do want to bring your zeocluster/var folder under version control directly keep in mind that it is best to keep the whole var folder outside of versioning as suggested by Aspeli in the Professional Plone 4 Development book. So what I do is to route the copies through my /home/PloneBackups folder. This is safe and gives a lot of control over what you are doing.

  1. Set up SVN somewhere. I’ve used a Dreamhost account for that for ages, and moved my svn repositories to a VPS some time ago now that they have become so cheap.
  2. Read and practice a bit with SVN through the command line. This page on tigris.org works your through the basics. I have struggled with the naming convention for svn on my VPS because it is not routed through https. So try it out outside of your project first. I absolutely adore KDESVN on my local machine because it is faster than most interfaces and as intuitive. But there are many options out there.
  3. Create the svn folder that you want to use to do the file exchange between the servers (mine is called “/home/PloneBackups/copies”, to separate them from proper backups using repozo which I keep in /PloneBackups).
  4. On the server hosting your plone site install svn (sudo apt-get install subversion). Change directory to usr/local/Plone/zeocluster/var/ and add the complete folders using svn add filestorage and svn add blobstorage.
  5. Now go to your staging server and check out the copies in stagingserver/home/PloneBackups.
  6. Then from there copy the blobstorage and filestorage folders to your zeocluster/var folder. If you copy them, it is easy to restore them back once you have fiddled enough with the control panel and ZMI to want to start over.

I do the same to the development server so I can take into account the content that the users are adding to the site when testing.