senorita.plonetool Python package providing you plonetool command which allows you to easily create, maintain, diagnose and migrate Plone sites. The script is the culmination of headache and alcohol abuse since 2004.

plonetool is designed for a multisite hosting situations where a small enterprise is hosting several Plone sites from different clients running on the same server. The tool is applicable outside its orignal use case as it generally adheres the best practices of Linux and Plone world.

• The required packages and other global server setup is automatically done you by plonetool. You can start with a fresh server installation.
• The server hosts multiple Plone sites in /srv/plone folder, as per guidelines Linux Filesystem Hierarchy.
• The Plone sites share a Python installation which is created by collective.buildout.python recipe (Python 2.7, Python 2.4).
• For additional security, every Plone site installation is only accessible by its own UNIX user account with password disabled.
• The script can create fresh Plone site installations or migrate (copy) one from the existing server over SSH.
• Some basic automated site maintenance is put in the place: nighly restart cron job, automatic site database packaging, site start up when the server goes up, log rotate

plonetool support Ubuntu / Debian servers and it's tested with Ubuntu 12.04 LTS. For other Linux distributions please run unified installer by hand.

There exist only master version of the tool and more or lessing rolling releases. We suggest install the tool under /root with virtualenv for easy update.

To get started with plonetool on a clean server do the following:

sudo -i # root me babe!
apt-get install curl
git clone git://github.com/miohtama/senorita.plonetool.git
cd senorita.plonetool
curl -L -o virtualenv.py https://raw.github.com/pypa/virtualenv/master/virtualenv.py
python virtualenv.py venv
. venv/bin/activate
python setup.py develop

Now you have command plonetool in PATH from venv/bin/plonetool. You can directly invoke this command as /root/senorita.plonetool/venv/bin/plonetool.

The following assumptions are made how you manage your Plone deployments.

You can have multiple Plone sites as described by LSB services run on the server:

/srv/plone/site1
/srv/plone/site1/buildout.cfg
/srv/plone/site1/var
/srv/plone/site1/src
/srv/plone/site1/eggs
...
/srv/plone/site2
...
/srv/plone/python  # Shared Python interpreters installation

Each site has an UNIX user with the site installation name as the username (e.g. site1). These users have password login disabled; use either sudo or ssh with public key authentication to log in for the site maintenance work.

The suggestion is to add your passphrase protected public SSH key to the Plone UNIX user for login:

sudo -i
# Make sure site1 user has ssh configuration folder
install -d /home/site1/.ssh
# Copy-paste your public SSH key line from your local ~/.ssh/id_rsa.pub file
echo "Long line goes here XXX" >> /home/site1/authorized_keys
chown -R site1:site1 /home/site1/  # Make site1 owner of the file
chmod -R o-rwx /home/site1/.ssh  # Restrict SSH key permissions

Now you should be log in as the site1 user and do the sysadmin tasks:

ssh site1@yourserver
cd /srv/plone/site1
bin/buildout
# ... etc ...

Plone sites use Python interpreters compiled with collective.buildout.python:

/srv/plone/python/python-2.7/bin/python # Plone 4.x
/srv/plone/python/python-2.4/bin/python # Plone 3.x

The sites have an init.d script created as:

/etc/init.d/site1
/etc/init.d/site2
...

All sites on the server are set up to be restarted once in a night by /etc/cron.daily/plone-restartscript. If you use clustered install this happens in graceful manner, without affecting the site users (too much).

The site log rotation is handled internally by the buildout.

TODO: Pack the site database automatically.

Because this script may sudo to different UNIX users and poke root-managed global Python installation, the only way to run this script in create, install etc. use cases is to run as root.

You can execute Plone tool directly from its installation location:

/root/senorita.plonetool/venv/bin/plonetool

Some commands like --fixbuildout work fine without root, though.

This command gets available Plone versions from Github installer repo.

Example:

plonetool --ploneversions

Use this command to get available Plone versios for running install (as below).

This command downloads, installs and set-ups Plone site for multisite hosting on the server. Plone versions are available on Github using `Plone unified installer <https://github.com/plone/Installers-UnifiedInstaller/>´_.

The site is integrated with the server maintenance structure as described in Create an empty Plone installation..

To install the latest Plone version as yoursitename:

plonetool --install /srv/plone/yoursitename #

Or more advanced:

plonetool --version 4.2 --user myunixuser --install /tmp/plone-test/yoursitename

The command should be able to resume errors, especially if running buildout fails due to network errors. After the installation plonetool checks that your site is fully functional (starts up properly).

Please note that by default all Plone sites use port (range) starting at 8080. Currently plonetools install does not change this. You must manually edit buildout.cfg to allocate free TCP/IP ports on the server, so that all sites have unique ports.

The major difference between running Unified Installer by hand and using plonetool are

• plonetool forces you to follow Linux Standard Base server layout. Init and restart scripts support multiple sites on the same server.
• UNIX user for Plone site maintenance is configured for you automatically
• Sites on the server share the Python interpreter (/srv/plone/python)
• plonetool sets file system permissions in more restrictive manner
• plonetool supports Plone 3.x installations

In both the cases, buildout skeleton is setup by the same create_instance.py script.

This command creates an empty server structure where you can drop in your Plone site.

Example:

plonetool --create /srv/plone/mysitename

Does

• Sets up a /srv/plone/python with all Python interpreters using buildout.python
• Creates UNIX user mysitename
• Installs more friendly shell, ZtaneSH, for this user
• Creates /srv/plone/mysitename
• Creates Ubuntu/Debian LBS start/stop script in /etc/init.d
• Sets up automatic restart in /etc/cron.daily/plone-restarts
• Sets up log rotate
• If you give --enablessh your SSH agent's public key gets registered to authorized_keys in the created UNIX user and you can SSH in

Does not do

• Set up site backups

Copies a site (over SSH) from a source server to this server.

• Copies site buildout, site data and custom src/
• Rebootstraps buildout on the new server
• Buildout and site startup check after the migration

Read basics about SSH public key handshaking first. All migration happens over SSH, password free.

Example:

# Start the process on your local computer
# Setup passwordless SSH key exchange to the old server
ssh-copy-id user@oldserver.com

# Now SSH into the new server
# Make sure you have ssh'ed to the server using ForwardAgent option
ssh -A root@newserver.com

# Migrate the site from the old server
plonetool --migrate /srv/plone/newsite oldunixuser@oldserver.example.com:/srv/plone/oldsite

# You can retype the command above to resume the migration

You can also migrate Plone 3.3 site using automatically install/srv/plone/python/python-2.4/bin/python:

plonetool --migrate --python /srv/plone/python/python-2.4/bin/python /srv/plone/newsite oldunixuser@oldserver.example.com:/srv/plone/oldsite

You cannot run migrate command in screen, as because if your SSH agent connection dies remote file copying over SSH hangs.

More info about copying Plone sites

You can use script to check whether an installation under /srv/plone works:

plonetool --check /srv/plone/mysite

It checks

• plonectl command provided
• bin/plonectl instance fg or bin/plonectl client1 fg starts the site

This is a useful shortcut for

• Nightly Plone restarts
• Start all Plone sites on the server bootup

Simply run as root:

plonetool --restartall /srv/plone

It will restart all Plone sites found in /srv/plone.

Stops all sites cleanly.

Example:

plonetool --stopall /srv/plone

Automatically modify buildout.cfg and base.cfg in place to reflect modern Plone best pratices, effectively upgrading and fixing old buildouts to be run with plonetool.

Usage:

# Automatically discovers buidlout.cfg, base.cfg
plonetool --fixbuildout /srv/plone/mysite

Automatizes

• Log rotation enable
• Add missing plonectl command
• Strip out shared egg cache (../buildout-cache/eggs...)
• Add allow-hosts to buildout.cfg
• Workarond [unifiedinstaller] bug

When migrating sites, plonetool plainly accepts any SSH hosts you give it without allowing you manually to check known_hosts fingerprints. Please check all host fingerprints before running the script.

The script automatically disables all possible shared buildout egg cache and download cache folders it finds in buildouts.

Instead, only on local development machines I recommend adding a buildout global configuration file ~/.buildout/default.cfg:

# OSX example
[buildout]
eggs-directory = /Users/moo/code/buildout-cache/eggs
download-cache = /Users/moo/code/buildout-cache/downloads
extends-cache = /Users/moo/code/buildout-cache/extends

Your Plone buildout installation must come with functionality plonectl command provided by plone.recipe.unifiedinstaller buildout recipe.

Add it to your buildout if needed:

parts =
    ...
    unifiedinstaller


[unifiedinstaller]
# This recipe installationls the plonectl script and a few other convenience
# items.
# For options see http://pypi.python.org/pypi/plone.recipe.unifiedinstaller
recipe = plone.recipe.unifiedinstaller
user = admin:admin  # This is not used anywhere after site creation

More complex example with two ZEO front end clients:

[unifiedinstaller]
# This recipe installs the plonectl script and a few other convenience
# items.
# For options see http://pypi.python.org/pypi/plone.recipe.unifiedinstaller
recipe = plone.recipe.unifiedinstaller
user = admin:admin  # This is not used anywhere after site creation
zeoserver = zeoserver
clients = client1 client2

Currently the script recommends /srv/plone file system layout, but this is not a limitation and you can have /var/www etc. layout also.

plonetool comes with an automatic test suite which stresses out all the commands provided.

The script heavily uses Python sh package.

If you need more advanced Python deployment recipes check Salt Stack.

To senorita.plonetool is automatically synced on the server when editing files locally:

. venv/bin/activate
pip install watchdog
watchmedo shell-command --patterns="*.py" --recursive --command='rsync -av --exclude=venv --exclude=.git . yourserver:~/senorita.plonetool'

Lightweight unit tests provider:

. venv/bin/activate
python -m unittest discover senorita.plonetool

A command-line script which executes all commands against an empty (discarable) UNIX server:

sudo -i
. ~/senorita.plonetool/venv/bin/activate
~/senorita.plonetool/src/senorita/plonetool/test-all.sh

Please note that running this is extremery slow. If you have failing cases consider re-run them by hand by copy-pasting them from the script.

Mikko Ohtamaa (Twitter, Facebook)