BaGoMa - A script to Backup Google Mail.

bAgOmA - A script to backup a bag o' mail.

This software comes in two flavors:

1. bagoma.py = the full-featured CLI version
2. gui.pyw = a simple GUI wrapper for bagoma.py

bagoma.py [options] -e <Email>

Or, for the CLI-phobic:

gui.pyw

Download the zip file containing the stand-alone exe then, execute:

gui.exe

The more adventurous can try from a DOS shell:

bagoma.exe [options] -e <Email>

Alternatively, install python from python.org, CygWin, or ActiveState, then from a DOS or CygWin shell:

python bagoma.py [options] -e <Email>

• Home: http://sourceforge.net/p/bagoma/
• GitHub: http://github.com/gburca/BaGoMa
• Downloads: http://sourceforge.net/projects/bagoma/files/
• Documentation: http://bagoma.sourceforge.net/ and http://ebixio.com/blog/2011/05/28/smart-backup-for-your-gmail-account/
• Project page: http://sourceforge.net/projects/bagoma/
• Freshmeat: http://freshmeat.net/projects/bagoma/

BaGoMa backs-up and restores the contents of a GMail account. It can restore all the labels (folder structure), as well as the flags (seen/read, flagged) of a message. It differs from other similar solutions in a few important ways:

• It is Open Source Software. You can read the code to make sure your password and email contents remain private.

• Some backup solutions force you to use only ASCII characters in your labels. BaGoMa has no such restrictions on label names. Specifically, foreign characters and '/' (the label hierarchy delimiter) are allowed.

• BaGoMa is tuned to work specifically with GMail. Each message is only downloaded and saved once. Backup solutions that are designed to work with regular IMAP accounts will download each message multiple times (once per label) when used to back up a GMail account. That can significantly increase the bandwidth requirements and the amount of storage required for backup. Additionally, a faithful restoration of your GMail account might not be possible from such a backup.

• BaGoMa allows you to read/verify the local backup with an email client. You no longer have to wonder if your backup is good. Fire up an email client and verify it (see the MAILDIR section below for details).

BaGoMa requires your GMail account to be IMAP accessible. Check the Mail Settings and make sure "Enable IMAP" is selected under the "Forwarding and POP/IMAP" tab. In the "Folder Size Limits" section on the same tab, make sure the following option is selected: "Do not limit the number of messages in an IMAP folder".

BaGoMa never deletes any email from your account. A restore simply compares the contents of your local backup with the contents of your GMail account, and uploads any messages missing from your GMail account. It never deletes messages from GMail to make it match the local backup.

Google sometimes imposes limits on how much data can be transfered from your account. If you run into such a limitation, just wait a few days and try the backup again. BaGoMa will pick up from where it left off. If the temporary loss of IMAP access is critical to you, think twice before using this script. As far as I'm aware, web access to your email should continue unimpeded even if IMAP access is blocked.

-h, --help : show a help message and exit

-e EMAIL, --email=EMAIL : the email address to log in with (MANDATORY)

-p PWD, --pwd=PWD : the password to log in with (will prompt if missing)

-d BACKUPDIR, --dir=BACKUPDIR : the backup/restore directory [default: same as email]

-a ACTION, --action=ACTION : the action to perform: backup, restore, compact, printIndex, maildir or debug. [default: backup]

* The function of the *backup* and *restore* actions should be obvious.

* BaGoMa retains all backed up messages, even if they are no longer
present on the GMail server during a subsequent backup. The *compact*
action will compact the local storage by deleting all such messages.

* The *printIndex* action displays all the meta-information BaGoMa
operates on. This action is primarily intended for developers.

* The *maildir* action creates a [Maildir type directory][MaildirFormat]
and sym-links all the backed-up email messages into it so that the mail
can be inspected using a mail reader that supports Maildir directly
(ex: [Mutt]). See the **MAILDIR** section below for more details.

* The "debug" action is also intended for developers and drops the user
into an interactive python session.

--dryRun : When combined with the compact action, shows what files would be deleted without actually deleting them.

-s SERVER, --server=SERVER : the GMail server to use [default: imap.gmail.com]

--port=PORT : the IMAP port to use for GMail [default: 993]

-m MAILDIR, --maildir=MAILDIR : Used with the maildir action to specify the Maildir directory to create [default: Maildir.BaGoMa]

-l LOGLEVEL, --log=LOGLEVEL : the console log level (DEBUG, INFO, WARNING, ERROR, CRITICAL) [default: WARNING]

-f LOGFILE, --file=LOGFILE : the log file (set it to 'off' to disable logging) [default: log.txt]

-c CONFIG_FILE, --config=CONFIG_FILE : the config file to use for options not specified on the command line. See below for more details. [default: .BaGoMa]

--version : show the version number

There's nothing really to install. Once you unpack the downloaded package, you'll find the bagoma.py script in the top directory. On Linux you can run the script directly. On Windows, you'll need to install python first (if you don't already have it) from python.org, CygWin, or ActiveState, then from a DOS or CygWin shell:

python bagoma.py [options] -e <Email>

Man page can be viewed with:

man -l man/bagoma.1

For those that know what they're doing:

python setup.py install

It's usually not a good idea to provide the password on the command line. When BaGoMa is used from a script (cron job), the recommended solution is to create a configuration file and use it to store the password. The configuration file uses the ini format with the email address as the section. For example:

[email_1@gmail.com]
pwd = pass1

[email_2@gmail.com]
pwd = pass2

Note: Before spending time on the details in this section, be aware that this feature makes use of symbolic links (symlinks), so if you're not on a UNIX machine it will probably not work for you.

Having all the email messages available locally as simple files has certain advantages, such as being able to use standard UNIX tools such as grep to search through them. It would be even more useful however if it were possible to use a full-featured email client (MUA) to navigate through the backup and take advantage of all the MUA features. This feature allows you to do just that.

First a word of CAUTION. Any modifications made by the MUA may corrupt the backup. BaGoMa will not check for local modifications made to the backup. As long as the MUA only modifies the symlinks (by renaming, moving, or deleting them) you should be OK. To be on the safe side, I suggest changing the permissions on the backup directory contents so that the MUA can not make any modifications (for example, run the MUA as an unprivileged user that has only read permissions).

To use this feature, run:

python bagoma.py -e <Email> -a maildir

BaGoMa will create a directory called Maildir.BaGoMa and populate it with symlinks to messages in your backup. This new directory is in a proper Maildir format, so it can be accessed with a MUA that handles the Maildir format. A sample configuration file muttrc.BaGoMa is included and can be used with Mutt to access Maildir.BaGoMa. To view the messages modify muttrc.BaGoMa to point to the location of your Maildir.BaGoMa directory and run:

mutt -R -F muttrc.BaGoMa

If that doesn't work, bypass the system config file by adding the -n option:

mutt -n -R -F muttrc.BaGoMa

If you figure out how to get this working with other email clients, please consider documenting your experience (and sharing it on GitHub) so that others may benefit as well.

BaGoMa will NOT backup any of the following:

• Account settings (filter, etc...)
• Contacts
• Chat history
• Labels you've chosen to hide in IMAP
• Superstars (the additional star icons that can be enabled in Labs)
• Contents of Trash and Spam

The IMAP protocol does not provide a trully unique ID for an email message (UIDVALIDITY + UID doesn't count). In order to determine that message A in folder P is the same as message B in folder Q (and only needs to be downloaded once), the script computes its own unique ID using the following header fields:

• From:
• To:
• Cc:
• Date:
• Subject:
• X-GMail-Received:
• Message-Id:
• The server assigned timestamp of when the message was received

Two messages that have identical values (ignoring whitespace) for all of the fields enumerated above are considered to be identical and will only be downloaded and saved once. The chance of two messages having the same Message-Id header field is pretty slim. The chance of all the header fields mentioned above being identical is almost 0.

Latest version available at: http://sourceforge.net/projects/bagoma

Project homepage is at: http://bagoma.sourceforge.net/

The source code is hosted both on SourceForge.net and on GitHub at http://github.com/gburca/BaGoMa. Collaboration will be easier on GitHub.

The imaplib library is used to do the heavy lifting. In addition to the official imaplib documentation, some other useful information can be found at: http://www.doughellmann.com/PyMOTW/imaplib/

The README file uses Markdown formatting with pandoc extensions so that it can be converted to a man page.

If you want to help, see the TODO file.

Bugs can be reported at the project's GitHub or SourceForge page. Better yet, provide patches through GitHub to fix the bugs.

Bug reports should include relevant portions of the log file.

Copyright (C) 2011-2012 by Gabriel Burca. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.