Thanks for checking out our repository. We love this project, hope you do too.

Well, you probably want to know to how run this code.

This repository contains (mostly) Python code written on top of Django and other Python modules. To develop this code, you'll need some third-party software installed.

Here are the steps you'll have to follow:

• Install dependencies (some with the package manager, others with buildout)
• Set up caching
• Set up the database
• Read the wiki pages at https://openhatch.org/wiki/Category:Hacking_OpenHatch

Then you can run your own instance of the OpenHatch website and play with the code from an interactive shell.

OpenHatch is currently designed to run on Python 2.6.

We develop on Debian 6.0 and Ubuntu 10.x, and currently we run the openhatch.org website on Debian 6.0.

If you're running a different Operating system, you might need to adjust the instructions.

Most of the dependencies are handled by "buildout" -- more on that below. But you must install a few using your system package manager.

This is what we do on Debian testing/unstable:

sudo apt-get install python2.6-dev python-libxml2 memcached python-mysqldb python-setuptools libxml2-dev libxslt-dev mysql-server mysql-client python-xapian python-setuptools python-imaging subversion

Or the equivalent for your system. (It's important that you get Python 2.6 or greater -- some of the OpenHatch code depends on Python 2.6 or higher.)

On some Operating Systems you may not need (or be able) to install all of the above packages, depending on what comes prebuilt with Python as default. If you do find a package for any of the above for your system, you should install it.

Subversion is a dependency ONLY if expect to run unit tests. - See the section ## Testing ##

The other dependencies (like Django) are handled by "buildout", so keep reading patiently and we'll get to that.

We use memcached to save the output of particularly slow functions in the OpenHatch code. You already installed it in the previous section, so now start it:

sudo /etc/init.d/memcached start

If memcached complains that it is not enabled, configure it by reading /etc/default/memcached. It's also possible that memcached may have been started automatically by your system when you installed it.

This is used by OpenHatch/Django for all the website caching, such as the full list of projects, as well as individual webpage requests. You can read more about Django caching at http://docs.djangoproject.com/en/dev/topics/cache/.

OpenHatch uses Buildout to manage dependencies.

Run these commands from the top level directory of the repository, and grab some coffee, snacks, or both. This may take awhile.

python2.6 bootstrap.py
bin/buildout

You'll download a whole bunch of Python code and run other setup tasks. This is controlled by setup.py and buildout.cfg in the codebase.

In this section, we create an empty MySQL database, and then create the tables that you need to store data for the OpenHatch instance you'll run. If you want, you can load data from the main openhatch.org site into your instance! Keep reading; that's called "importing a data snapshot."

As background:

Make sure you know how to connect to MySQL. Generally, to create a database, you have to be root. You can use the command mysql -u root -p, then type your MySQL root password. This will open a mysql session. To exit, type "exit" and press enter.

1. Run the database creation script. (This also sets up permissions within MySQL.)

   mysql -uroot -p < mysite/scripts/database_01_create.sql

If you want to know what it does, read the file. It is reasonably well-commented.

(A note: If you ever decide you want to drop the database, just send "DROP DATABASE oh_milestone_a" to MySQL, then re-run the database creation script. You do not have to recreate the users.)

2. Exit from MySQL, and run this command, again from the top level directory:

   ./bin/mysite syncdb --noinput

By passing "noinput", you request that Django not ask you questions. It's important that we not let Django create a super-user at this stage.

3. Run the "migrations" to migrate your database from the empty state into the state it should be in.

   ./bin/mysite migrate

There's a note about troubleshooting migrations below.

4. You're done! The OpenHatch code will now run, and you make modifications.

Now is a good time to import a data snapshot, if you want. See https://openhatch.org/wiki/Importing_a_data_snapshot for more information about that. You can read our privacy policy at https://openhatch.org/policies-etc/.

The following command will start the OpenHatch webserver:

./bin/mysite runserver

This server really doesn't like running in the background, so run it in a separate tab or inside a GNU screen session. If everything went well and the server started you should see something like this

Django version 1.1.2, using settings 'mysite.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Now load http://localhost:8000/ in your web browser. If you see a familiar-looking page then congratulations! You now have your own personal OpenHatch server to hack away at!

To run the server on an IP other than loopback, use the syntax

bin/mysite runserver IP:PORT

where IP and PORT are what IP and port you want the server to run on. This and more info is available with

bin/mysite runserver --help

If you're having troubles at this point, see "Troubleshooting" below, or post your question to #openhatch on irc.freenode.net.

Sometimes the migrations don't work, and South will give you a MySQL script so you can fix things. But South isn't perfect. You might have to fix the syntax errors in this MySQL. For starters, you can run this vim command in normal mode, it will delete the string ' = ' when it appears at the beginning of a new line.

:%s/^ = //

Then remove those trailing square brackets. You may need to replace South's strange, Pythonesque code for dates, with proper MySQL code.

Profile importing is done using the Celery daemon. Run it using the following command:

bin/mysite celeryd -c 8

As with the main server, it is a good idea to run this in a separate tab or inside a GNU screen session, rather than in the background.

At present, there is no inbuilt automation of bug importing (celeryd was causing too many problems). Run the following command to start a bug import:

bin/mysite customs_daily_tasks

Since this can take a while, it is a good idea to run this somewhere where it won't prevent you from using your computer - for example, inside a GNU screen session.

Subversion repositories for the svn training missions are stored in mysite/missions-userdata/svn. This directory must be available via svnserve for users to be able to do the svn missions. See mysite/missions-userdata/svn/README for more details.

As you make changes to the source code, you'll want to run unit tests to make sure that everything is still hunky-dory. To run all tests, except the rather slow tests in the customs app, run this command:

bin/test_sans_customs

To run the tests in the customs app:

bin/mysite test customs

For more about tests visit: http://openhatch.org/wiki/Automated_testing Also, take a look at hudson (which runs our Automated Tests). It's especially helpful if you are trying to solve an issue that is failing tests on the Openhatch server: http://linode2.openhatch.org:8080/

Why would we need to do this?

Run these commands:

cd oh-mainline # Make sure you're in this directory. I'm not sure if it will work elsewhere.
python bootstrap.py
bin/buildout -o # The o flag is for "offline", so you don't hit the network looking for dependencies

This product includes GeoLite data created by MaxMind, available from http://www.maxmind.com/.

This data is automatically downloaded by buildout. In theory we/you should re-download the database every month, when MaxMind does a refresh. To do so, run these commands from the top level directory of the repository:

rm -rf parts/download_geocoding_db # Remove our cached copy
bin/buildout # If there's no cached copy, buildout will automatically download the file

We have a map at http://localhost:8000/people/. To accelerate search on those people, we use Haystack, a Django layer on top of search engines. In our case, we use the search engine known as Xapian.

We used to use SOLR, which was complicated to set up and deploy.

Start in oh-mainline, then run this command:

bin/mysite rebuild_index

(say yes)

There's a management command that runs necessary maintenance tasks. At time of writing, it tries to keep our cache of recommended bugs more or less up to date with the state of the bug table in our database.

To run this maintenance task hourly, run these commands:

# The following use of GNU screen is helpful for running maintenance tasks,
# but is not necessary.
screen -RD # Create an instance of screen, or attach to an existing one. 
# Type Ctrl-a, c to open a new screen
while (true); do ./bin/mysite profile_hourly_tasks; sleep 1h; done
# Type Ctrl-a, d to hide ("detach from") the screen

If you're on Ubuntu 10.04 Lucid Lynx, you might get this sort of exception:

File "/home/stump/openhatch/parts/sqlite_mysite/django/db/backends/mysql/base.py", line 13, in raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e) django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: libmysqlclient_r.so.15: cannot open shared object file: No such file or directory

This is because the MySQL_python distribution is a binary egg that contains extension modules linked to libmysqlclient_r.so.15, while in Lucid the soname version number changed to 16. To fix this, simply install a compatibility package provided by your distribution. For Ubuntu and Debian:

$ sudo apt-get install libmysqlclient15off

Thanks to http://meta.osqa.net/questions/2204/ubuntu-1004-problem-with-missing-libmysqlclient_rso15 for this hint.

An earlier version of this document said to remove the egg from your filesystem. You should not do that, but instead add the compatibility package listed above.

As an aside, the Django cache is held completely in temporary memory by memcached, so clearing the cache is as simple as:

sudo /etc/init.d/memcached restart

• If you're running Ubuntu 10.04 Lucid Lynx, when you execute bin/mysite runserver, you might see this error: File "/usr/lib/python2.6/dist-packages/ctypes/init.py", line 20, in raise Exception, ("Version number mismatch", version, _ctypes_version) Exception: ('Version number mismatch', '1.0.2', '1.1.0')

Here's one person's solution: "The problem can be solved by removing the python-ctypes package, which, it seems, is obsolete in Lucid and was left over from earlier versions." https://bugs.launchpad.net/caffeine/+bug/560636. (Note that you might see different version numbers than the ones above.) To remove the package, try sudo apt-get remove python-ctypes.

When you want to add a jQuery UI component, go here:

http://jqueryui.com/download

Select the following options, plus yours:

• Sortable
• Accordion
• Tabs
• Progressbar

Note that the site will automatically select any dependencies (like jQuery UI's "Core").

First, under "Theme", select "No Theme". Under "Version", select "1.7.2 (stable release, for jQuery 1.3.2). Then click your little cartoon hand on the Download button.

Unzip the file in /tmp/, and just extract the file js/jquery-ui-1.7.2.custom.min.js, and cp it to mysite/static/js/. That will overwrite the existing jQuery UI bundle.

Be sure to check using git diff that the change you've introduced in git's view of that file is exactly what you expect.

Finally, don't forget to add your component to the list above, so the next person does the right thing.

• The Free Software projects that we depend on — see buildout.cfg for a list.
• Our contributors and consultants are listed at http://openhatch.org/+projects/OpenHatch. If you've helped to create OpenHatch, you deserve to be listed there!
• The fantastic people who have helped us:
  • Nelson Pavlosky
  • Shotput Ventures, who mentored and funded us in the summer of 2009.
  • The Institute for Security, Technology, and Society and the Neukom Institute, both at Dartmouth, who sponsored Parker Phinney's internship during the Dartmouth winter 2010 term.
  • Google Summer of Code, who are sponsoring John Stumpo's internship during the summer of 2010.
• See CREDITS for other files we use.

…on the project blog, Identi.ca, Twitter, IRC (#openhatch on irc.freenode.net), and the Face-Book.