clowder - A group of cats

herding cats - An idiom that refers to a frustrating attempt to control or organize a class of entities which are uncontrollable or chaotic

Managing multiple repositories can be pretty frustrating. There are a number of existing options:

• git submodules
• subtree merging
• Google's repo tool
• braid
• gr
• git-subrepo

All of these have their own approach, but many are based on submodules or subtrees. Submodules and subtrees create a tight coupling between repositories because of the way dependencies are stored. Much has been written about their drawbacks elsewhere. Google's repo tool takes a different approach, but is closely tied to Google's development workflow

clowder uses a similar approach as repo, but using yaml instead of xml for the configuration file. URL information and relative project locations on disk are specified in a clowder.yaml file. This file is checked into its own repository. The use of a separate file to track projects allows for detailed information about the dependencies between them to be stored, but each repository is still essentially independent. Projects can track branches, or be tied to specific tags or commits

The primary purpose of clowder is synchronization of multiple repositories, so normal development still takes place in individual repositories with the usual git commands

• git
• Python 3

macOS comes with git preinstalled. To install Python 3 with Homebrew

$ brew install python3

$ sudo apt install git
$ sudo apt install python3-pip

Currently clowder has only been tested on Windows with Cygwin. The following dependencies should be installed

• git
• python3-pip
• python3

To install clowder from PyPI

$ sudo pip3 install clowder-repo

To upgrade to the latest version

$ sudo pip3 install clowder-repo --upgrade

For terminal autocompletion add the following line to your bash profile

command -v clowder >/dev/null 2>&1 && eval "$(register-python-argcomplete clowder)"

This example is based on the Swift projects (see the full Swift projects clowder.yaml)

First reate a directory to contain all the Swift projects

$ mkdir swift-source
$ cd swift-source

$ clowder init git@github.com:jrgoodle/swift-clowder

The clowder init command does the following

• Clones the Swift clowder repo in the swift-source/.clowder directory
• Creates a symlink pointing to the primary clowder.yaml file in the repository

$ clowder herd

clowder herd updates the state of the projects. When the command is run, the following happens

• If any projects don't have a clean git status then clowder exits
• Projects are cloned if they don't currently exist
• Each project fetches the latest changes
• If the current git ref checked out doesn't match the clowder.yaml configuration, the correct ref will be checked out
• The latest changes are pulled for branches. For commits and tags, the commits are checked out into a detached HEAD state

$ clowder status

$ clowder branch # Print all local branches
$ clowder clean # Discard any changes in projects
$ clowder diff # Print git diff for all projects
$ clowder forall -c 'git status' # Run command in all project directories
$ clowder herd -b my_branch # Herd a specified branch if it exists, otherwise use default ref
$ clowder link -v 0.1 # Set clowder.yaml symlink to a previously saved version
$ clowder repo run 'git status' # Run command in .clowder directory
$ clowder save 0.1 # Save a version of clowder.yaml with current commit sha's
$ clowder start my_feature # Create new branch 'my_feature' for all projects
$ clowder stash # Stash changes in all projects
$ clowder prune stale_branch # Prune branch 'stale_branch' for all projects

See the clowder commands doc for documentation of all command options

For example output from individual commands see the clowder command examples

See the clowder.yaml doc for an explanation of the clowder.yaml configuration file

See the clowder repo doc for a description of the structure of the clowder repo cloned in the .clowder directory

See the forks doc for a description of the behavior of clowder commands with forks

See CONTRIBUTING.md for information on setting up your environment for development and for contribution guidelines