Clyde Package Manager will allow you to build C and C++ packages for RTEMS and make development easier with a few helpful tools like documentation templates.

This package is in an early alpha stage. It is not ready to be adopted outside of Vecna, due to some hardcoded dependencies on a gerrit server.

Future versions will support multiple 'backends', to download packages, including arbitrary git urls, github username/package-name specifications and more.

Documentation ============ Clyde is a purpose built package manager and build system for C and C++. It has built in support for features that are helfpul in developing firmware using the RTEMS Realtime Operating System.

This project is actively being used at Vecna. However, it is custom tailored to our use cases and may not be useful as is. Future releases may improve the internal stucture to make Clyde more generic, supporting different tool chains, RTOSs, etc.

Clyde is currently broken up into the clyde and clyde2 commands. As we migrate away from the original clyde, more commands will move into clyde2. Eventually clyde will be replaced by clyde2. Packages

Clyde projects are built by a hierarchical collection of one or more clyde packages. Each project consists of a set of C and/or C++ files, header files, and a configuration file. The simplest package will contain a single C file, and a simple configuration file that specifies no dependencies. Clyde takes care of the process of actually compiling projects, including dependencies, and handling most compiler flags. It does this by parsing the configuration file (config.yaml), and generating a build.nina file usable by the ninja build tool which actually is responsible for calling gcc and creating the final executable or library.

Let's take a look at a clyde package.

cd ~/ mkdir clyde-demo && cd clyde-demo clyde2 init application clyde fetch hello # Fetch a hello-world example C++ file:

tree
.
├── build
├── clyde2.log
├── config.yaml
├── include
└── src
    └── hello.cpp

• build - An empty directory that will contain build artifacts, such as .o files generated from compiling .c and .cpp files.
• clyde2.log - A log file containing the most detailed output from running clyde. This log system is currently incomplete.
• config.yaml - Main project configuration. This file specifies basic information about a packages, such as name and version, as well as dependencies and more advanced features
• include/clyde-demo - Package include files should live here. Notice that an empty package creates include/package-name.

• src - C and C++ source files for this package.

  • src/hello.cpp - A simple C++ file pulled in with clyde fetch hello

Assuming that we have a basic package, we can build it in two steps.

1. Use clyde2 to create a build.ninja file from the package. This is done with the clyde2 gen command.
2. Use ninja to invoke the actual compilation and linking steps.

For example:

cd ~/clyde-demo
clyde2 gen
tree
├── build.ninja
├── clyde2.log
├── config.yaml
├── deps
├── include
│   └── clyde-demo
├── src
│   └── hello.cpp
└── versions.txt
4 directories, 5 files

ninja
[2/2] g++ -MMD -MF  prefix/bin/clyde-demo.d -std=c++11  -fdiagnostics-color=always -Iprefix/include -Iinclude build/src/hello.o -o prefix/bin/clyde-demo

tree
.
├── build
│   └── src
│       ├── hello.o
│       └── hello.o.d
├── build.ninja
├── clyde2.log
├── config.yaml
├── deps
├── include
│   └── clyde-demo
├── prefix
│   └── bin
│       └── clyde-demo
├── src
│   └── hello.cpp
└── versions.txt
8 directories, 8 files

There area few interesting things to look at:

A build.ninja file was created. You should take a look at it A versions.txt file was created. It lists the exact git tag used to build each package in the dependency tree. For now, it will only list the top level package.

After building, a few more files are created, notably:

build/src/hello.o
build/src/hello.o.d
prefix/bin/clyde-demo

The two files in build are the object file created from hello.c, and a dependency file calculated by analyzing the the #include directives and include paths in hello.c Right now, this will be almost empty Dependencies

One of the largest features of clyde2 is the package management aspect. Clyde packages can depend on other clyde packages, and clyde will automatically download them and add them to the build.ninja when you re-run gen.

In our extended example, let's add some unit tests by pulling in the CppUTest dependency. Edit config.yaml to look like this :

author: Isaac Gutekunst
author-email: isaac.gutekunst@vecna.com
cflags: {gcc: -std=c++11}
name: clyde-demo
type: application
url: http://example.com
version: 0.0.0
requires:
    clyde-demo-lib:
        version: ">=1.0.0"

A few things to notice: The version of clyde-demo listed is a version range. This represents a version specification to download. However, we could have just as easily used a "==1.0.2", or "*". Clyde understands these version specifications and selects the "best" version by parsing the version strings as a semantic version.

If we re-run clyde2 gen, clyde-demo-lib will be downloaded from git, and placed in the deps folder. The build.ninja file will be updated to include these dependencies.

tree
.
├── build
│   └── src
│       ├── hello.o
│       └── hello.o.d
├── build.ninja
├── clyde2.log
├── config.yaml
├── deps
│   └── clyde-demo-lib
│       ├── config.yaml
│       ... Truncated to conserve space
├── include
│   └── clyde-demo
├── prefix
│   └── bin
│       └── clyde-demo
├── src
│   └── hello.cpp
└── versions.txt

Let's update src/hello.cpp

#include <iostream>
#include <clyde-demo-lib/demo.h>
int main(int argc, char * argv[]) {
    print_demo_text("Demo Text");
}

Now, let's build and run our new project:

ninja
./prefix/bin/clyde-demo

Building

Building is performed in two steps

Generating the build.nina using clyde2 gen Actually compiling using ninja

Since the build.ninja describes all dependencies, editing any file, and then running ninja will cause it to properly rebuild the appropriate.

You should re-reun clyde2 gen whenever you update a dependency, or change config.yaml

--fast flag

If you have a lot of dependencies, the time it takes to check the remote server for each version of a package is rather slow. If you pass the --fast flag to clyde2 gen, it will only look at the local git tags. Future versions will have a hybrid approach that will fetch remotes, but only once per invocation of clyde2 gen. Platforms

One unique feature of clyde that other package managers do not have is the notion of platforms and variants. Platforms are places where clyde packages can run, such as on a linux computer, or on a RTEMS embedded system. Currently the main supported platforms are rtems and linux.

You can specify which platform to build for by specifying --platform when invoking clyde2 gen. When you specify --platform rtems, you may get an error Could not find rtems in / . No BSP specified. Specify with --rtems. In this case, you should specify the path to the RTEMS BSP e.g $RTEMS/arm-rtems4.11/stm32f7x/. This is the same as what you would set RTEMS_MAKEFILE_PATH to.

When you set the platform to rtems, clyde will pass RTEMS specific CFLAGS to the build process. If the type of the clyde package is application, clyde will perform a final link, producing an elf file that can be loaded onto a target system. Variants

Clyde has a concept of a variant. A basic package has an implicit variant src. The src variant is enabled by default. This is why the C and C++ files in src are built by default. Additional variants are created by creating new directories in a clyde package that don't conflict with the reserved names (src, include, build, prefix). For example, you can make a test variant, and put unit tests in there. Activating variants

Variants are activated when conditions specified in the config.yam are met. The conditions are specified using a simple language.

All variants must be listed in config.yaml Each variant must have a when clause Each variant can also have a replaces clause Each variant can have a cflags clause Each variant can have a requires clause

When clauses are true when all the predicates are true.

Predicates are statements about the configuration variables variant, platform and compiler.

Some examples might make more sense then a wall of text:

variants:
    - test:
        when: {variant: test}
        requires:
            cpputest: {version: "*"}
        replaces: src
        cflags: -DUNIT_TEST

This example uses all the clauses. It is active when the variant == test. This is true when you run clyde2 with --variant test. If the variaant variable is set to test, the actual variant becomes active. Its effects will then be processed.

In this case, the src variant will be disabled (because of the replaces clause), clyde will pull in the latest version of cpputest (requires clause), and define the UNIT_TEST symbol due to the cflags clause.

These clauses allow you to construct platform specific variants, such as a package that compiles different C++ files depending on whether it is compiled for RTEMS or Linux.

Here's a little example:

variants:
    - linux: 
        when: {platform: linux}
        cflags: -DPLATFORM_LINUX
    - rtems:
        when: {platform: rtems}
        cflags: -DPLATFORM_RTEMS

In this case, the rtems or linux variant will be activated depending on the target platform. Note: The platform is specified on the command line with --platform.

Controlling Verbosity

Some clyde2 commands can be suffixed with --verbose to get more detailed output. On a successful build, clyde will print a full dependency graph as JSON. On a failure, --verbose will print a full stack trace, not just a message. Special RTEMS Functionality

As described in variants and platforms, clyde has special functionality for building for the RTEMS target. See those sections for details. Template Support

Clyde supports pulling in project and file templates. For a list of templates, type clyde fetch --list :

clyde fetch --list

Available templates
docs             - Sphinx and doxygen generation files
gitignore        - Gitignore file
ycm              - YouCompleteMe VIM autocomplete templates
empty-test       - Empty test skeleton
quickstart       - Basic clyde package template
hello            - Basic clyde hello project
rtems-init       - Basic initialization template for STM32F7 BSP
rtems-init-f4    - Basic initialization template for STM32F4 BSP
rtems-quickstart - RTEMS Top level project with a all files needed to get going fast

To fetch a specific template, use clyde fetch template-name :

clyde fetch rtems-quickstart

Loading /home/igutek/.clyde/config
Created docs/index.rst
Created docs/Makefile
Created docs/conf.py
Created docs/doxyfile
Warning: Refusing to overwrite existing file .gitignore
Warning: Refusing to overwrite existing file .ycm_extra_conf.py
Created src/rtems_entry_stm32f7.cpp
Copied RTEMS initialization for STM32F7. For the F4, fetch rtems-init-f4

This output shows a useful feature: The template system will not overwrite existing files. Template configuration

The clyde python package itself contains a list of supported templates. It lives in clydepm/templates/template_config.yaml Directives

The file is a yaml file with a root entry named templates, containing a list of mappings. templates

Top level list of templates

Each mapping should contain an include, directory, and description. include

Every file in the include will be copied to directory in the package. To rename a file when copying it, wrap the filename in an as statement:

• Makefile.sphinx: {as: Makefile}

This is useful if you have multiple Makefiles that should be named Makefile in the final installation. directory

The target directory for the items in this template. If you want to copy to multiple packages, create multiple packages, each with its own directory, and group those into a meta-package.

description

A short description of the template

message

A message to be printed when the template is instantiated meta-include

If a template has a meta-include directive, it will also instantiate all templates listed under the meta-include statement

Templates themselves will be standard text files using either the Jinja2 or Mustache templating language. As these engines don't support any notion of directory structure, there will be a template configuration file that describes what a template package is.

At the time of this writing, template_config.yaml contained the following configuration. :

# All clyde templates are initially configured 
# in this file.
templates:
    - docs:
        description: Sphinx and doxygen generation files
        directory: docs
        include:
            - index.rst
            - Makefile.sphinx: {as: Makefile}
            - conf.py
            - doxyfile

    - gitignore:
        description: Gitignore file
        directory: $PWD
        include:
            - gitignore: {as: .gitignore}

    - ycm:
        description: YouCompleteMe VIM autocomplete templates
        directory: $PWD
        include:
            - ycm_extra_conf.py: {as: .ycm_extra_conf.py}

    - empty-test:
        description: Empty test skeleton
        message: Please update your config.yaml to add a test variant
        variants:
        directory: test
        include:
            - cpputest-empty.cpp: {as: test.cpp}

    - quickstart:
        description: Basic clyde package template
        meta-include:
            - docs
            - gitignore
            - ycm
            - hello

    - hello:
        description: Basic clyde hello project
        message: Installed basic hello project
        directory: src
        include:
            - hello.cpp
    - rtems-init:
        description: Basic initialization template for STM32F7 BSP
        message: Copied RTEMS initialization for STM32F7. For the F4, fetch rtems-init-f4
        directory: src
        include:
            - rtems_entry_stm32f7.cpp
    - rtems-init-f4:
        description: Basic initialization template for STM32F4 BSP
        message: Copied RTEMS initialization for STM32F4. For the F7, fetch rtems-init
        directory: src
        include:
            - rtems_entry_stm32f4.cpp
    - rtems-quickstart:
        description: RTEMS Top level project with a all files needed to get going fast
        meta-include:
          - docs
          - gitignore
          - ycm
          - rtems-init

• Supporting variou backends besides gerrit
• Friendly defaults and configuration options
• More generic concept of platforms
• Merge clyde clyde2 and properly deprecate clyde
• Ability to run tests on real hardware (clyde test --variant rtems might do something specil)