A free-form web data notebook - "Data management for little guys"

Current status (2015-01-30): public prototype, released for evaluation and feedback.

For information about the current release, please refer to the V0.1 release notes

See Getting started with Annalist

See Installing and setting up Annalist, and Configuring Annalist to use OpenID Connect.

See the Guide to using Annalist

There are also a number of demonstration screencast videos, and accompanying scripts, which can be found at demonstration/evaluation scripts page.

A Linked Data Notebook, supporting collection, organization and sharing of structured and semi-structured data. The name "annalist" derives from "a person who writes annals".

• Easy data: out-of-box data acquisition, modification and organization of small data records.
• Flexible data: new record types and fields can be added as-required.
• Sharable data: use textual, easy to read file formats that can be shared by web, email, file transfer, version management system, memory stick, etc.
• Remixable data: records that can be first class participants in a wider ecosystem of linked data, with links in and links out.

In Annalist, I hope to create a generic data notebook which can be used for diverse purposes, in which I have been motivated by the needs of small academic research groups, and my own past experiences running a small business. I want to deliver a self-hostable, web-based tool that will, "out-of-box", allow collection of web accessible, linked data without prior design of its structure. I aim to allow structure in data to be developed as needs arise. Some of my ideas for this are drawn from pre-web PC tools (e.g. WordPerfect Notebook and Blackwell Idealist) which used simple text based file formats to drive flexible, small-scale databases. I find these products occupied a sweet spot that hasn't since been matched by any web-based software of which I'm aware.

The work on Annalist is in its early stages, but I'm committed to open development from the outset, so you can see all the technical work and notes to date here. As of August 2014, the basic data entry, form generation and configuration logic is working and it is possible to create simple collections of data. The biggest area of missing functionality is support for a range of different data types in a data record (e.g. numbers, dates, images, links to spreadsheet data), and proper support for linking in and out (though these are possible using URIs in simple text fields). It should be possible for a technically competent person to install and run the software, and get an indication of the direction I'm going.

Note that all the active development takes place on the "develop" branch. Tested versions are periodically merged to the default-visible 'master' branch. In due course, I plan to follow a "gitflow"-inspired working style that uses the "master" branch for released, tested software.

The release notes give an summary of the current released state of the software.

Details of current feature development and status are available in the TODO list and GitHub issues.

Plans for future features and developments are sketched in the Development roadmap.

Feedback can be provided via annalist-discuss forum or github issue tracker.

Active development is taking place on the "develop" branch in GitHub - see https://github.com/gklyne/annalist/tree/develop

Ideas motivating the creation of Annalist arose from discussions with past colleagues in the Zoology department at Oxford University, particularly Dr David Shotton who led the BioImage project, and Dr Helen White-Cooper who patiently explained her processes and data handling needs as a genomic researcher.

David Flanders, formerly of JISC and now at University of Melbourne, provided encouragement and support for the development of software tools to support academic activities, and in particular raising the profile of engineering and management skills needed to create effective software tools for academics. A previous project, Shuffl, conducted under a JISC Rapid Innovation program that he ran, provided many technical and management lessons that have contributed to the development of Annalist.

More recently, work in the European Wf4Ever project, which explored the use of Research Objects to capture details of scientific method along with experimental data, raised many questions about the nature of tools needed to support publication of reproducible and re-usable research data. The numerous discussions and experiments have informed many of the design ideas and choices that have gone in to Annalist.