Python support library for the Humanitarian Exchange Language (HXL) data standard. It supports both Python 2.7+ and Python 3.

About HXL: http://hxlstandard.org

The hxl() function (in the package hxl) reads HXL from a file object, filename, URL, or list of arrays and makes it available for processing, much like $() in JQuery:

import sys
from hxl import hxl

dataset = hxl(sys.stdin)

You can add additional methods to process the data. This example shows an identity transformation in a pipeline (See "Generators", below):

for line in hxl(sys.stdin).gen_csv():
    print(line)

This is the Same transformation, but loading the entire dataset into memory as an intermediate step (see "Filters", below):

for line in hxl(sys.stdin).cache().gen_csv():
    print(line)

There are a number of filters that you can apply in a stream after a HXL dataset. This example uses the with_rows() filter to find every row that has a #sector of "WASH" and print the organisation mentioned in the row:

for row in hxl(sys.stdin).with_rows('#sector=WASH'):
    print('The organisation is {}'.format(row.get('#org')))

This example removes the WASH sector from the results, then counts the number of times each organisation appears in the remaining rows:

url = 'http://example.org/data.csv'
result = hxl(url).with_rows('#sector!=WASH').count('#org')

The following filters are available:

Sinks take a HXL stream and convert it into something that's not HXL.

To validate a HXL dataset against a schema (also in HXL), use the validate sink:

is_valid = hxl(url).validate('my-schema.csv')

If you don't specify a schema, the library will use a simple, built-in schema:

is_valid = hxl(url).validate()

If you include a callback, you can collect details about the errors and warnings:

def my_callback(error_info):
    # error_info is a HXLValidationException
    sys.stderr.write(error_info)

is_valid = hxl(url).validate(schema='my-schema.csv', callback=my_callback)

Generators allow the re-serialising of HXL data, returning something that works like an iterator. Example:

for line in hxl(url).gen_csv():
    print(line)

The following generators are available (you can use the parameters to turn the text headers and HXL tags on or off):

libhxl uses the Python requests library for opening URLs. If you want to enable caching (for example, to avoid beating up on your source with repeated requests), your code can use the requests_cache plugin, like this:

import requests_cache
requests_cache.install_cache('demo_cache', expire_after=3600)

The default caching backend is a sqlite database at the location specied.

This repository includes a standard Python setup.py script for installing the library and scripts (applications) on your system. In a Unix-like operating system, you can install using the following command:

python setup.py install

If you don't need to install from source, try simply

pip install libhxl

Once you've installed, you will be able to include the HXL libraries from any Python application, and will be able to call scripts like hxlvalidate from the command line.