This package provides a collection of modules for writing Python programs that access an ICAT service using the SOAP interface. It is based on Suds and extends it with ICAT specific features.

See also the home page of the package at http://icatproject.org/user-documentation/python-icat/

Python:

• Python 2.6, 2.7, or 3.1 and newer. Python 2.6 requires patching the sources, see below. Python 2.5 is too old and will not work.

Required Library packages:

• Suds, either the original version or the fork by Jurko Gospodnetić. The latter is recommended as the original version is not maintained any more and contains bugs. Python 3 requires the jurko fork.

• argparse

  Only if you are using Python 2.6 or 3.1, argparse became part of the Python standard library in 2.7 and 3.2 respectively.

Optional library packages, only needed to use certain extra features, not required to install or use python-icat itself:

• PyYAML

  Only needed to use the YAML backend of icatdump.py and icatingest.py and to run the example scripts (see below).

• lxml

  Only needed to use the XML backend of icatdump.py and icatingest.py.

• Requests

  Only needed for the example scripts using the ICAT RESTful interface, icatexport.py and icatimport.py.

• pytest >= 2.8

  Only if you want to run the tests.

python-icat follows the standard Python conventions of packaging source distributions. See the documentation on Installing Python Modules for details or to customize the install process.

1. Download the sources, unpack, and change into the source directory.

2. Build:

   $ python setup.py build

3. Test (optional, see below):

   $ python setup.py test

4. Install:

   $ python setup.py install

The last step might require admin privileges in order to write into the site-packages directory of your Python installation.

If you are using Python 2.6, apply python2_6.patch after the first step:

1a. Patch:

$ patch -p1 < python2_6.patch

It removes the use of certain language features (dict comprehensions, curly braces notation of sets, bytes string literal prefix 'b') that were introduced in Python 2.7.

The directory doc/html in the source distribution contains the API documentation of python-icat in HTML form generated by Sphinx. The starting point is doc/html/index.html.

Example scripts can be found in doc/examples. This is mostly an unsorted collection of test scripts that I initially wrote for myself to try things out.

Almost all scripts use example_data.yaml as input for test data. Of course for real production, the input will come from different sources, out of some workflow from the site. But this would be dynamic and site specific and thus not suitable, neither for testing nor for the inclusion into example scripts. So its easier to have just one blob of dummy input data in one single file. That is also the reason why the example scripts require PyYAML.

There is no need to run the tests at all. The test suite is mostly useful to the maintainer of python-icat.

Mosts tests require a test ICAT server to talk to. These tests are disabled by default, unless you configure such a test server. To do so, place an icat.cfg file into tests/data. This file must have at least the configuration sections "root", "useroffice", "acord", "ahau", "jbotu", "jdoe", "nbour", and "rbeck" with the options and credentials to access the test server as the respective user. See doc/examples for an example. Obviously, this implies that your authentication plugin must also have these users configured.

WARNING: the tests are destructive! They will delete all content from the test server and replace it with example content. Do not configure the tests to access a production server!

You can safely run the tests without configuring any test server. You will just get many skipped tests then.

• For Python 2, the return value of the string representation operator of class Query may be a Unicode object if any of the conditions contains Unicode. This violates the specification that requires the result to be a string object. But it makes non-ascii characters in queries work. A formal correct implementation would be much more cumbersome and less useful. For Python 3, there is no distinction between Unicode and string objects anyway.
• The entries in the no_proxy configuration variable are matched against the host part of the URL by simple string comparison. The host is excluded from proxy use if its name ends with any item in no_proxy. It is not checked whether the matching part starts with a domain component. E.g. setting no_proxy=ion.example.org will not only exclude icat.ion.example.org, but also lion.example.org, although the latter is not in the ion.example.org domain. IP adresses are not supported in no_proxy. This is a limitation in the implementation of the underlying Python library.

• There is a bug in the original version of Suds: the way it deals with datetime values is broken. Suds converts all incoming datetime values from the server into what it believes to be local time and then throws all time zone information away. The problem is that Suds' conception of the local time is flawed such that the result from this conversion may be wrong. There is no reliable way to fix the values afterwards.

  One can work around this by setting the local time zone to UTC. (The conversion to UTC is done correctly in Suds.) As a result, all datetime values retrieved from the server will be in UTC, which at least is well defined. You may either set the environment variable TZ to 'UTC' before calling your script or add the lines :

  import os
  os.environ['TZ'] = 'UTC'

  at the beginning of your script. This must be set before importing other modules (in particular suds) in order to be effective.

  As a general rule, python-icat assumes all datetime values that do not contain time zone information to be UTC.

  Note that the bug is fixed in the fork by Jurko Gospodnetić which is recommendated anyway.

• With Python 3.1, icatdump.py emits a spurious error message like :

  Exception ValueError: 'I/O operation on closed file.' 
  in <_io.TextIOWrapper name='<stdout>' encoding='UTF-8'> ignored

  on exit when writing the dump file to stdout (the default). You can happily ignore this message. The cause is that icatdump.py closes its output file when finished and that Python 3.1 emits this error message on exit whenever stdout has been closed, even if there was no further output after closing. I assume this is a bug in Python 3.1.

• Misleading "context" is displayed in error tracebacks with Python 3.1 and 3.2. Python 3 introduced a "feature" called Exception Chaining and Embedded Tracebacks. This causes irrelevant and misleading "error context" to be displayed in the traceback of error messages. python-icat tries to suppress this in the most annoying cases, but this suppression mechanism is only available with Python 3.3 and newer.

Version numbers of python-icat comply with Python standards as defined in PEP 440. They try to adhere to the principles of Semantic Versioning, but may deviate from the formatting defined therein when it is incompatible with PEP 440.

Copyright 2013, 2014, 2015 Helmholtz-Zentrum Berlin fuer Materialien und Energie GmbH All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.