Skip to content
This repository has been archived by the owner on Dec 10, 2021. It is now read-only.

ONSdigital/sdx-store

Repository files navigation

sdx-store

Build Status Codacy Badge codecov

Scalable service for storing SDX data (backed by PostgreSQL).

Prerequisites

A running instance of PostgreSQL.

Installation

This application presently installs required packages from requirements files:

  • requirements.txt: packages for the application, with hashes for all packages: see https://pypi.org/project/hashin/
  • test-requirements.txt: packages for testing and linting

It's also best to use pyenv and pyenv-virtualenv, to build in a virtual environment with the currently recommended version of Python. To install these, see:

Getting started

Once your virtual environment is set, install the requirements:

$ make build

To test, first run make build as above, then run:

$ make test

It's also possible to install within a container using docker. From the sdx-store directory:

$ docker build -t sdx-store .

Usage

Start the sdx-store service using the following command:

$ python server.py

API

There are seven endpoints:

  • GET /invalid-responses - returns a json response of all invalid survey responses in the connected database
  • POST /queue - Publishes a message to a corresponding rabbit message queue based on the message content. Returns a 200 response and JSON value {"result": "ok"} if the publish succeeds or a 500 response with JSON value {"status": 500, "message": <error>} if it does not.
  • GET /healthcheck - returns a json response with key/value pairs describing the service state
  • POST /responses - store a json survey response
  • GET /responses - retrieve a JSON response of all valid survey responses in the connected responses.
  • GET /responses/<tx_id> - retrieve a survey by id
  • DELETE /responses/old - delete responses older than a number of days set in config
  • GET /feedback/<feedback_ID> - retrieve a JSON response of a valid ID

Query Parameters

The /responses , /invalid-responses and /feedback endpoints support paging using URL query parameters.

  • per_page: The number of responses to return per page. Must be in the range 1-100. Defaults to 1.
  • page: The page number to return. Must be 1 or higher if set. Defaults to 1.

Configuration

Some of important environment variables available for configuration are listed below:

Environment Variable Example Description
RABBIT_CS_QUEUE sdx-cs-survey-notifications Name of the Rabbit CS queue
RABBIT_CTP_QUEUE sdx-ctp-survey-notifications Name of the Rabbit CTP queue
RABBIT_CORA_QUEUE sdx-cora-survey-notifications Name of the Rabbit CORA queue
RABBITMQ_HOST rabbit Name of the Rabbit queue
RABBITMQ_PORT 5672 RabbitMQ port
RABBITMQ_DEFAULT_USER rabbit RabbitMQ username
RABBITMQ_DEFAULT_PASS rabbit RabbitMQ password
RABBITMQ_DEFAULT_VHOST %2f RabbitMQ virtual host
RABBITMQ_HOST2 rabbit RabbitMQ name
RABBITMQ_PORT2 rabbit RabbitMQ port
SDX_STORE_RESPONSE_RETENTION_DAYS 90 Youngest response that will get deleted

License

Copyright (c) 2016 Crown Copyright (Office for National Statistics)

Released under MIT license, see LICENSE for details.