Skip to content
This repository has been archived by the owner on Jan 5, 2022. It is now read-only.

sul-dlss-deprecated/sparql-loader

Repository files navigation

Rialto SparqlLoader

CircleCI

Calling a lambda over http through API Gateway

curl --http1.1 --data-urlencode update@basic_insert.txt -H 'X-API-Key: <API KEY>' https://d176x3sh52.execute-api.us-west-2.amazonaws.com/development/rialto-sparql-loader

Where basic_insert.txt is a text file with a sparql query like:

PREFIX dc: <http://purl.org/dc/elements/1.1/> INSERT DATA { <http://example/book3> dc:title    'A new book' ; dc:creator  'A.N.Other' . }

Testing

Install dependencies

  1. Create a python virtual environment
  2. Activate your virtual environment
source env/bin/activate
  1. Install dependencies
pip install -r requirements.txt

Unit testing

pytest -vv -k unit

Integration testing

  1. Start localstack and blazegraph via docker.
docker-compose up
  1. Run the test
AWS_ACCESS_KEY_ID=999999 AWS_SECRET_ACCESS_KEY=1231 pytest -vv

Building an AWS Lambda deployment package

Per the AWS Documentation, a deployment package is made from the virtualenv installed dependencies.

  1. Create a python virtual environment
  2. Activate your virtual environment
source env/bin/activate
  1. Install dependencies
pip install -r requirements.txt
  1. Create zip file
zip sparql-loader.zip handler.py sns_client.py neptune_client.py
  1. Copy dependencies into zip file
cd env/lib/python3.6/site-packages/
zip -r ../../../../sparql-loader.zip honeybadger isodate psutil rdflib rdflib_sparql requests

Note: We are packaging the minimum level of dependencies to try to keep our deployment package small.