Clone this repository from here
git clone git@github.com:CottageLabs/localvoices.git
Once you've got it, you will need to update the submodules on which it depends. Go into the localvoices directory you just cloned, then:
git submodule init
git submodule update
You probably want to run this in a virtual environment, so make one in a suitable location and activate it:
virtualenv --no-site-packages lv
source lv/bin/activate
Install the application; got into the localvoices/esprit directory:
pip install -e .
Go up a level to the root localvoices directory:
pip install -e .
We also need to install and start Elasticsearch, which can be done with a binary like this one (we require > 0.90):
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.tar.gz
tar -zxvf elasticsearch-0.90.10.tar.gz
cd elasticsearch-0.90.10/bin
./elasticsearch
Now start the localvoices application, from the localvoices directory
python portality/app.py
If you want to pre-load the test data, you can then do
python test/loadfromcsv.py
You should now be able to access the API as defined in the technical specification below, or explore the data through the primitive javascript UI provided (described in the next section)
At the moment, the application is provided with a JavaScript driven UI which provides primitive access to the datastore. There are two main components to this:
portality/static/js/localvoices.js
This is a full client library which provides API access to the Local Voices service and basic forms for creating and editing songs, singers and versions. The forms provided are complete but not necessarily particularly pleasing to the eye, so it is anticipated that they will be replaced and/or styled further during UI development.
portality/templates/index.html
portality/static/js/examples.js
This is a set of HTML layouts (using Jinja2), and jQuery functions/event handlers providing basic behaviour to the UI. Each part of the UI implements a thin wrapper around the REST API calls documented further down. So, code in examples.js should be considered just a demonstration as to how to bind the components build during UI development with the localvoices.js function calls.
From an API point of view, the following search operations are required:
1/ Search for singers:
- geographical constraint
- free-text string constraint
What does the free-text string cover: just the singer metadata or also the singer's songs?
Returns:
- A list of singers with some basic info and their lat/long
2/ Search for songs:
- geographical constraint
- free-text string constraint
The free-text string constraint will cover both songs and song variations
Returns:
- A list of songs with some basic info and their lat/long
3/ All information associated with a singer
- The singer metadata
- The list of song versions by that singer
4/ All information associated with a song
- The song metadata
- The list of versions of that song
- The list of singers who sung that song
- The list of related songs
5/ Search free-text across all content
Returns:
- A list of singers
- A list of songs
6/ List all singers alphabetically
Returns:
- A list of singers
7/ List all songs alphabetically
Returns:
- A list of songs
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"name" : {
"first" : "<first name>",
"middle" : "<middle name or initial>",
"last" : "<surname>",
"aka" : [<nicknames and translations>]
},
"groups" : [<list of bands/choirs>],
"gender" : "<male|female>",
"location" : [
{
"relation" : "<nature of singers relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
]
"born" : "<(partial) datestamp>",
"died" : "<(partial) datestamp>",
"biography" : "<biographical summary>",
"photo_url" : "<url for singer's photo>",
"source" : "<source of information about this singer>",
"versions" : [<opaque identifiers of song versions>]
"created_date" : "<created date>",
"last_updated" : "<last updated date>"
}
- id - the opaque system identifier for the singer; not for use by the outside world
- lv_id - The Local Voices identifier for the singer
- name - a compound attribute describing the singer's name, and name variants
- groups - bands/choirs that the singer has been associated with
- gender - gender of the singer
- location - a list of locations relevant to the singer. Each location is a compound attribute with the lat/long and the name of the place, as well as an optional relationship the place has to the singer
- born - date the singer was born. May be a date fragment (e.g. just the year)
- died - date the singer died (if relevant). May be a date fragment (e.g. just the year)
- biography - free text biographical information
- versions - the list of song versions performed by this singer
- created_date - a timestamp, system assigned, for the moment the record was created
- last_updated - a timestamp, system assigned, for the last time the record was updated
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<canonical song title>",
"alternative_title" : [<list of alternative song titles>],
"summary" : "<free text summary>",
"location": [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"time_period" : {
"from" : "<time period indicator>",
"to" : "<time period indicator>"
},
"composer" : "<free text name of composer>",
"relations" : [<opaque identifier of related song>],
"references" : [<references of all versions>],
"versions" : [<opaque identifier of versions>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>"
}
- id - the opaque system identifier for the song; not for use by the outside world
- lv_id - The Local Voices identifier for the song
- title - this is the most commonly agreed-upon title for the song
- alternative_title - this is all of the titles of the song variants
- location - a list of locations relevant to the song. Each location is a compound attribute with the lat/long and the name of the place, as well as an optional relationship the place has to the song
- time_period - a vague date assertion through to a specific date (e.g. Late 17th Century, or c. 2000), and optionally a range using "from" and "to" (if no range, just uses "from")
- composer - name of the composer of this song
- relations - other songs which have relationships to this song (in an unspecified manner)
- classifications - list of classifications aggregated from the song variants
- versions - the list of versions of this song
- created_date - a timestamp, system assigned, for the moment the record was created
- last_updated - a timestamp, system assigned, for the last time the record was updated
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<title of this version>",
"alternative_title" : [<list of alternative titles>],
"summary" : "<free text summary>",
"language" : [<list of languages>],
"media_url" : [<list of media links>],
"lyrics" : "<the lyrics of the version>",
"photo_url" : "<a photo representing the version>",
"singer" : "<opaque identifier of the singer>",
"collector" : "<free text name of collector>",
"source" : "<free text source of material>",
"collected_date" : "<partial date for collection>",
"location" : [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"comments" : "<free text comments>",
"tags" : [<list of tags>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>"
}
- id - the opaque system identifier for the version; not for use by the outside world
- lv_id - The Local Voices identifier for the version
- title - this is the name of this version of the song
- alternative_title - any alternative titles for this version of the song
- language - one of English, Scots, Gaelic, Other
- location - a list of locations relevant to the version. Each location is a compound attribute with the lat/long and the name of the place, as well as an optional relationship the place has to the version
- references - other numbering or classification schemes for this version (e.g. the ROUD number)
- comments - free text comments on the version
- tags - tags associated with the version
- created_date - a timestamp, system assigned, for the moment the record was created
- last_updated - a timestamp, system assigned, for the last time the record was updated
Data Storage will be provided entirely in Elasticsearch, but we will use part of the index like a relational data store for convenience, and construct searchable indices for the various datatypes from that relational store.
6 datatypes are required to model the content:
- Song Store - metadata for the song
- Version Store - metadata for the version
- Singer Store - metadata for the singer
- Song2Version - link table from Songs to Versions
- Singer2Version - link table from Singers to Versions
- Song2Song - link table from Songs to other Songs
Documents held in the Song Store will have the following form:
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<canonical song title>",
"summary" : "<free text summary>",
"location": [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"time_period" : {
"from" : "<time period indicator>",
"to" : "<time period indicator>"
},
"composer" : "<free text name of composer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
}
Documents held in the Version Store will have the following form:
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<title of this version>",
"alternative_title" : [<list of alternative titles>],
"summary" : "<free text summary>",
"language" : [<list of languages>],
"media_url" : [<list of media links>],
"lyrics" : "<the lyrics of the version>",
"photo_url" : "<a photo representing the version>",
"collector" : "<free text name of collector>",
"source" : "<free text source of material>",
"collected_date" : "<partial date for collection>",
"location" : [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"comments" : "<free text comments>",
"tags" : [<list of tags>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
}
Documents held in the Singer Store will have the following form:
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"name" : {
"first" : "<first name>",
"middle" : "<middle name or initial>",
"last" : "<surname>",
"aka" : [<nicknames and translations>]
},
"groups" : [<list of bands/choirs>],
"gender" : "<male|female>",
"location" : [
{
"relation" : "<nature of singers relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
]
"born" : "<(partial) datestamp>",
"died" : "<(partial) datestamp>",
"biography" : "<biographical summary>",
"source" : "<source of information about this singer>",
"photo_url" : "<url for singer's photo>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
}
Documents held in the Song2Version link table will have the following form:
{
"id" : "<opaque internal identifier>",
"song_id" : "<opaque identifier for the song>",
"version_id" : "<opaque identifier for the version>",
"created_date" : "<date this relationship was created>"
}
Documents held in the Singer2Version link table will have the following form:
{
"id" : "<opaque internal identifier>",
"singer_id" : "<opaque identifier for the singer>",
"version_id" : "<opaque identifier for the version>",
"created_date" : "<date this relationship was created>"
}
Documents held in the Song2Song link table will have the following form:
{
"id" : "<opaque internal identifier>",
"song_ids" : [
"<opaque identifier for the second song>",
"<opaque identifier for the first song>"
]
"created_date" : "<date this relationship was created>"
}
3 indices are required to service the various search requirements
- Song Index - metadata for the song, embedded list of versions with embedded singers
- Singer Index - metadata for the singer, embedded list of versions
- Version Index - metadata for the version and metadata for the singer and relationship to the song
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<canonical song title>",
"summary" : "<free text summary>",
"location": [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"time_period" : {
"from" : "<time period indicator>",
"to" : "<time period indicator>"
},
"composer" : "<free text name of composer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"order_by_title" : "<title used for sorting>",
"canonical_location" : {
"lat" : "<canonical latitude for the song>",
"lon" : "<canonical longitude for the song>",
},
"alternative_title" : [<list of alternative song titles>],
"relations" : [
{
"id" : "<opaque identifier for related song>",
"title" : "<canonical title of related song>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"versions" : [
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<title of this version>",
"alternative_title" : [<list of alternative titles>],
"summary" : "<free text summary>",
"language" : [<list of languages>],
"media_url" : [<list of media links>],
"lyrics" : "<the lyrics of the version>",
"photo_url" : "<a photo representing the version>",
"collector" : "<free text name of collector>",
"source" : "<free text source of material>",
"collected_date" : "<partial date for collection>",
"location" : [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"comments" : "<free text comments>",
"tags" : [<list of tags>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"singer" : {
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"name" : {
"first" : "<first name>",
"middle" : "<middle name or initial>",
"last" : "<surname>",
"aka" : [<nicknames and translations>]
},
"groups" : [<list of bands/choirs>],
"gender" : "<male|female>",
"location" : [
{
"relation" : "<nature of singers relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
]
"born" : "<(partial) datestamp>",
"died" : "<(partial) datestamp>",
"biography" : "<biographical summary>",
"photo_url" : "<url for singer's photo>",
"source" : "<source of information about this singer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_name" : "<canonicalised version of the singer name>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"born_date" : "<full datestamp>",
"died_date" : "<full datestamp>"
}
}
]
}
- canonical_location - canonical lat/long to be used for geographical queries. These will require ES type "geo_point"
- alternative_title - these are computed from the version titles and alternative titles
- relations - these are songs which are related in some way to the current song. We just include their title and their id, for convenience in display
- references - aggregate of reference numbers computed from the version references
- versions - a list of all of the song versions, computed from Song2Version, with each version containing an embedded singer, computed from Singer2Version. Note that this versions object is almost identical to the individual versions objects in the Versions Index
- In the "singer" field of the versions object we incorporate the canonical_name, canonical_location, born_date and died_date as simple computed fields
- Note that we also incorporate the canonical_location field for the version itself, which is a simple computed field
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"name" : {
"first" : "<first name>",
"middle" : "<middle name or initial>",
"last" : "<surname>",
"aka" : [<nicknames and translations>]
},
"groups" : [<list of bands/choirs>],
"gender" : "<male|female>",
"location" : [
{
"relation" : "<nature of singers relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
]
"born" : "<(partial) datestamp>",
"died" : "<(partial) datestamp>",
"biography" : "<biographical summary>",
"source" : "<source of information about this singer>",
"photo_url" : "<url for singer's photo>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_name" : "<canonicalised version of the singer name>",
"order_by_name" : "<form of name to calculate sorting on>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"born_date" : "<full datestamp>",
"died_date" : "<full datestamp>",
"versions" : [
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<title of this version>",
"alternative_title" : [<list of alternative titles>],
"summary" : "<free text summary>",
"language" : [<list of languages>],
"media_url" : [<list of media links>],
"lyrics" : "<the lyrics of the version>",
"photo_url" : "<a photo representing the version>",
"collector" : "<free text name of collector>",
"source" : "<free text source of material>",
"collected_date" : "<partial date for collection>",
"location" : [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"comments" : "<free text comments>",
"tags" : [<list of tags>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"song" : {
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<canonical song title>",
"summary" : "<free text summary>",
"location": [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"time_period" : {
"from" : "<time period indicator>",
"to" : "<time period indicator>"
},
"composer" : "<free text name of composer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_location" : {
"lat" : "<canonical latitude for the song>",
"lon" : "<canonical longitude for the song>",
},
"alternative_title" : [<list of alternative song titles>]
}
}
]
}
- canonical_name - a human readable name computed from the compound "name" field
- canonical_location - canonical lat/long to be used for geographical queries. These will require ES type "geo_point"
- born_date - full datestamp computed from the partial datestamp in "born" (e.g. if born says "1975", born_date says "01-01-1975")
- died_date - full datestamp computed from the partial datestamp in "died" (e.g. if died says "1975", died_date says "01-01-1975")
- versions - a list of all of the song versions, computed from Singer2Version, with each version containing an embedded song, computed from Song2Version. Note that this versions object is almost identical to the individual versions objects in the Versions Index
- Note that in the "song" field in the version object, we incorporate the canonical_location and the alternative_title as simple computed fields. We do not incorporate the more complex computed fields, as these are unnecessary for searching on this object.
- Note that we also incorporate the canonical_location field in the version itself, as a simple computed field
{
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<title of this version>",
"alternative_title" : [<list of alternative titles>],
"summary" : "<free text summary>",
"language" : [<list of languages>],
"media_url" : [<list of media links>],
"lyrics" : "<the lyrics of the version>",
"photo_url" : "<a photo representing the version>",
"collector" : "<free text name of collector>",
"source" : "<free text source of material>",
"collected_date" : "<datestamp of collection>",
"location" : [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"references" : [
{
"type" : "<reference type, e.g. ROUD>",
"number" : "<reference number>"
}
],
"comments" : "<free text comments>",
"tags" : [<list of tags>],
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"singer" : {
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"name" : {
"first" : "<first name>",
"middle" : "<middle name or initial>",
"last" : "<surname>",
"aka" : [<nicknames and translations>]
},
"groups" : [<list of bands/choirs>],
"gender" : "<male|female>",
"location" : [
{
"relation" : "<nature of singers relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
]
"born" : "<(partial) datestamp>",
"died" : "<(partial) datestamp>",
"biography" : "<biographical summary>",
"photo_url" : "<url for singer's photo>",
"source" : "<source of information about this singer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_name" : "<canonicalised version of the singer name>",
"canonical_location" : {
"lat" : "<canonical latitude for the singer>",
"lon" : "<canonical longitude for the singer>",
},
"born_date" : "<full datestamp>",
"died_date" : "<full datestamp>",
},
"song" : {
"id" : "<opaque internal identifier>",
"lv_id" : "<local voices identifier>",
"title" : "<canonical song title>",
"summary" : "<free text summary>",
"location": [
{
"relation" : "<nature of song's relationship to place>",
"lat" : "<latitude>",
"lon" : "<longitude>",
"name" : "<textual name of place>"
}
],
"time_period" : {
"from" : "<time period indicator>",
"to" : "<time period indicator>"
},
"composer" : "<free text name of composer>",
"created_date" : "<created date>",
"last_updated" : "<last updated date>",
"canonical_location" : {
"lat" : "<canonical latitude for the song>",
"lon" : "<canonical longitude for the song>",
},
"alternative_title" : [<list of alternative song titles>]
}
}
- canonical_location - canonical lat/long to be used for geographical queries. These will require ES type "geo_point"
- singer - record for the singer of this version of the song. Note that we incorporate the canonical_name, canonical_location, born_date and died_date as simple computed fields
- song - record for the song this is a version of. Note that we incorporate the canonical_location and the alternative_title as simple computed fields. We do not incorporate the more complex computed fields, as these are unnecessary for searching on this object.
1/ Add or update a song
- Add the song to the Song Store
- Compute any new relations to versions and any removed relationships. New relations should be added to Song2Version, redundant relationships should be removed
- Compute any new relations to other songs and any removed relationships. New relations should be added to Song2Song, redundant relationships should be removed
- Regenerate the Song Index for this song
- Regenerate the Song Index for all songs whose relationships were affected
- Regenerate the Version Index for all versions whose relationships were affected
- Regenerate the Singer Index for all singers whose versions were affected
2/ Add or update a singer
- Add the singer to the Singer Store
- Compute any new relations to versions and any removed relationships. New relations should be added to Singer2Version, redundant relationships should be removed
- Regenerate the Singer Index for this singer
- For each version referenced by the singer record, regenerate the Version Index document
- For each song associated with an affected version, regenerate the Song Index document
3/ Add or update a version
- Add the version to the Version Store
- Compute any new relations to songs and any removed relationships. New relations should be added to Song2Version, redundant relationships should be removed
- Compute any new relations to singers and any removed relationships. New relations should be added to Singer2Version, redundant relationships should be removed
- Regenerate the Version Index for this version
- Regenerate the Song Index for the related song
- Regenerate the Singer Index for the related singer
4/ Delete a song
- Delete all the relations from Song2Version
- Delete all the relations from Song2Song
- Delete the song from the Song Store
- Delete the song from the Song Index
- For each related song, update the Song Index document
- For each version, delete the version as per (6)
Note that deleting a song deletes all of the associated versions
5/ Delete a singer
- Delete all the relations from Singer2Version
- Delete the singer from the Singer Store
- Delete the singer from the Singer Index
- Update the Version Index for the affected version
- Update the Song Index for the song to which the affected version is related
Note that deleting a singer does not delete the song versions
6/ Delete a version
- Delete all the relations from Song2Version
- Delete all the relations from Singer2Version
- Delete the version from the Version Store
- Delete the version from the Version Index
- Update the Singer Index for the affected singer
- Update the Song Index for the affected song (remember the Song may have been deleted as per (4))
1/ Provide the main search API which satisfies search scenarios (1), (2) and (5) above
GET /search?<params>
The following URL params are permitted
- from_lat - upper-most latitude for search results
- to_lat - lower-most latitude for search results
- from_lon - left-most longitude for search results
- to_lon - right-most longitude for search results
- place - placename to search for
- q - free-text query string
- type - one or more of "singer", "song", "version" as a comma-delimitted list
- from - the result number to commence listing from. Defaults to 0, and is used for result set paging
- size - the size of the result set, and can be used to determine the "from" value for the next request when paging
- max - true/false to return all of the information about every object. Defaults to false. If True, will return just song title, id, and geo-location
This returns JSON representations of the objects that match, along with paging information
{
"from" : "<position of cursor in result set>",
"size" : "<requested number of records>",
"count" : "<number of records actually returned on this page>",
"total" : "<total number of results to the query (including those not listed on this page)>",
"results" : [
<results ordered by relevance>
]
}
If max is set to True you will receive the full object as it appears in the index layer. If max is set to False, you will receive the following information, depending on the result object type:
- id (all types)
- title (song and version)
- canonical_location (all types)
- canonical_name (singer)
- singer.id (version)
- singer.canonical_name (version)
Note also that in this API, each result will contain an "_type" element with one of "song", "singer", "version", as search may operate across all types, and the user of the API will need to be able to distinguish between the object types. For example
{
"_type" : "song",
... other song metadata ...
}
2/ Retrieve full details about a singer
GET /singer/<id>
- id - the system's internal identifier for the singer
This returns a full JSON representation of the singer, including all of the song variations they are responsible for, as held in the index (not the store)
3/ Retrieve full details about a song
GET /song/<id>
- id - the system's internal identifier for the song
This returns a full JSON representation of the song, including all of the song variations, as held in the index (not the store)
4/ Retrieve full details about a song
GET /version/<id>
- id - the system's internal identifier for the version
This returns a full JSON representation of the version, including the related song and singer, as held in the index (not the store)
5/ List the singers alphabetically
GET /singers?<params>
The following URL params are permitted:
- from - the result number to commence listing from. Defaults to 0, and is used for result set paging
- size - the size of the result set, and can be used to determine the "from" value for the next request when paging
- letter - the specific letter the singer's name should start with
Returns a JSON list of names and singer ids in alphabetical order, with the count of the song versions associated, along with associated paging information
{
"from" : "<position of cursor in result set>",
"size" : "<requested number of records>",
"count" : "<number of records actually returned on this page>",
"total" : "<total number of results to the query (including those not listed on this page)>",
"results" : [
{
"id" : "<opaque internal identifier for singer>",
"name" : "<canonical version of singer's name>",
"version_count" : "<count of the number of song versions associated>"
}
]
}
6/ List the songs alphabetically
GET /songs?<params>
The following URL params are permitted:
- from - the result number to commence listing from. Defaults to 0, and is used for result set paging
- size - the size of the result set, and can be used to determine the "from" value for the next request when paging
- letter - the specific letter the singer's name should start with
Returns a JSON list of names and song ids in alphabetical order, with the count of the song versions associated, along with associated paging information
{
"from" : "<position of cursor in result set>",
"size" : "<requested number of records>",
"count" : "<number of records actually returned on this page>",
"total" : "<total number of results to the query (including those not listed on this page)>",
"results" : [
{
"id" : "<opaque internal identifier for song>",
"name" : "<canonical version of song>",
"version_count" : "<count of the number of song versions associated>"
}
]
}
NOTE: this will only be developed if there is sufficient time
GET /ac/<field>?q=<search>
is the substring of the value to be autocompleted
The field can be one of the following:
-
place - returns a list of placenames
-
composer - returns list of composers that have previously been entered into the system
-
tag - returns a list of tags that have previously been entered into the system
[ "", "", ... ]
1/ Create a new singer
this operation also permits for the update of existing singers
POST /singers
<JSON representation of singer>
Returns the internal identifier of the singer created.
If the JSON representation of the singer contains an "id" field, then this will behave in the same way as (2).
The structure of the JSON representation is:
{
"id" : "<opaque singer identifier>",
"singer" : {
<singer object as per the storage model>
},
"versions" : [<list of opaque identifiers for versions>]
}
- If the "id" field is provided, this will overwrite an existing record with the same identifier
- If the "singer" field is provided, then the singer metadata will be set as this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about singers.
- If the "versions" field is provided, then the singer will be related to the provided versions
Therefore, you may use this API request in the following ways:
- Make a brand new singer, by just including the "singer" field
- Overwrite an existing singer, by including the "id" and "singer" fields
- Update an existing singer's versions by including the "id" and the "versions" fields
- Create a new singer and immediately relate them to existing versions by including the "singer" and "versions" fields
2/ Update an existing singer
that this API call is for REST-compliance, but the POST operation in (1) contains all the same functionality
PUT /singer/<id>
<JSON representation of singer>
- id - the internal identifier of the singer (as returned by the POST operation)
Does not return anything
The structure of the JSON representation is:
{
"singer" : {
<singer object as per the storage model>
},
"versions" : [<list of opaque identifiers for versions>]
}
- If the "singer" field is provided, then the existing singer metadata will overwritten with this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about singers.
- If the "versions" field is provided, then the existing version relationships will be overwritten with these new ones
Therefore, you may use this API request in the following ways:
- Overwrite an existing singer, by including "singer" field
- Update an existing singer's versions by including "versions" field
- Both of the above at the same time
3/ Delete a singer
NOTE: delete is quite destructive - we may choose to provide only soft-delete at this stage, or to omit delete altogether
DELETE /singer/</id>
- id - the internal identifier of the singer (as returned by the POST operation)
Does not return anything
4/ Create a new song
this operation also permits for the update of existing songs
POST /songs
<JSON representation of song>
Returns the internal identifier of the song created
If the JSON representation of the song contains an "id" field, then this will behave in the same way as (5).
The structure of the JSON representation is:
{
"id" : "<opaque song identifier>",
"song" : {
<song object as per the storage model>
},
"versions" : [<list of opaque identifiers for versions>],
"songs" : [<list of opaque identifiers for other songs>]
}
- If the "id" field is provided, this will overwrite an existing record with the same identifier
- If the "song" field is provided, then the song metadata will be set as this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about songs.
- If the "versions" field is provided, then the song will be related to the provided versions
- If the "songs" field is provided, then the song will be related to the provided songs
Therefore, you may use this API request in the following ways:
- Make a brand new song, by just including the "singer" field
- Overwrite an existing song, by including the "id" and "song" fields
- Update an existing song's versions or related songs by including the "id" and/or the "versions"/"songs" fields
- Create a new song and immediately relate them to existing versions and songs by including the "singer", "versions" and "songs" fields
5/ Update an existing song
that this API call is for REST-compliance, but the POST operation in (4) contains all the same functionality
PUT /song/<id>
<JSON representation of song>
- id - the internal identifier of the song (as returned by the POST operation)
Does not return anything
The structure of the JSON representation is:
{
"song" : {
<song object as per the storage model>
},
"versions" : [<list of opaque identifiers for versions>],
"songs" : [<list of opaque identifiers for other songs>]
}
- If the "song" field is provided, then the song metadata will overwritten with this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about songs.
- If the "versions" field is provided, then any existing versions relationships will be overwritten with these new ones
- If the "songs" field is provided, then any existing song relationships will be overwritten with these new ones
Therefore, you may use this API request in the following ways:
- Overwrite an existing song, by including "song" field
- Update an existing songs's versions or related songs by including the "versions" and "songs" fields
- Both of the above at the same time
6/ Delete a song
NOTE: delete is quite destructive - we may choose to provide only soft-delete at this stage, or to omit delete altogether
DELETE /song/</id>
- id - the internal identifier of the song (as returned by the POST operation)
Does not return anything
NOTE: delete of a song will result in the delete of all the versions associated with that song
7/ Create a new version
this operation also permits for the update of existing versions
POST /versions
<JSON representation of version>
Returns the internal identifier of the version created
If the JSON representation of the song contains an "id" field, then this will behave in the same way as (7).
The structure of the JSON representation is:
{
"id" : "<opaque version identifier>",
"version : {
<version object as per the storage model>
},
"song" : "<opaque identifier of song this is a version of>",
"singer" : "<opaque identifier of singer this was performed by>"
}
- If the "id" field is provided, this will overwrite an existing record with the same identifier
- If the "version" field is provided, then the version metadata will be set as this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about versions.
- If the "song" field is provided, then the version will be related to the provided song
- If the "singer" field is provided, then the version will be related to the provided singer
Therefore, you may use this API request in the following ways:
- Make a brand new version, by just including the "version" field
- Overwrite an existing version, by including the "id" and "version" fields
- Update an existing version's song relationship or singer relationship by including the "id" and/or the "song"/"singer" fields
- Create a new version and immediately relate it to an existing song and singer by including the "version", "song" and "singer" fields
8/ Update an existing version
that this API call is for REST-compliance, but the POST operation in (7) contains all the same functionality
PUT /version/<id>
<JSON representation of version>
- id - the internal identifier of the version (as returned by the POST operation)
Does not return anything
The structure of the JSON representation is:
{
"version : {
<version object as per the storage model>
},
"song" : "<opaque identifier of song this is a version of>",
"singer" : "<opaque identifier of singer this was performed by>"
}
- If the "version" field is provided, then the version metadata will be overwritten with this metadata. Note that this must be as per the storage model, NOT as per the index model. That is, only post the data about versions.
- If the "song" field is provided, then the version will be related to the provided song
- If the "singer" field is provided, then the song will be related to the provided singer
Therefore, you may use this API request in the following ways:
-
Make a brand new version, by just including the "version" field
-
Overwrite an existing version, by including the "id" and "version" fields
-
Update an existing version's song relationship or singer relationship by including the "id" and/or the "song"/"singer" fields
-
Create a new version and immediately relate it to an existing song and singer by including the "version", "song" and "singer" fields
-
Overwrite an existing version, by including "version" field
-
Update an existing version's song or singer by including the "song" and "singer" fields
-
Both of the above at the same time
9/ Delete a version
NOTE: delete is quite destructive - we may choose to provide only soft-delete at this stage, or to omit delete altogether
DELETE /version/</id>
- id - the internal identifier of the version (as returned by the POST operation)
Does not return anything
Geocoding: http://en.wikipedia.org/wiki/List_of_United_Kingdom_locations http://www.gazetteer.org.uk/purchase.php http://www.svincent.com/CrystalObelisk/Names/towns.txt http://www.nearby.org.uk/ https://developers.google.com/maps/documentation/geocoding/
Relational data in ES: http://www.elasticsearch.org/blog/managing-relations-inside-elasticsearch/