Skip to content

PacificCommunity/ckanext-spc-pdh

Repository files navigation

ckanext-spc

Requirements

Installation

To install ckanext-spc:

  1. Activate your CKAN virtual environment, for example:

    . /usr/lib/ckan/default/bin/activate
  2. Install the ckanext-spc and its requirements into your virtual environment:

    pip install ckanext-spc
    pip install -r dev-requirements.txt
  3. Add spc to the ckan.plugins setting in your CKAN config file (by default the config file is located at /etc/ckan/default/production.ini).
  4. For the SpcNadaHarvester to work, install this branch of ckanext-ddi: github.com/roly97/ckanext-ddi/tree/nada_harvester. Alternatively, install the original ckanext-ddi extension github.com/liip/ckanext-ddi and then replace the files ddiharvester.py and metadata.py with the changed files found in ckanext-ddi_changes directory in this repo.
  5. Update SOLR schema:

    <field name="topic" type="string" indexed="true" stored="true" multiValued="true"/>
  6. Update DB schema:

    paster spc db-upgrade -c config.ini
  7. Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:

    sudo service apache2 reload

Config Settings

Config options:

scheming.dataset_schemas = ckanext.spc.schemas:dataset.json

Development Installation

To install ckanext-spc for development, activate your CKAN virtualenv and do:

git clone https://github.com/DataShades/ckanext-spc.git
cd ckanext-spc
python setup.py develop
pip install -r dev-requirements.txt
paster spc db-upgrade -c config.ini

Template macros

This extension provides new template macro _nested_field which is used for rendering fields in nested schemas. It can be easily added to any page that have access to dataset metadata via next snippet:

{{ nested_field(parent_field_name, index_of_current_field_amoung_other_siblings, current_field_name, parent_data_dict, parent_errors_dict) }}

More examples can be found in templates/macro/spc.html.

SPC specific examples

Recently updated dataset(within thematic area)::

http://ckan.url/api/action/package_search?q=extras_thematic_area_string:%22Climate%20Change%22&sort=metadata_modified+desc

Most popular dataset(within thematic area)::

http://site.url/api/action/package_search?q=extras_thematic_area_string:%22Climate%20Change%22&sort=extras_ga_view_count+desc

Interactive widgets

Develeopment server with the examples started via:

npm run dev-widgets

When development phase completed, build widgets with:

npm run build-widgets

Widgets registered inside SPCWidgets global JS variable. Use them form your CKAN JS modules as following:

var widget = new SPCWidgets.SearchForm({target: document.querySelector(selector)})

Search form

Running the Tests

To run the tests, do:

pytest --ckan-ini test.ini ckanext/spc

General Information

Harvesters

OAI-PMH

  • Name used in config: spc_oaipmh_harvester
  • Dataset type: publications
  • Sources:
Url Title Settings Organization
http://www.spc.int/DigitalLibrary/SPC/OAI Social Development Program {"set": "SDP_PDH", "topic": "Gender and Youth"}
spc-sdp
http://www.spc.int/DigitalLibrary/SPC/OAI Climate Change and Environmental Sustainability {"set": "CCES_PDH", "topic": "Climate Change"} spc-cces
http://www.spc.int/DigitalLibrary/SPC/OAI Fisheries, Aquaculture & Marine Ecosystems {"set": "FAME_PDH", "topic": "Fisheries"} spc-fame
http://www.spc.int/DigitalLibrary/SPC/OAI Geoscience, Energy and Maritime {"set": "GEM_PDH", "topic": "Geoscience"} spc-gem
http://www.spc.int/DigitalLibrary/SPC/OAI Land Resources Division {"set": "LRD_PDH", "topic": "Land Resources"} spc-lrd
http://www.spc.int/DigitalLibrary/SPC/OAI Public Health Division {"set": "PHD_PDH", "topic": "Health"} spc-phd
http://www.spc.int/DigitalLibrary/SPC/OAI Statistics for Development Division {"set": "SDD_PDH", "topic": "Official Statistics "} | spc-sdd

DKAN

  • Name used in config: spc_dkan_harvester
  • Dataset type: dataset
  • Sources:
Url Title Settings Organization

GBIF

  • Name used in config: spc_gbif_harvester
  • Dataset type: biodiversity_data
  • Sources:
Url Title Settings Organization
http://api.gbif.org GBIF SPREP published {"topic": "Fisheries", "hosting_org": "cd3512e7-886c-4873-b629-740abe8ae74e", "q": "-spc"} sprep
http://api.gbif.org GBIF SPC published {"topic": "Fisheries", "hosting_org": "cd3512e7-886c-4873-b629-740abe8ae74e", "q": "+spc"} spc-fame

PRDR Publications Harvester

  • Name used in config: spc_prdr_publications_harvester
  • Dataset type: publications
  • Sources:
Url Title Settings Organization
https://prdr-dev.spc.links.com.au/api/action/publications_list SPC PRDR Publications {"topic": "Energy"} spc-gem

PRDR Data(energy-resource) Harvester

  • Name used in config: spc_prdr_res_energy_harvester
  • Dataset type: dataset
  • Sources:
Url Title Settings Organization
https://prdr-dev.spc.links.com.au/api/action/energy_resources_list SPC PRDR Data {"topic": "Energy"} spc-gem

SPREP

  • Name used in config: spc_sprep_harvester
  • Dataset type: dataset
  • Sources:
Url Title Settings Organization

https://pacific-data.sprep.org

Inform Regional Data Portal {"topic_mapping": {"Atmosphere and Climate": "Climate Change", "Info": null, "Land": "Land Resources", "Biodiversity": "Fisheries", "Build Environment": "Economic Development", "Coastal and Marine": "Fisheries", "Culture and Heritage": "Gender and Youth", "Inland Waters": "Geoscience"}}

sprep

Datasets types

  • Biodiversity data - EML schema
This Dataset type has multiple fields which includes subfields:
  • Creator
  • Metadata Provider
  • Associated Party
  • Keyword Set
  • Coverage
  • Maintenance
  • Contact
  • Methods
  • Project
  • Dataset - DCAT schema
  • Geographic data - ANZLIC schema
  • Publications - Dublin Core schema

CKAN Patches

curl https://patch-diff.githubusercontent.com/raw/ckan/ckan/pull/5546.diff | git apply curl https://patch-diff.githubusercontent.com/raw/ckan/ckan/pull/5594.diff | git apply

About

CKAN repo for the Pacific Data Hub, an open data portal for datasets from the Pacific.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published