Interface to REST services for planetary imagery with the goal of affording the transfer of mission data into an openly available and homogeneous data store.
Documentation is available at: https://usgs-astrogeology.github.io/bayleef/
The CLI allows for the programatic searching, downloading and uploading (to a Postgres server) of USGS data.
The workflow is :
login -> search -> download -> upload
Before you can do anything, you must login using the CLI, logging in requires that you have a USGS EROS account https://ers.cr.usgs.gov/login/ and have authorization to download orders from the RESTful api.
bayleef login [user] [pass]
usgs search [dataset] --node [node] --start-date [start date] --end-date [end date] --longitude [lng] --latitude [lat] --dist [radius]
Returns a JSON string with containing result metadata. The JSON blob doesn't mean much in terms of human readability, but with the optional --geojson, the response is turned into GeoJSON which can be rendered to visualize footprints in a viewer (e.g. QGIS, Github Gist).
Breakdown:
- [node] : Name of the catalog to search in (e.g. cwic for CWIC/LSI Explorer, ee for Earth Explorer, ect.), see docs for a list of available nodes
- [dataset] : The specific collection to download from (e.g. LANDSAT_8_C1 for landsat 8 collection 1), see docs for a list of available datasets
- [start-data]/[end-data] : Dates when dataset are aquired in yyyy-mm-dd format
- [longitude]/[latitude] : lon/lat point the image footprint should intersect
- [radius] : radius in meters from the lat lon point, if --dist is passed in, any image with a footprint overlapping the circle will be accepted.
The search JSON response comes with a download URL. It would be trivial to download a individual image from the download string, but for users more interested in the shotgun approach of simply grabbing all the images that match a search criteria and dump them into a download directory, there is bayleef batch-download.
bayleef batch-download accepts the JSON string returned from bayleef search and mass downloads all the images listed and transforms them into a common format (GeoTIFFs). For example, you can run a spatiotemporal search over Hawaii, and pipe the resulting JSON blob into batch-download.
bayleef search [args] | bayleef batch-download [root] [node]
Breakdown:
- [args] : the same args you use in
search, as the results define what is downloaded - [root] : Download location. For each dataset, bayleef has a predefined path structure. For example, Landsat 8 is
[year]/[row]/[path]/id.iff. So you want to reuse the same root directory for every download operation. - [node] : Name of the catalog to search in (e.g. cwic for CWIC/LSI Explorer, ee for Earth Explorer, ect.), see docs for a list of available nodes
** NOTE ** Each dataset requires a custom ingestion function for batch-downloading, so check the docs for a list of datasets available for download.
Once the files are downloaded into a homogenous structured format using batch-download, they need to be uploaded into a PostGIS database for querying and processing. All tables are stored into schemas matching the dataset name.
bayleef to-sql --host [HOST] --port [PORT] --user [USER] --password [PASS] [DATASET] [ROOT] [DB]
Breakdown:
- [HOST]/[PORT]/[USER]/[PASSWORD] : Conenction and login info for the PostgreSQL instance
- [dataset] : The specific collection to download from (e.g. LANDSAT_8_C1 for landsat 8 collection 1), see docs for a list of available datasets
- [ROOT] : Should be the same as the root passed into
batch-download, the location of the downloaded files. - [DB] Name of the database to upload to.
# Get night time Landsat 8 images given a bounding box and time range
bayleef search LANDSAT_8_C1 --lower-left -165 15 --upper-right -150.0 28.0 --start-date 2013-04-01 --end-date 2013-05-1 where 20513 Night --geojson
