This document summarizes the current state of the Automated RecoGnition Of Species (ARGOS) processing pipeline.

The ARGOS systems presumes that data is stored according to a specific file structure. The config.py file defines the ARGOS_ROOT variable, which specifies the location of the top of the ARGOS file system. This allows the user to easily run on multiple systems. All data is then arranged according to the following schema:

ARGOS_ROOT/year/month/day/site_name/obliques
ARGOS_ROOT/year/month/day/site_name/altitude/images/...
ARGOS_ROOT/year/month/day/site_name/altitude/maps/...

The obliques folder contains all "oblique" images and videos captured for a given site, typically from a variety of altitudes. The images/ directory contains all high resolution images captured at the given site at the specified altitude. The maps/ directory contains the high-resolution georeferenced map file, map.tif, as well as a lower-resolution map called map_small.jpg. As additional species maps are generated for the flight they will be written to the corresponding maps/ directory.

As a concrete example, here is the location of the first image taken during the St. John's 66 ft flight on 03 August 2018:

ARGOS_ROOT/2018/08/03/st_johns_marsh/66/images/DJI_0001.JPG

And the corresponding georeferenced map can be found at:

ARGOS_ROOT/2018/08/03/st_johns_marsh/66/maps/map.tif

Note that by convention all site names are lowercase and contain only letters, numbers, and underscores; punctuation (including hyphens) and spaces are verboten.

ARGOS relies on a locally-hosted MongoDB database. The database is named ARGOS, and it contains four collections:

targets
ground_truth
annotations
imagery

The targets collection stores descriptions of all valid Annotation Targets (see below). The ground_truth collection stores all field-collected, georeferenced ground truth. The annotations organizes all manual tile annotations made via the user interface. Finally, the imagery collection keeps track of the high-resolution images, and their approximate locations (in latitude and longitude).

Note also that when the database module is imported, targets, ground_truth, and a so-called truth_tree are loaded into memory. (The truth_tree is a ball tree, a metric tree that allows for very fast searches over points in multidimensional spaces, and which dramatically speeds up the association of ground truth points with maps and images.) These objects are used throughout the ARGOS software, and—particularly for the computation of the truth tree—pre-computing these objects can save a lot of time.

ARGOS is concerned with producing maps. We start with two primary maps, generated by stitching together individual drone images. The first is a 2 inch/pixel resolution map in TIF format that is fully georeferenced; this map is used for building the detailed target species heat maps, etc. The second is a 20 inch/pixel resolution map in JPG format. This is used for web-display interfaces. As described above, all maps are stored in the directory corresponding to the date, site, and altitude of the flight.

Annotation targets include plant species that the ARGOS system will learn to classify, such as Frangula alnus, but also non-biological features such as sand, water, rocks, and man-made features such as asphalt.

Viable Annotation Targets (ATs) are stored in the MongoDB database in the targets collection. They are available via the database.py module by calling get_targets(). The AT objects contain six fields: scientific_name, codes, common_name, physiognomy, category, and color_code.

As an example:

{
    "scientific_name": "Achillea millifolium",
    "codes": ["AM"],
    "common_name": "Yarrow",
    "physiognomy": "Forb",
    "category": "Native",
    "color_code": "#0cb577"
}

Categories include ['Invasive', 'Native', 'Natural Feature', 'Man-made Feature']. Possible values for physiognomy include ['Forb', 'Graminoid', 'Shrub', 'Tree', 'N/A']. Codes are used to associate collected ground truth with a target. Color codes determine the color used during image annotation, as well as the color used to display ground truth points on maps and high-resolution images.

In order to ingest annotation targets into the database, run:

> python ingest.py

This will clear the database of existing ATs and insert new ones, as defined in the ingest.py and in the truth/target_key.xlsx file. It will also scan the file structure for images that have not yet been ingested. As it finds new images, it will extract metadata, create an image object, and insert it into the database. The ingest script will not delete the contents of the imagery collection because metadata extraction is a relatively expensive process (~2 minutes/flight).

As indicated above, images are stored at the lowest level in the ARGOS directory structure. Individual image information, including geolocation information, is also stored in the MongoDB in the imagery collection.

A list of nearby images can be obtained via a GET request to /images/:map_id/?row=0.5&col=0.75, where the row and col query parameters specify the point of interest in terms of the fractional width (col) and height (row) of the map. A list of the ten nearest images are returned.

Field collected ground truth is also loaded into the database via the ingest.py script. The ingest.py script will load all the shape files available in the files located in the truth/ directory. These are inserted into the database, and have the following format:

{
    "geolocation": [-83.48602, 43.70615, 137.464798],
    "latlon": [43.70615, -83.48602],
    "name": "LS 08-06-2018 10:26:53",
    "code": "LS",
    "symbol": "Flag, Blue",
    "datetime": "2018-08-06T14:26:48Z"
}

Ground truth needs to be mapped to images and maps in order to dislpay the location of important plants and features. In the case of georeferenced maps, this can be done using the place_ground_truth_on_map function, in the ground_truth_mapping.py module.

The ARGOS API provides access to images, etc. We define the following endpoints:

The /maps endpoint returns an array of map objects, as defined above.

This endpoint returns a specific map object, as well as the associated ground truth information and map image size:

{
    "map": target_map_object,
    "ground_truth": nearby_ground_truth_list,
    "unique_truth": unique_ground_truth_list,
    "image_rows": 450,
    "image_cols": 658
}