Fetch, read, and process ground motion amplitude data.

shakemap-amp-tools is a project designed to provide a number of functions ancillary to the ShakeMap project, mostly those having to do with finding and parsing strong ground motion data and converting it into the XML data format used by ShakeMap. This repository includes the following tools:

• amps2xml Convert peak ground motions in Excel format to ShakeMap XML data structure.
• fdsnfetch Retrieve data from an fdsn client (i.e. IRIS and ORFEUS).
• ftpfetch Retrieve directories or files containing data from FTP url.
• ingvfetch Retrieve ShakeMap xml data from the INGV Strong Motions Web Service
• sm2xml Convert a directory of strong motion data files into ShakeMap xml.

Conda install:

• conda install amptools

Automated install:

• git clone https://github.com/usgs/shakemap-amp-tools.git
• cd shakemap-amp-tools
• bash intall.sh
• conda activate amptools

Manual install:

• conda install numpy
• conda install pandas
• conda install openpyxl
• conda install lxml
• pip install git+https://github.com/usgs/shakemap-amp-tools.git

• pip install numpy
• pip install pandas
• pip install openpyxl
• pip install lxml
• pip install git+https://github.com/usgs/shakemap-amp-tools.git

Updating conda install:

• conda update amptools

Updating automated install:

• cd shakemap-amp-tools
• git pull --ff-only https://github.com/usgs/shakemap-amp-tools.git master
• bash install.sh

Updating manually installed:

• pip install --upgrade git+https://github.com/usgs/shakemap-amp-tools.git

This tool converts a directory of ground motion time series data into an xml file formatted for ShakeMap. The input file must be in a format supported by amptools:

• Cosmos

  • Volume 1 and 2 are supported. No preprocessing applied.

• CWB

  • Provided by Taiwan Central Weather Bureau. No preprocessing applied.

• DMG

  • Synonymous with CSMIP. No preprocessing applied.

• Geonet

  • Provided by New Zealand GNS. Currently Volume 1 and 2 are supported.
  • Data is converted from mm/s2 to cm/s2 when read in.

• KNET and KIKNET

  • Provided by Japan National Research Institute for Earth Science and Disaster Resilience.
  • Time is converted from Japanese local time to UTC time (9 hour time shift).
  • A 15 second time delay (added by logger) is removed.
  • The data is detrended (linear and demean) as specified by documentation.

• Obspy

  • All obspy supported formats are acceptable when accompanied by a JSON file containing metadata.

• SMC

  • Standard U.S.G.S. format specified by the National Strong Motion Project. No preprocessing applied.

• USC

  • Provided by Los Angeles Basin Seismic Network, University of Southern California. No preprocessing applied.
  • Volume 1 is supported. No preprocessing applied.

This tool allows a user to fetch raw waveforms from an fdsn client. Clipped waveforms are automatically removed. Each waveform is also demeaned and the instrument response is removed. Data is output as miniseed or sac, along with an accompanying metadata file for each station.

This tool allows a user to fetch a file or directory of files from an FTP server. If a username and password are required (KNET, for example) they can be provided as command line options.

This tool allows a user to fetch a ShakeMap xml file from the INGV Strong Motions Web Service. EventIDs from any of the following catalogs will be converted to an INGV eventID and will be used to retrieve a ShakeMap xml file if one is available:

• UNID
• EMSC
• USGS
• ISC

The INGV catalog is a combination of the CNT and ESM catalogs. With an INGV eventID, priority is given to ESM. In order to specify CNT or ESM, the catalog specific eventID and specifying the source catalog is required.

This tool converts an excel file of peak ground motion data into an xml file formatted for use by ShakeMap. Excel spreadsheets must contain only tabular (rows/columns) data.

A "complete" peak ground motion spreadsheet would look like this:

A minimum intensity only peak ground motion table would look like this:

A minimum pga only peak ground motion table would look like this:

The reference section in the first row is mandatory, and should consist of a cell in the first column with "reference" in it, and the cell in the second column containing the reference string.

The required columns are:

• station: Usually a short abbreviated code.
• lat: Latitude of the station where data was collected.
• lon: Longitude of the station where data was collected.
• network: Usually a short code indicating the seismic network who collected the data.

In addition to these required columns, there must be additionally at least:

• intensity: Modified Mercalli Intensity (MMI) at the station location.

AND/OR:

• channel (more on this below), containing at least one of the following:
  • pga: Peak Ground Acceleration, in units of %g.
  • pgv: Peak Ground Velocity, in units of cm/s.
  • psa03: Peak Spectral Acceleration at 0.3 seconds, in units of %g.
  • psa10: Peak Spectral Acceleration at 1.0 seconds, in units of %g.
  • psa30: Peak Spectral Acceleration at 3.0 seconds, in units of %g.

The channel columns can be one of three different naming schemes:

• XXE, XXN, XXZ: Three channels named according to the scheme found at http://www.fdsn.org/seed_manual/SEEDManual_V2.4_Appendix-A.pdf where XXE/XXN are horizontal channels, and XXZ is the vertical channels.
• H1, H2, Z: Three channels where the direction of the channel is not known, where H1/H2 are horizontal channels and Z is the vertical channel.
• UNK: The "channel" to use when only summary peak ground motion values are known.

For the "XXE/XXN/XXZ" or "H1/H2/Z" naming schemes, at least one of the horizontal channels must be included.

Links to example spreadsheets:

• Complete
• Intensity only
• PGA only

The data file readers are modeled after obspy file readers, and have a standard interface.

Data file readers are located in amptools/io/[format]/core.py.

This core.py module should take the following form:

def is_format(filename):
    # code to examine candidate file and determine if it is of the type specified.
    # return True if file is correct type, False otherwise.

def read_format(filename,**kwargs):
    # code to read file and return an obspy Stream object.

In order to add an intensity measurement types (IMT) calculation, add a file with the type as the file name under pgm/imt/. The calculation should be called calculate_[TYPE], where [TYPE] matches the file name. The argument should always be stream. The second function should always be imcs (a list of requested components). All other functions in the file should be hidden by an underscore (example def _get_horizontals(stream)). The calculate function should return a dictionary of each component and the resulting values. Example:

{
    'HN2': 81.234,
    'GMROTD0.0': 86.784,
    'GMROTD100.0': 96.446,
    'HN1': 99.249,
    'HNZ': 183.772,
    'GMROTD50.0': 92.177,
    'GREATER_OF_TWO_HORIZONTALS': 99.249
}

StationSummary should be updated to handle the new IMT in gather_pgms.

In order to add an intensity measurement component (IMC) calculation, add a file with the component as the file name under pgm/imc/. The calculation should be called calculate_[COMPONENT], where [COMPONENT] matches the file name. The argument should always be stream. All other functions in the file should be hidden by an underscore (example def _get_horizontals(stream)). The calculate function should return a single value or a dictionary of each component and the resulting values. Imt calculations should be updated to handle a dictionary if one is returned. Otherwise, single values will automatically be handled.

Greater of two horizontals example: 99.249

Channels example:

{
    'HN1': 99.249,
    'HN2': 81.234,
    'HNZ': 183.772
}

GMRotD example:

{
    0.0: 103.299,
    50.0: 119.925,
    100.0: 125.406
}