Skip to content

Hydrology Processor is used to process hydrochemistry/hydrology data

Notifications You must be signed in to change notification settings

kendall-s/HyPro2_py

Repository files navigation

HyPro

Please note this is a very rough version, originally built without allocation and for processing underway hydrochemitry data. It was built upon to become a more full-featured HyPro.

HyPro is an application used to interactively process hydrochemistry data.

Features

  • Read in nutrient, dissolved oxygen, salinity, log sheets
  • Provide interactive UI for processing and QC'ing data
  • Plotting functionality with SBE CTD data
  • Collation and packaging of data into usable export formats
  • Stores processed data locally in a SQLite database file

Running HyPro

Setup a Python environment using the included requirements.txt, there is not anything complex in the environment so it should not have any issues being created. If there is issue try running conda and using the environment.yml file instead.

Run the MainMenu.py file from the project root to open HyPro.

Building an .exe

Pyinstaller is included in the environment and is used to generate a .exe for HyPro. Included in the root of the project is a .spec file. Pyinstaller uses the .spec file as configuration for the .exe build.

In a terminal at the root of the project run: pyinstaller ./HyPro_py.spec

A dist folder will be created and contain the built .exe

Processing

HyPro has two main windows to control functionality. The main menu allows projects to be created, loaded and imported. A project is a container for a body of work, i.e. a voyage or survey. Each project allows you to set project-specific setings and parameters for data processing.

The second main window is the processing window, this provides the functionality for reading, processing and viewing data.

Project folder structure

project_folder
∟ 📄 project.hypro
∟ 📄 project.db
∟ 📄 projectParams.json
∟ 📁 CTD
    ∟ 📄 projectCTDFile.ROS
∟ 📁 Nutrients
    ∟ 📄 projectNutrientFile001.SLK
    ∟ 📄 projectNutrientFile001.CHD
∟ 📁 Oxygen
    ∟ 📄 projectOxygenFile001.LST
∟ 📁 Salinity
    ∟ 📄 projectSalinityFile001.XLSX
∟ 📁 Sampling
    ∟ 📄 projectSampleFile001.XLSX

The project folder contains some files that keep hold of the project metadata and processed data. The folders are used to place the raw data files to be processed from each of the instruments.

CTD

CTD data is ingested using the ROS format output by Seasave. This file format includes the recorded CTD data from when bottles are fired. The generation of the file is optional within Seasave, so ensure these files are being produced.

Nutrients

This folder contains both the SLK and CHD nutrient data files. After processing, additional folders are created that contain the processing data and plots.

Oxygen

Oxygen measurements are processed from the LST format that is generated by SCRIPPS automated titrator. Ensure that only 1 header is present in the LST file.

Salinity

The Salinity measurements are pulled from the Excel export that the salinometer software generates.

Sample Logsheet

Samples collected from the CTD can be input into Hypro using this data format. It is a simple XLSX in the following structure. Headings (match order): rosette position, oxygen bottle labels, oxygen draw temp., salinity bottle labels and nutrient labels. Each row corresponds to a rosette position. Deployment number is generated using the file name and the number associated.

Accessing and inspecting project database files

The table structure is incredibly simplistic and is mostly de-normalised. This decision was made such that data kept within the database can be easily understood and inspected by people unfamiliar with relational databases. This decision would hopefully reduce any reliance from hydrochemists on software engineering staff.

A free tool, SQLite browser, can be downloaded and used to directly open the database file. With this tool, the data can be viewed and edited in a similar fashion to an Excel file with multiple sheets.

About

Hydrology Processor is used to process hydrochemistry/hydrology data

Resources

Stars

Watchers

Forks

Packages

No packages published