Software for sensor devices.

• Measure uniformity of growth.
• Provide curve of growth in time.

• Algorithm is plant-specific.
• Plant in tray is known.
• Measurements across different plant species are not comparable.
• Day of growth is known. Starting day and hour must be known.
• White, constant light.
• Substrate is white or black and plastic.
• Distance and angle are fixed.

• Definition of architectural aspects (image storage, JSON, etc.)
• Design enclosing case
• Hardware specification document
• Design and implement controls on web interface
• Algorithms: segmentation, analysis, calibration
• Commented source code
• Operation guide

Keep record of these (info.sh):

• SD serial
• SD cid
• Pi serial
• eth0 MAC address
• usb0 MAC address
• wlan0 MAC address

DHCP on usb0, eth0 and wlan0.

Change via script:

• Hostname / hosts file (change_hostname.sh)
• Wifi SSID and password (wifi.py)

• List all pictures on S3
• Check if local file exists
• Download pictures that are not here yet
• Group images with the same hostname and batch identifier
• Open downloaded images
• Cython segmentation function
• Optionally download images from S3
• Display multiple images with the discarded areas, for debugging purposes
• Stable biomass bounding box (also useful for cropping)
• Unit tests, integration tests
• Modularize the image processing pipeline
• Test median blur and blur
• Segmentation: model hole color and dead leaves color
• Segmentation: model substrate color
• Holes: consider their hierarchy
• Segmentation: multi-dimensional k-means clustering as a tool
• Lab, Luv, YUV color spaces
• Superpixels
• Probability map instead of binary mask: likelihood of a pixel to belong to a class based on color and spatial location (neighborhood); blur to diffuse probability
• Find biomass dominant tone based on saturation
• Detect if scene is dark
• Hires image and charts
• Find biomass segmentation algorithm params automatically
• Upload MQTT
• Detect if scene is not static (motion detection)
• Substrate-based uniformity algorithm
• Store custom group name
• DrawSmoothChart() and DrawChart() do not draw phantom point at time zero
• Series --> batch
• MQTT only for group zero
• Limit spline propagation experimented in branch spline-parts-and-spline-until-break
• Hours of operation shown and set on web interface, per sensor
• Plant species and group as dip.py argument (in batch name)
• Use batch_species and legacy functions to select special algorithms
• Web chart: variable ticks, fixed 15 days period
• Download no longer downloads also .csv files
• Make upload of csv optional
• Web output of processing (local network and Internet)
• List and link all available sensors and batches (including previous) *.csv for one group
• Directories prior/, downloaded/, timelapse/, CSV/ and possibly other dirs need group string
• Create group subdir under downloaded/, prior/, CSV/, timelapse/
• Prior knowledge is stored
• Preview to set focus and set view
• Launch dip.py on server with cron
• Link chart back to control panel
• A big chunk of zero.py (batch_start, minutes...) is general and should be taken out
• Send mail notifications
• Default, generic routine with just motion and brightness (to replace RoutineDisplay as default)
• Possible issue: Download from S3 error 'The read operation timed out'
• Possible issue: browser caches local csv file and does not update it even when refreshing
• Web chart: add URL parameter to highlight a specific moment
• Thumbnail size can be specified in URL (useful on local network)
• Comparison with average species curves
• Store all frames on sensor until end of batch (useful for processing on sensor)
• Create summary timeline with thumbnails
• Control panel and chart in same web page
• 'Ping' via HTTP request
• Buttons
• Make console log available on the web
• Check valid hours; always on mode if scene is lit
• Image processing on sensor
• Timelapse with and without superimposed graphics, but only if there is at least one more frame processed
• Windowless cap and dip (windowed is a command line option)
• Hardware check based on MAC addresses and serials (see info.sh)
• Background subtraction
• Perspective warping for perspective invariant areas
• Servers list in git: an http request containing image name is done by cap.py until a server responds positively (web server in dip.py, or calling dip.py)
• Mask
• Textural information: spatial frequency function pixel by pixel, textons and their neighborhood histograms; Sobel, Scharr and Laplacian on these channels separately: saturation, brightness, luminance + Canny (blurred)
• Retain pixel probability from previous frames (running average)
• Segmentation: Otsu thresholding and adaptive thresholding
• Color analysis inside biomass: histograms
• Holes: pixel-by-pixel

• Set shutter speed
• Print all camera properties
• Save/Load camera properties
• Arrow keys select property and change its value
• Get/Set camera property value by name
• Set optimal camera properties programmatically
• Preview on/off
• Save image locally, timely
• Alternatively show a zoomed full resolution image (useful for focusing) or just a convenient thumbnail
• Add uptime to EXIF
• Add source code version to EXIF
• Print source code version when TAB is pressed
• Print date and time the process started (plus uptime) when TAB is pressed
• Add all camera properties to EXIF
• Determine best resolution for both camera versions
• Manage network failures (upload queue)
• Upload image to S3
• Decide whether to use same resolution for both models
• Detect camera hardware version
• Detect camera presence
• Unzoom when taking picture
• Disable preview automatically (when taking picture)
• Start cap.py at boot
• Update git repo, relaunch cap.py, reboot and shutdown remotely via script on the 4 sensors at once
• Integrate microphone
• Sensor color calibration == Find camera parameters automatically
• Web interface: number of images in queue
• Print 24 errors (with the average squared error)
• Web interface
• Click 4 times instead of 24, perspective grid
• Store date and time of crop start
• Send sensor health warning (if something is wrong among cpu, memory, disk, temperature, camera properties, calibration...)
• Compare camera properties that are a result of color calibration and auto calibration across multiple sensors. It seems like shutter speed should be at least 5000. Probably find some custom target RGB values with new colorcheckers.
• Search brightness and contrast values (after the gains are good). Or make sure brightness is 50 and contrast is 10 and so on.
• Further test the exposure metering modes
• Possibly freeze awb but not iso, expo, shutter
• Camera on/off (250mA, no continuous mode)

# lsblk to list devices
# partition(s) must be unmounted (with umount; partitions, not the disk itself)
sudo dd if=/dev/mmcblk0 of=sdimage.img bs=4096 conv=notrunc  # if=/dev/sdb bs=4M

# lsblk to list devices
# partition(s) must be unmounted (with umount; partitions, not the disk itself)
# be very careful with what follows, see https://www.raspberrypi.org/documentation/installation/installing-images/linux.md
sudo dd if=sdimage.img of=/dev/mmcblk0 bs=4096 conv=notrunc  # of=/dev/sdb bs=4M

sudo kill -USR1 $(pgrep ^dd)

https://chrome.google.com/webstore/detail/vnc®-viewer-for-google-ch/iabmpiboiopbgfabjmgeedhcmjenhbla

sshfs pi@****.ddns.net:basil pi

fusermount -u pi

sshpass -p **** ssh pi@****.ddns.net 'bash -s' < do.sh
sshpass -p **** ssh pi@192.168.1.*** 'cd /home/pi/basil ; git pull ; sudo /sbin/shutdown now'

blueshift	eth: 192.168.0.6	wifi: DHCP	Pi NoIR Camera V2
redshift	eth: 192.168.0.7	wifi: DHCP	Pi NoIR Camera V2
noir		eth: 192.168.0.8	wifi: DHCP	Pi NoIR Camera V2
visible		eth: 192.168.0.9	wifi: DHCP	Pi Camera Module V2

Focus set at 350mm.

To look for local sensors:

sudo nmap -sP 192.168.1.0/24 | awk '/^Nmap/{ip=$NF}/B8:27:EB/{print ip}'

Resolution set to 1280x720 (16:9)

Append @/home/pi/basil/autostart.sh to /home/pi/.config/lxsession/LXDE-pi/autostart

# 7 fps, other options include hd720, hd480
ffmpeg -r 7 -pattern_type glob -i '*.jpeg' -s hd1080 -vcodec libx264 timelapse.mp4

# with crop w:h:x:y
ffmpeg -r 7 -pattern_type glob -i '*.jpeg' -s hd1080 -vcodec libx264 -filter:v "crop=1050:871:857:776" timelapse.mp4

exiftool -keywords <filename>

Save username and password for 4 months:

git config --global credential.helper 'cache --timeout=10000000'

pip install Cython
sudo apt-get install python-scipy
pip install paho-mqtt
sudo apt-get install ffmpeg

pip install twisted
sudo pip install service_identity
pip install service_identity

The last command fails but then Twisted is happy anyway.

• Initial: Needs light. Must be done every time the sensor is turned on.
• Auto: Needs light + scene.
• Color: Needs light + colorchecker. Can provide final error(s).

Edit /etc/rc.local and add the following lines above exit 0:

# Disable HDMI
/usr/bin/tvservice -o

Other interesting settings in /boot/config.txt.

git update-index --assume-unchanged FILE

Must be executed on each machine. That means this is part of the installation process.