dfatool is a set of utilities for automated performance model generation. It supports a wide range of application domains, including

• unattened energy measurements and energy model generation for embedded peripherals,
• unattended performance (“non-functional property”) measurements and NFP model generation for software product lines and SPL-like software projects, and
• data analysis and performance model generation for arbitrary data sets out of text-based log files.

Measurements and models for peripherals (generate-dfa-benchmark.py and analyze-archive.py) generally focus on energy and timing behaviour expressed as a Priced Timed Automaton (PTA) with Regression Model Trees (RMT).

Measurements and models for software product lines (explore-kconfig.py and analyze-kconfig.py) focus on ROM/RAM usage and may also include attributes such as throughput, latency, or energy. The variability model of the software product line must be expressed in the Kconfig language. Generated models can be used with kconfig-webconf. This allows for Retrofitting Performance Models onto Kconfig-based Software Product Lines.

Models for arbitrary other kinds of configurable components (analyze-log.py) rely on logfiles that contain "[::] Key Attribute | parameters | NFP values" lines. Here, only analysis and model generation are automated, and users have to generate the logfiles by themselves.

The name dfatool comes from the fact that benchmark generation for embedded peripherals relies on a deterministic finite automaton (DFA) that specifies the peripheral's behaviour (i.e., states and transitions caused by driver functions or signalled by interrupts). It is meaningless in the context of software product lines and other configurable components.

The remainder of this README references domain-specific data acquisition and model generation how-tos as well as dependencies and global options and environment variables.

• Measuring non-functional properties ("performance attributes") of software product lines

Legacy documentation; may be outdated:

• Energy Benchmarks with Kratos (DE)
• Energy Benchmarks with Multipass (DE)
• Performance Benchmarks for Multipass (DE)

It can be helpful to visualize acquired data points to get a feel for how the observed performance attributes behave. Most of the options and methods documented here work for all three scripts: analyze-archive, analyze-kconfig, and analyze-log.

• Textual Data Analysis
• Visual Data Analysis

dfatool supports six types of performance models:

• CART: Regression Trees
• DECART: Regression Trees with exclusively binary features/parameters
• LightGBM: Regressin Forests
• XGB: Regression Forests
• LMT: Linear Model Trees
• RMT: Regression Model Trees with non-binary nodes
• Least-Squares Regression

Least-Squares Regression is essentially a subset of RMT with just a single tree node. LMT and RMT differ significantly, as LMT uses a learning algorithm that starts out with a DECART and uses bottom-up pruning to turn it into an LMT, whereas RMT build a DECART that only considers parameters that are not suitable for least-squares regression and then uses least-squares regression to find and fit leaf functions.

By default, dfatool uses heuristics to determine whether it should generate a simple least-squares regression function or a fully-fledged RMT. Arguments such as --force-tree and environment variables (below) can be used to generate a different flavour of performance model; see Modeling Method Selection. Again, most of the options and methods documented here work for all three scripts: analyze-archive, analyze-kconfig, and analyze-log.

• Model Visualization and Export
• Modeling Method Selection
• Assessing Model Quality

Legacy documentation; may be outdated:

• Generating performance models for software product lines
• Data Analysis and Performance Model Generation from Log Files

Python 3.7 or newer with the following modules:

• matplotlib
• numpy
• scipy
• scikit-learn
• yaml
• zbar

On Debian Bullseye, all required modules are available as Debian packages.

Please only commit blackened code. It's best to check this with a pre-commit hook:

#!/bin/sh

if git rev-parse --verify HEAD >/dev/null 2>&1
then
	against=HEAD
else
	# Initial commit: diff against an empty tree object
	against=4b825dc642cb6eb9a060e54bf8d69288fbee4904
fi

# Redirect output to stderr.
exec 1>&2

black --check $(git diff --cached --name-only --diff-filter=ACM $against | grep '\.py$')

The following variables may be set to alter the behaviour of dfatool components.

Suitable for kconfig-webconf

The NFP values should be exactly as described by the selected configuration options.

• Kconfig
• CART without choice nodes
• CART with choice nodes
• RMT without choice nodes
• RMT with choice nodes

• Kconfig
• CART
• RMT