versions

(build-version: , build-date: )

documentation

https://wltp.readthedocs.org/

live-demo

sources

https://github.com/JRCSTU/wltp

keywords

UNECE, automotive, car, cars, driving, engine, emissions, fuel-consumption, gears, gearshifts, rpm, simulation, simulator, standard, vehicle, vehicles, WLTC, NEDC

copyright

2013-2020 European Commission (JRC-IET)

A python-3.6+ package to generate the gear-shifts of Light-duty vehicles running the WLTP driving-cycles, according to UNECE's GTRs.

The calculator accepts as input the vehicle's technical data, along with parameters for modifying the execution of the WLTC cycle, and it then spits-out the gear-shifts of the vehicle, the attained speed-profile, and any warnings. It does not calculate any CO₂ emissions.

An "execution" or a "run" of an experiment is depicted in the following diagram:

.-----------------.                         .------------------.
:      Input      :                         :      Output      :
;-----------------;                         ;------------------;

; +--test_mass ; ____________ ; +--pmr ;

; +--n_idle ; | | ; +--wltc_class ;

; +--f0,f1,f2 ; ==> | Cycle | ==> ; +--... ;

; +--wot/ ; | Generator | ; +--cycle ;

; +-- ; ; | +-- ;

; +--n2vs ; ; +--gwots ;

; +-- ; ; +-- ;

'-----------------' '------------------'

The Input, Output and all its contents are instances of datamodel (trees of strings, numbers & pandas objects)

• Launch the example jupyter notebooks in a private demo server ().
• Otherwise, install it locally, preferably from the sources (instructions below).
• pip install "extras" (e.g. pip install wltp[all]):
  • plot, excel, all, dev, notebook, test, doc

Python-3.6+ is required and Python-3.7 or Python-3.8 recommended. It requires numpy/scipy and pandas libraries with native backends.

pip install <package-file-v1.2.3.whl>

Download the sources,

• either with git, by giving this command to the terminal:

  git clone https://github.com/JRCSTU/wltp/ --depth=1

• or download and extract the project-archive from the release page: https://github.com/JRCSTU/wltp/archive/v1.1.0.dev0.zip

From within the project directory, run one of these commands to install it:

• for standard python, installing with pip is enough (but might):

  pip install -e .[test]

• for conda, prefer to install the conda-packages listed in Notebooks/conda/conda-reqs.txt, before running the same pip command, like this:

  conda install  --override-channels -c ankostis -c conda-forge -c defaults --file Notebooks/conda/conda-reqs.txt
  pip install -e .[dev]

• Check installation:

  $ wltp --version
  ...
  
  $ wltp --help
  ...

  See: wltp-usage

• Recreate jupyter notebooks from the paired *.py "py:percent" files (only these files are stored in git-repo), by executing the bash-script:

  Notebooks/recreate_ipynbs.sh

• Run pyalgo on all AccDB cars to re-create the H5 file needed for CarsDB-compare notebook, etc:

  Notebooks/recreate_pyalgo_h5.sh

import pandas as pd
from wltp import datamodel
from wltp.experiment import Experiment

inp_mdl = datamodel.get_model_base()
inp_mdl.update({
    "unladen_mass": None,
    "test_mass": 1100,  # in kg
    "p_rated": 95.3,  # in kW
    "n_rated": 3000,  # in RPM
    "n_idle": 600,
    "n2v_ratios": [122.88, 75.12, 50.06, 38.26, 33.63],

    ## For giving absolute P numbers,
    #  rename `p_norm` column to `p`.
    #
    "wot": pd.DataFrame(
        [[600, 0.1],
        [2500, 1],
        [3500, 1],
        [5000, 0.7]], columns=["n", "p_norm"]
    ),
    'f0': 395.78,
    'f1': 0,
    'f2': 0.15,
})
datamodel.validate_model(inp_mdl, additional_properties=True)
exp = Experiment(inp_mdl, skip_model_validation=True)

# exp = Experiment(inp_mdl)
out_mdl = exp.run()
print(f"Available values: \n{list(out_mdl.keys())}")
print(f"Cycle: \n{out_mdl['cycle']}")

See: python-usage

The files and folders of the project are listed below (see also architecture:Architecture):

+--bin/                     # (shell-scripts) Utilities & preprocessing of WLTC data on GTR and the wltp_db
|   +--bumpver.py           # (script) Update project's version-string
+--wltp/                    # (package) python-code of the calculator
|   +--cycles/              # (package) code & data for the WLTC data
|   +--experiment           # top-level code running the algo
|   +--datamodel            # schemas & defaults for data of algo
|   +--cycler               # code for generating the cycle
|   +--engine               # formulae for engine power & revolutions and gear-box
|   +--vehicle              # formulae for cycle/vehicle dynamics
|   +--vmax                 # formulae estimating `v_max` from wot
|   +--downscale            # formulae downscaling cycles based on pmr/test_mass ratio
|   +--invariants           # definitions & idempotent formulae for physics/engineering
|   +--io                   # utilities for starting-up, parsing, naming and spitting data
|   +--utils                # software utils unrelated to physics or engineering
|   +--cli                  # (OUTDATED) command-line entry-point for launching this wltp tool
|   +--plots                # (OUTDATED) code for plotting diagrams related to wltp cycles & results
|   +--idgears              # (OUTDATED) reconstructs the gears-profile by identifying the actual gears
+--tests/                   # (package) Test-TestCases
    +--vehdb                # Utils for manipulating h5db with accdb & pyalgo cases.
+--docs/                    # (folder) documentation
|   +--pyplots/             # (DEPRECATED by notebooks) scripts plotting the metric diagrams embedded in the README
+--Notebooks/               # Jupyter notebooks for running & comparing results (see `Notebooks/README.md`)
    +--AccDB_src/           # AccDB code & queries extracted and stored as text
+--setup.py                 # (script) The entry point for `setuptools`, installing, testing, etc
+--requirements/            # (txt-files) Various pip-dependencies for tools.
+--README.rst
+--CHANGES.rst
+--LICENSE.txt

First run python or ipython REPL (Read-Eval-Print Loop) and try to import the project to check its version:

If everything works, create the datamodel of the experiment. You can assemble the model-tree by the use of:

• sequences,
• dictionaries,
• pandas.DataFrame,
• pandas.Series, and
• URI-references to other model-trees.

For instance:

For information on the accepted model-data, check the code:Schemas:

You then have to feed this model-tree to the ~wltp.experiment.Experiment constructor. Internally the pandalone.pandel.Pandel resolves URIs, fills-in default values and validates the data based on the project's pre-defined JSON-schema:

Assuming validation passes without errors, you can now inspect the defaulted-model before running the experiment:

Now you can run the experiment:

To access the time-based cycle-results it is better to use a pandas.DataFrame:

You can export the cycle-run results in a CSV-file with the following pandas command:

>>> df.to_csv('cycle.csv')                                                      # doctest: +SKIP

For more examples, download the sources and check the test-cases found under the /tests/ folder.

The command-line usage below requires the Python environment to be installed, and provides for executing an experiment directly from the OS's shell (i.e. cmd in windows or bash in POSIX), and in a single command. To have precise control over the inputs and outputs (i.e. experiments in a "batch" and/or in a design of experiments) you have to run the experiments using the API python, as explained below.

The entry-point script is called wltp, and it must have been placed in your PATH during installation. This script can construct a model by reading input-data from multiple files and/or overriding specific single-value items. Conversely, it can output multiple parts of the resulting-model into files.

To get help for this script, use the following commands:

$ wltp --help                               ## to get generic help for cmd-line syntax
$ wltcmdp.py -M vehicle/full_load_curve     ## to get help for specific model-paths

and then, assuming vehicle.csv is a CSV file with the vehicle parameters for which you want to override the n_idle only, run the following:

$ wltp -v \
    -I vehicle.csv file_frmt=SERIES model_path=params header@=None \
    -m vehicle/n_idle:=850 \
    -O cycle.csv model_path=cycle

In Windows and OS X you may utilize the excellent xlwings library to use Excel files for providing input and output to the experiment.

To create the necessary template-files in your current-directory you should enter:

$ wltp --excel

You could type instead wltp --excel {file_path} to specify a different destination path.

In windows/OS X you can type wltp --excelrun and the files will be created in your home-directory and the excel will open them in one-shot.

All the above commands creates two files:

wltp_excel_runner.xlsm

The python-enabled excel-file where input and output data are written, as seen in the screenshot below:

After opening it the first tie, enable the macros on the workbook, select the python-code at the left and click the Run Selection as Python button; one sheet per vehicle should be created.

The excel-file contains additionally appropriate VBA modules allowing you to invoke Python code present in selected cells with a click of a button, and python-functions declared in the python-script, below, using the mypy namespace.

To add more input-columns, you need to set as column Headers the json-pointers path of the desired model item (see python-usage below,).

wltp_excel_runner.py

Utility python functions used by the above xls-file for running a batch of experiments.

The particular functions included reads multiple vehicles from the input table with various vehicle characteristics and/or experiment parameters, and then it adds a new worksheet containing the cycle-run of each vehicle . Of course you can edit it to further fit your needs.

Some general notes regarding the python-code from excel-cells:

• On each invocation, the predefined VBA module pandalon executes a dynamically generated python-script file in the same folder where the excel-file resides, which, among others, imports the "sister" python-script file. You can read & modify the sister python-script to import libraries such as 'numpy' and 'pandas', or pre-define utility python functions.
• The name of the sister python-script is automatically calculated from the name of the Excel-file, and it must be valid as a python module-name. Therefore do not use non-alphanumeric characters such as spaces(), dashes(-) and dots(.) on the Excel-file.
• On errors, a log-file is written in the same folder where the excel-file resides, for as long as the message-box is visible, and it is deleted automatically after you click 'ok'!
• Read http://docs.xlwings.org/quickstart.html

The Python code is highly modular, with testability in mind. so that specific parts can run in isolation. This facilitates studying tough issues, such as, double-precision reproducibility, boundary conditions, comparison of numeric outputs, and studying the code in sub-routines.

Computations are vectorial, based on hierarchical dataframes, all of them stored in a single structure, the datamodel. In case the computation breaks, you can still retrieve all intermediate results till that point.

The computation code is roughly divided in these python modules:

The blueprint for the underlying software ideas is given with this diagram:

Note that currently there is no scheduler component, which will allow to execute the tool with a varying list of available inputs & required data, and automatically compute only what is not already given.

This program imitates to some degree the MS Access DB (as of July 2019), following this 08.07.2019_HS rev2_23072019 GTR specification (docs/_static/WLTP-GS-TF-41 GTR 15 annex 1 and annex 2 08.07.2019_HS rev2_23072019.docx, included in the docs/_static folder).

The latest official version of this GTR, along with other related documents maybe found at UNECE's site:

• http://www.unece.org/trans/main/wp29/wp29wgs/wp29grpe/grpedoc_2013.html
• https://www2.unece.org/wiki/pages/viewpage.action?pageId=2523179

The WLTC-profiles for the various classes were generated from the tables of the specs above using the devtools/csvcolumns8to2.py script, but it still requires an intermediate manual step involving a spreadsheet to copy the table into ands save them as CSV.

The GTR's velocity traces have overlapping split-time values, i.e. belonging to 2 phases, and e.g. for class1 these are the sample-values @ times 589 & 1022:

Some programs and most spreadsheets do not handle overlapping split-time values like that (i.e. keeping a separate column for each class-phase), and assign split-times either to the earlier or the later phase, distorting thus the duration & number of time samples some phases contain!

For instance, Access-DB tables assign split-times on the lower parts, distorting the start-times & durations for all phases except the 1st one (deviations from GTR in bold):

The inverse distortion (assigning split-times on the higher parts) would preserve phase starting times (hint: downscaling algorithm depends on those absolute timings being precisely correct):

On a related issue, GTR's formula for Acceleration (Annex 1 3.1) produces one less value than the number of velocity samples (like the majority of the distorted phases above). GTR prescribes to (optionally) append and extra A=0 sample at the end, to equalize Acceleration & Velocities lengths, but that is not totally ok (hint: mean Acceleration values do not add up like mean-Velocities do, see next point about averaging).

Since most calculated and measured quantities (like cycle Power) are tied to the acceleration, we could refrain from adding the extra 0, and leave all phases with -1 samples, without any overlapping split-times:

Actually this is "semi-VA0" phasings, above, with the last part equally distorted by -1 @ 1610. But now the whole cycle has (disturbingly) -1 # of samples & duration:

We can resolve this, conceptually, by assuming that each Acceleration-dependent sample signifies a time-duration, so that although the # of samples are still -1, the phase & cycle durations (in sec) are as expected:

Summarizing the last "VA0+" phasing scheme:

• each step signifies a "duration" of 1 sec,
• the duration of the final sample @ 1610 reaches just before 1611sec,
• # of samples for all phases are symmetrically -1 compared to Velocity phases,
• it is valid for Acceleration-dependent quantities only,
• it is valid for any sampling frequency (not just 1Hz),
• respects the Dijkstra counting (notice the parenthesis signifying open right intervals, in the above table), BUT ...
• Velocity-related quantities cannot utilize this phasing scheme, must stick to the original, with overlapping split-times.

Calculating mean values for Acceleration-related quantities produce correct results only with non-overlapping split-times.

It's easier to demonstrate the issues with a hypothetical 4-sec cycle,  composed of 2 symmetrical ramp-up/ramp-down 2-sec phases (the "blue" line in the plot, below):

• The final A value has been kept blank, so that mean values per-phase add up, and phases no longer overlap.

• Applying the V-phasings and the extra 0 on mean(A) would have arrived to counterintuitive values, that don't even sum up to 0:
  • up-ramp: $\left(\frac{5 + 5 + (-5)}{3} =\right) 1.66m/sec^2$
  • down-ramp: $\left(\frac{(-5) + (-5) + 0}{3} =\right) -3.33m/sec^2$

All phases in WLTC begin and finish with consecutive zeros(0), therefore the deliberations above do not manifest as problems; but at the same time, discovering off-by-one errors & shifts in time (wherever this really matters e.g. for syncing data), on arbitrary files containing calculated and/or measured traces is really hard: SUMs & CUMSUMs do not produce any difference at all.

The tables in the next section, along with accompanying CRC functions developed in Python, come as an aid to the problems above.

As reported by wltp.cycles.cycle_phases(), and neglecting the advice to optionally add a final 0 when calculating the cycle Acceleration (Annex 1 2-3.1), the following 3 phasing are identified from velocity traces of 1Hz:

• V: phases for quantities dependent on Velocity samples, overlapping split-times.
• VA0: phases for Acceleration-dependent quantities, -1 length, NON overlapping split-times, starting on t=0.
• VA1: phases for Acceleration-dependent quantities, -1 length, NON overlapping split-times, starting on t=1. (e.g. Energy in Annex 7).

• The ~wltp.cycles.crc_velocity() function has been specially crafted to consume series of floats with 2-digits precision (~wltp.invariants.v_decimals) denoting Velocity-traces, and spitting out a hexadecimal string, the CRC, without neglecting any zeros(0) in the trace.
• The checksums for all phasings are reported by ~wltp.cycles.cycle_checksums(), and the the table below is constructed. The original checksums of the GTR are also included in the final 2 columns.
• Based on this table of CRCs, the ~wltp.cycles.identify_cycle_v_crc() function tries to match and identify any given Velocity-trace:

... where if a some cycle-phase is identified as:

• V phasing, it contains all samples;
• VA0 phasing, it lacks -1 sample from the end;
• VA1 phasing, it lacks -1 sample from the start.

For instance, let's identify the V-trace of class1's full cycle:

>>> from wltp.datamodel import get_class_v_cycle as wltc >>> from wltp.cycles import identify_cycle_v as crc >>> V = wltc("class1")

>>> crc(V) # full cycle ('class1', None, 'V') >>> crc(V[:-1]) # -1 (last) sample ('class1', None, 'VA0')

The crc() <wltp.cycles.crc_velocity> function returns a 3-tuple: (i-class, i-phase, i-kind):

• When i-phase is None, the trace was a full-cycle.

Now let's identify the phases of "Access-DB":

>>> crc(V[:590]) # AccDB phase1 respects GTR ('class1', 'phase-1', 'V') >>> crc(V[590:1023]) # AccDB phase2 has -1 (first) sample ('class1', 'phase-2', 'VA1') >>> crc(V[:1023]) # cumulative AccDB phase2 respects GTR ('class1', 'PHASE-2', 'V')

• When i-phase is CAPITALIZED, the trace was cumulative.
• Phase2 is missing -1 sample from the start (i-kind == VA1).

>>> crc(V[1023:]) # AccDB phase3 ('class1', 'phase-1', 'VA1')

• Phase3 was identified again as phase1, since they are identical.

Finally, clipping both samples from start & end, matches no CRC:

>>> crc(V[1:-1]) (None, None, None)

To be note, all cases above would have had identical CUMSUM (GTR's) CRCs.

This project is hosted in github. To provide feedback about bugs and errors or questions and requests for enhancements, use github's Issue-tracker.

For submitting code, use UTF-8 everywhere, unix-eol(LF) and set git --config core.autocrlf = input.

The typical development procedure is like this:

0. Install and arm a pre-commit hook with black to auto-format you python-code.
1. Modify the sources in small, isolated and well-defined changes, i.e. adding a single feature, or fixing a specific bug.
2. Add test-cases "proving" your code.
3. Rerun all test-cases to ensure that you didn't break anything, and check their coverage remain above the limit set in setup.cfg.

4. If you made a rather important modification, update also the CHANGES file and/or other documents (i.e. README.rst). To see the rendered results of the documents, issue the following commands and read the result html at build/sphinx/html/index.html:

   python setup.py build_sphinx                  # Builds html docs
   python setup.py build_sphinx -b doctest       # Checks if python-code embedded in comments runs ok.

5. If there are no problems, commit your changes with a descriptive message.
6. Repeat this cycle for other bugs/enhancements.

7. When you are finished, push the changes upstream to github and make a merge_request. You can check whether your merge-request indeed passed the tests by checking its build-status on the integration-server's site (TravisCI).

   Hint

   Skim through the small IPython developer's documentation on the matter: The perfect pull request

8. Generate the Sphinx documents in ./wltp.git/docs/_build/html/ with this command:

   python setup.py build_sphinx

   Access the generated documents through a a web-server, for graphtik graphs to work correctly, with this bash-command (remove the final & on Windows):

   python -m http.server 8080 --directory ./wltp.git/docs/_build/html/ &

• Author:

  • Kostis Anagnostopoulos

• Contributing Authors:

  • Heinz Steven (test-data, validation and review)
  • Georgios Fontaras (simulation, physics & engineering support)
  • Alessandro Marotta (policy support)
  • Jelica Pavlovic (policy support)
  • Eckhard Schlichte (discussions & advice)

See also architecture:Architecture.