A tool that implements the process of feature extraction, model generation and statistical tests developed in the paper.

Since it uses autoskelarn, it will probably only work on Linux systems.

First, some autosklearn dependency packages must be installed. For the full story see: https://automl.github.io/auto-sklearn/master/installation.html

In short:

sudo apt-get install build-essential swig

Then install the required Python packages:

pip install -r requirements.txt

python main.py [args]

The file tool/settings.py contains several options to modify execution.

Pass the extraction option to the script. By default, the program will search BASE_DATA_PATH in settings.py to find directories with files to extract features from according to EXTRACTION_SCHEME, which defines the extraction process. By default, the resulting files will be output to the directory datasets, in the base directory of the tool as defined in FEATURE_OUTPUT_PATH.

Pass the generation option to the script. The files defined in MODEL_GENERATION_FILES_TO_LOAD will be used. If the files do not represent an absolute or relative path, they are searched for in FEATURE_OUTPUT_PATH. They are expected to be in .csv format. The options TARGET_COL and COLUMNS_TO_DISCARD define respectively the column with the labels of the dataset and the columns to be discarded. All other columns will be considered as features to train the model.

NUM_OF_TRIALS sets the number of iterations for each dataset and HPO algorithm specified in AUTOSKLEARN_HPO. The output file contains information extracted from the generated models will se saved in INFO_OUTPUT_FILE which by default is info/info.csv in the the base directory of the tool.

By default searches in info/info.csv as defined in STATISTICS_INPUT_FILE for a csv file. It will carray out statistica tests to determine whether there are sttistically significant differences in the values of STATISTICS_DATA_COLUMNS for each of the groups defined by the values of STATISTICS_DESCRIPTIVE_COLUMNS. By default, several directories with results in .csv format will be created in the statistics directory in the base directory of the tool.