• update Java
• update Checkstyle
• update Ubuntu => update python version
• JUnit5 latest version support

This is the source distribution of Ostfalia-Praktomat, a simple programming test runner for various programming languages.

The code is a fork (2019) from the KIT Praktomat (https://github.com/KITPraktomatTeam/Praktomat). It adds the ProFormA interface (https://github.com/ProFormA/proformaxml) which enables Praktomat to be used as a test runner backend in learning management systems. Most of the unused code is removed in order to ease maintenance.

The ProFormA format for tasks is 2.0 and 2.0.1 with some limitations. The ProFormA format for HTTP request and response is 2.0 and 2.1.

The code is currently only used as a 'docker composition'. So the installation manual for a plain Linux server is not up-to-date, but you can follow the Dockerfile.

The following programming languages and test frameworks are provided with the ProFormA interface.

For running SetlX tests (https://randoom.org/Software/SetlX/) you need to copy the setlx-2.7.x.jar into the extra folder and rename it to setlX-2.7.jar.

For test specific information look here.

The following types of submission are supported:

• one single file (embedded or attached to request)
• several files (embedded or attached to request)
• zip file containing several files
• URI of SVN repository to export the submission from (credentials are stored in .env file)

ProFormA Praktomat requires

• docker (https://docs.docker.com/engine/install/) and
• docker-compose (https://docs.docker.com/compose/install/).

Create an .env file containing credentials and other private data. A sample file is included as .env.example.

cp .env.example .env 

Modify credentials!

Adjust number of workers to fit your hardware.

By default Praktomat uses German locale which results in German Checkstyle messages. In order to adjust the locale settings, you can override the build argument LOCALE in the docker-compose.yml file. E.g. for American English set LOCALE to en_US.UTF-8:

build:
    ...
    args:
        LOCALE: en_US.UTF-8

For enabling HTTPS (port 443) you must

• docker-compose.yml: comment in - /data/certs:/etc/nginx/certs under nginx
• put your certificate files into /data/certs (or adjust folder name in docker-compose.yml)
• nginx/nginx.conf: comment in 443 configuration
• nginx/nginx.conf: set servername and adjust certificate file names

For using other test framework versions then you need to modify the following files:

• URLs in Dockerfile
• src/checker/checker/JUnitChecker.py
• src/checker/checker/CheckStyleChecker.py
• src/settings/docker.py

The SetlX library is not installed automatically. It can be downloaded from https://randoom.org/Software/SetlX/. The jar file must be copied into the extra subfolder and renamed to setlX-2.7.jar.

In order to build and start the docker composition simply run

docker compose build
docker compose up

The grading service is available on port 80

    http://localhost/api/v2/submissions

and port 8010 (circumventing the web server)

    http://localhost:8010/api/v2/submissions

A typical grading HTTP request in CURL is

curl -X POST --form submission.xml=@submission.xml -F "{solutionfilename}=@{solutionfile}" -F "{taskfilename}=@{taskfile}" http://localhost:8010/api/v2/submissions

with the following 'submission.xml'

<?xml version="1.0" encoding="utf-8"?>
<submission 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns="urn:proforma:v2.1" xmlns:praktomat="urn:proforma:praktomat:v2.3">

    <external-task uuid="{UUID}">
        <uri>http-file:{taskfilename}</uri>
    </external-task>
    <external-submission>
        <uri>http-file:{solutionfilename}</uri>
    </external-submission>
    <lms url="{your URI}">
        <submission-datetime>{timestamp}</submission-datetime>
        <user-id>{user id}</user-id>
        <course-id>{course id}</course-id>
    </lms>
    <result-spec format="xml" structure="separate-test-feedback" lang="de">
        <student-feedback-level>{level}</student-feedback-level>
        <teacher-feedback-level>{level}</teacher-feedback-level>
    </result-spec>

</submission>"

submission.xml can be transferred as a separate file or simply as data. Files are sent as multipart/form-data. The task file can be a zipped file or a simple xml file.

Note that embedding the student submission file(s) into submission.xml as embedded-txt-file or embedded-bin-file is also possible.

A sample for a timestamp is:

    2022-04-03T01:01:01+01:00

For submitting more than one file you have the following options:

1. create a list of embedded text or/and binary files in the files section in submission.xml.
2. external-submission: set http-file as file name list (comma separated without blanks) and use standard file upload
3. external-submission: compress all student submission files to a zip archive (with or without package paths for Java) and set http-file to appropriate filename.

Sample for http-file for Java submission files list:

    http-file:de/ostfalia/sample/User.java,de/ostfalia/sample/File.java

You can also omit the relative path for Java source files:

    http-file:User.java,File.java

In order to get progress information during upload and running tests another interface is provided.

    http://{serverhost}/api/v2/runtest

Same as http://{serverhost}/api/v2/submissions but sends server sent events while running tests.

    http://{serverhost}/api/v2/upload

Same as http://{serverhost}/api/v2/runtest but does not run tests. Only the task is uploaded into cache and a sandbox template is created if required for the programming language.

Since 4.5.0 an HTTP interface for the legacy learning management system LON-CAPA is provided.

The URI is

/api/v2/loncapasubmission

The following form fields are expected:

• submission_filename: Submission filename
• task: base64 coded ProFormA task
• task_filename: Filename of ProFormA task

The student submission is automatically put by LON-CAPA into LONCAPA_student_response.

The Praktomat stores tasks and results in a database and in the filesystem. In order not to run out of disk space the whole system should be reset from time to time (e.g. before starting a new semester). This can easily be done by calling

docker compose down
docker dompose up 

There is no need to back-up anything!

In case of a software update this is the recommended process:

1. docker compose down
2. update software (e.g. git pull)
3. ./remove_migrations.sh (delete old build files)
4. docker dompose build
5. docker dompose up