Autograding scripts for Wellesley's CS111 (Introductory CS) course. The program evaluates student Python code on a suite of test cases, and generates HTML outputs with error hints, as well as json logs. It supports grading for a variety of functions, including those with print statements and some support for graphics.

Currently in active development. The codebase is adapted from the Berkeley AI autograder.

Clone this repo and pull for latest changes before starting work.

1. Create a new directory for the problem set somewhere on your machine. Let's refer to it as /path/to/ps0x

2. Copy otterInspect.py, testCaseReader.py, casefile_creator.txt, honorcode.py and inspector/ to this new directory:

$ cp otterInspect.py /path/to/ps0x $ cp honorcode.py /path/to/ps0x $ cp testCaseReader.py /path/to/ps0x $ cp casefile_creator.txt /path/to/ps0x $ cp -R inspector /path/to/ps0x ```

1. Change to the the problem set directory

$ cd /path/to/ps0x/ ```

1. Delete the sample inspector/test_cases and inspector/pickled contents (but leave the directories).

$ rm -r inspector/test_cases/* inspector/picked/* ```

1. Edit casefile_creator.txt with your desired test cases, descriptions, and grade weights. For complex arguments or results, create a pickle file with a .pickle extension, drop it in the pickled/ directory, and put the filename in casefile_creator.txt. If you pickle arguments, the entire argument list should be pickled as a tuple.

   • Note that for functions with print side-effects, you must create an object of type ReturnPrint (defined in inspector/testClasses.py), pickle it, and set the "print" column in casefile_creator.txt to True.

   • For functions with randomness, set the seed to 0 before you generate your test cases.

2. Run testCaseReader.py

   $ python testCaseReader.py

   This populates the inspector/test_cases directory with files readable by the otterInspect.

3. You can now delete testCaseReader.py and casefile_creator.txt since the students don't need these.

4. Edit inspector/urls.txt where each line has a task id (like "task1") and the URL to the problem set description of the task.

5. Edit the first few lines of the main function in otterInspect.py by specifying the filenames for the student code, project name, and due date.

6. Place the solution files for the problem set, and all supporting files in your /path/to/ps0x directory. Test that running

   python otterInspect.py

   works as expected.

7. Delete all *.pyc files and the files generated by your testing runs of otterInspect:

$ rm *.pyc $ rm your_result.html $ rm -r .logs ```

1. Scrub the solutions to create skeleton files. Voila! The problem set is ready for release.