Skip to content

Transforms tokens into original source code (while preserving whitespace)

License

Notifications You must be signed in to change notification settings

myint/untokenize

Repository files navigation

untokenize

untokenize transforms tokens into source code. Unlike the standard library's tokenize.untokenize(), it preserves the original whitespace between tokens.

Build status

Usage

import untokenize
source_code = untokenize.untokenize(tokens)

Tests

To run the unit tests:

$ ./test_untokenize.py

There is also an acid test. It tokenizes Python code and confirms that the code generated by untokenize exactly matches the original source code from before tokenization:

$ ./test_acid.py

About

Transforms tokens into original source code (while preserving whitespace)

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published