A Splunk® Add-On providing the custom search commands cipher, crypt, encode, and hash for ciphering/deciphering, encrypting/decrypting, encoding/decoding and hashing fields and events at search time.

• Cipher/decipher fields or complete events during search time using RC4, ROT13, ROT47, XOR
• Encrypt/decrypt fields or complete events during search time using RSA, AES-128/192/256-CBC or AES-128/192/256-OFB
• Encode/decode fields or complete events during search time using Base32, Base58, Base62, Base64, Binary, Decimal, Hex, Octal
• Hash fields or complete events during search time using MD5, SHA1, SHA2 (224, 256, 384, 512), SHA3 (224, 256, 384, 512), Blake2
• Manage access to encryption and decryption functionality on a per-user or per-role basis via two shipped roles
• Manage usable encryption/decryption keys on a per-user or per-role basis via the app's configuration screen

Cross-compatible with Python 2 and 3. Tested on Splunk Enterprise 8.2.6 on Windows and Linux (64-bit).

Licensed under http://creativecommons.org/licenses/by-nc-sa/4.0/.

• Authors: Harun Kuessner
• Contributors: Windu Sayles, (formerly also: Simon Balz, Mika Borner, Christoph Dittmann)
• Version: 2.3.4
• License: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License [5]

crypt command syntax: crypt mode=<d|e> algorithm=<rsa|aes-cbc|aes-ofb> key=<key_name> <field-list>

mode: Mandatory. Set to e to encrypt, set to d to decrypt the given field list using the provided key.

algorithm: Mandatory. Set to the cryptographic algorithm you would like to use for encryption/decryption.

key: Mandatory. Set to the name of a key you (or your admin) configured previously.

Encryption results will be output base64 encoded. Decryption expects input fields to be base64 encoded, results will be output in ascii. For security purposes the whole search will fail if the crypto operation fails on a single field or event. See Requirements & Configuration for key setup.

encode command syntax: encode mode=<to|from> encoding=<base32|base58|base62|base64|base85|binary|decimal|hex|octal> <field-list>

mode: Mandatory. Set to to for transforming to the specified encoding, set to from to transform from the specified encoding.

encoding: Mandatory. Set to the encoding method to apply to the given field list.

If the encoding operation fails on a single field or event, this field/event will be skipped and returned as is.

hash command syntax: hash algorithm=<md5|sha1|sha224|sha256|sha384|sha512|sha3_224|sha3_256|sha3_384|sha3_512|blake2b|blake2s> [salt=<salt_name>] <field-list>

algorithm: Mandatory. Set to the hashing algorithm you would like to use. SHA3 and Blake2 are only available when using Python 3.

saltfile: Optional. Set to the name of a key you (or your admin) configured previously.

For security purposes the whole search will fail if the hash operation fails on a single field or event. See Requirements & Configuration for salt setup.

cipher command syntax: cipher mode=<d|e> algorithm=<rc4|rot13|rot47|xor> key= <field-list>

mode: Mandatory. Set to e to cipher, set to d to decipher the given field list using the provided key.

algorithm: Mandatory. Set to the cryptographic algorithm you would like to use for ciphering/deciphering.

key: Mandatory. Specify the key to use. Requirements vary depending on the chosen algorithm.

Cipher results will be output as hex strings (except ROT results where output is ascii). Deciphering (except ROT operations) expects input fields to be hex strings, results will be output in ascii. If the cipher operation fails on a single field or event, this field/event will be skipped and returned as is.

XOR the field "message" with byte "\x2f".

   search sourcetype="ctf" | cipher mode=e algorithm=xor key="2f" message | table _time alice, bob, message

Decode the hex contents of "payload" and display readable ascii values in a table.

   search sourcetype="snort" | encode mode=from encoding=hex payload | table _time src_ip, dest_ip, payload

Encrypt the values of the plaintext fields "subject" and "content" of sourcetype "mail" using rsa and the key "public.pem".

   search sourcetype="mail" | crypt mode=e algorithm=rsa key=public.pem subject content

Encrypt raw events of sourcetype "mail" using AES-256-CBC and collect the results in a summary index.

   search sourcetype="mail" | crypt mode=e algorithm=aes-cbc key=secret.txt _raw | collect index=summary

Decrypt the content of the already RSA encrypted and summary-indexed field "username" for output in plain text using RSA. The key file "private.pem" is encrypted with AES-256-CBC, so the correspondig password has to be set via the app's set up screen prior to using the key.

   search index=summary sourcetype="server::access" | crypt mode=d algorithm=rsa key=private.pem username | table _time action username

Hash a raw event containing some malware threat artifact using sha256.

   search index=threat_intel source=sample.txt | hash algorithm=sha256 _raw

1. Just unpack to $SPLUNK_HOME/etc/apps on your Splunk search head and restart the instance. Use the deployer in a distributed environment. Make sure, the app's folder name in $SPLUNK_HOME/etc/apps is TA-cryptosuite (Downloading apps from Github and uploading them to Splunk might result in different folder names).
2. Assign the roles 'can_encrypt' and/or 'can_decrypt' to users/roles who should be able to encrypt/decrypt data using the crypt command. The cipher, hash and encode commands will automatically be available to all users.
3. Read and follow the requirements & configuration section below.
4. Optional: Set python.version=python2 or python.version=python3 in commands.conf, inputs.conf and restmap.conf if you would like to explicitly specify the Python version to use. Otherwise this will be determined by your instance's global settings.
5. Optional: Change the app's visibility via Manage>Apps or app.conf if you wish. Configuring new keys/salts relies on an internal dashboard though.

In order for the crypt command of this add-on to be fully usable you'll need to create and upload cryptographic keys. Here's how. You can also find some examples and tips in the examples/ directory.

1.1. If you plan on using RSA, generate a valid RSA key pair (optionally with key encryption) and store it in PEM format.

     Example RSA key pair creation with currently supported parameters using OpenSSL:

         openssl genrsa [-aes256|-des|-des3 -passout pass:password] -out private.pem <2048|4096>

         openssl rsa -in private.pem -outform PEM -RSAPublicKey_out -out public.pem

         Note that using 1024 bit keys is considered unsafe and therefore not supported by the app.

         Important Note: Use of encrypted private RSA keys is only supported when having the "pycryptodomex" Python package installed (see below)!

1.2. If you plan on using AES, create a "key file".

     Example "key file" creation for AES:

         Put your key and IV into a plain ASCII file. Key has to go on line 1 and IV on line 2. They can either be just self-written plaintext strings, or hex-representations as output by e.g. OpenSSL:

         openssl enc [-aes-256-cbc|-aes-256-cfb] -k secret -P -md sha1

         The IV's length has to be 16 bytes/characters. The key's length has to be 16/24/32 bytes/characters respectively for 128/192/256 AES encryption. For security purposes 256 bit keys are recommended unless performance is important.

2. Go to the app's configuration dashboard in your preferred browser. Click "Create New Input".

     Name: Enter the name you would like the your key to be accessible through. E.g. the original key file's name.

     Type of Data: Select "New key".

     Key/Salt: Copy-paste the contents of your generated key file here. Your key will be stored encrypted in passwords.conf.

     RSA key encryption password: Only required if you are currently configuring an encrypted private RSA key. Supported key encryption algorithms are AES-256-CBC, DES-CBC and DES-EDE3-CBC. Important Note: Use of encrypted private RSA keys is only supported when having the "pycryptodomex" Python package installed (see below)!

     Authorized roles: Select which roles you want to grant permissions to use the key. Will be ORed with authorized users. Users/roles will also require the "can_encrypt" and/or "can_decrypt" role in order to use the crypt command in the first place.

     Authorized users: Select which users you want to grant permissions to use the key. Will be ORed with authorized roles. Users/roles will also require the "can_encrypt" and/or "can_decrypt" role in order to use the crypt command in the first place.

     Note: For a RSA key pair, you'll have to do this twice. Once for the private key and once for the public key.

If you plan on salting your hashes when using the hash command, create and store upload salts like so:

1. Go to the app's configuration dashboard in your preferred browser. Click "Create New Input".

     Name: Enter the name you would like the your salt to be accessible through.

     Type of Data: Select "New salt".

     Key/Salt: Enter or copy-paste your salt here. It will be stored encrypted in passwords.conf.

     RSA key encryption password: Leave empty.

     Authorized roles: Select which roles you want to grant permissions to use the salt. Will be ORed with authorized users.

     Authorized users: Select which users you want to grant permissions to use the salt. Will be ORed with authorized roles.

Note: Handling keys and salts this way is a security feature, since specifing the key/salt directly in a Splunk search directly would make them visible in the _internal index. Keys are not handled like this for the cipher command, as the implemented ciphers are generally not regarded as secure anymore.

On the use of encrypted private RSA keys and used Python libraries: By default the add-on uses a pure-Python implementation of the RSA (and AES) algorithm which comes shipped with the add-on for all platforms. This implementation does not support usage of encrypted private RSA keys. Their usage is only possible by installing the "pycryptodomex" Python package to your OS's installation of an instance of Python 3.7.8 or 2.7.17 (see https://www.pycryptodome.org/en/latest/src/installation.html). After the package's successful installation, configure this add-on to make use of the system Python's installed packages via the add-on's configuration page. Unfortunately due to Splunk's built-in Python's poor handling of 3rd party packages, the package cannot be installed to only that instance of Python.

• Attempts to modify _time will be ignored since Splunk always expects a valid _time field.
• Wildcards for field names are not supported (yet).
• Encryption and decryption of big quantities of data can be ressource intensive operations. Use with caution in old or feeble environments.
• Currently only AES-256-CBC, DES-CBC and DES-EDE3-CBC are supported for private RSA key file encryption.
• To implement proper key/salt management (without relying on my weak JavaScript skills for a management dashboard) the add-on leverages the comfort Splunk Add-On Builder grants.
  This is why your key/salt configurations are stored as modular input configurations. Don't worry, they are not used as such. A future version of the add-on might implement this better.

• Granting a user or role the role 'can_encrypt' or 'can_decrypt' needed for usage of the crypt command will also grant them the capability 'list_storage_passwords'.
  This enables the technically savvy user to potentially read more secrets via Splunk's REST API than desirable.
  Unfortunately as of now there is no way around this if the encryption/decryption keys and salts used for this app's function should not be stored in plaintext on disk.
  You can argue this way or that. My assumption is that only high-privileged users will use the crypto functionality in the first place, and thus prefferred encrypted keys over potentially exessive permissions.

• The keys and salts you upload are stored encrypted in passwords.conf. However to quote a Splunk blog post: "[...] Since the encrypted data and the encryption key are stored on the same machine, cryptographically this is equivalent to obfuscation. While some people might argue that this is very weak encryption, it is the best we can do with storing the encrypted data and encryption key on the same machine and it is definitely better than clear text passwords [...]"[0].

• The used libraries (see Attribution) have been chosen due to their compatibility with multiple Python versions and OSes. They might not be the best implementations of the respective algorithms and are certainly not the fastest, but more popular implementations posed compatibility challenges due to relying on platform-specific compiled c modules.

• The used RSA implementation is only capable of processing so many bytes of data at once during encryption, fields larger than that value have to be split into blocks. This might enable certain attacks on the RSA crypto system. To mitigate the chances of a successful attack, random padding (PKCS#1 v2 (OAEP)) is used and can not be disabled in this version of the add-on.

• Using 1024 bit RSA keys is generally considered unsafe and therefore not possible using with the add-on.

• I chose AES-CBC and AES-OFB as the only available modes as they offer the best balance between performance and security for the purposes of this add-on.

• It is not recommended to use MD5 or SHA1 for security purposes (e.g. storing password hashes) since these hashing algorithms are not regarded as safe anymore.

• It is not recommended to use the ciphers provided through the cipher command for security purposes (e.g. storing safely encrypted data) since these algorithms are not regarded as safe anymore.

• SHA3 and Blake2 are only available when using Python3 as your environment's interpreter.

• Update to Splunk Python SDK v2.x (currently has bugs)

• Better error handling
• Improve performance
• Add further ciphers (RC2, RC4-Drop, ...) and potentially arguments to control cipher parameters
• Add further encodings (Punycode, ...) and potentially arguments to control encoding parameters
• Disable helper inputs by default
• Potentially implement support for wildcards for field names

• Ensured future cloud compatibility (again)

• Fixed a bug where a wrong key might accidentally be used
• Added support for hex-formatted AES keys/IVs

• Ensured future cloud compatibility

• Implemented support for Base58 and Base62 encoding/decoding

• Repackaged using Splunk Add-On Builder v4.0.0
• Updated Splunk Python SDK to v1.6.16

• Implemented cipher command to cipher/decipher fields or complete events during search time using RC4, ROT13, ROT47, ROX
• Implemented encode command to encode/decode fields or complete events during search time using Base32, Base64, Binary, Decimal, Hex, Octal
• Improved documentation
• Updated Splunk Python SDK to v1.6.15, removing a bug which can cause Splunk instances to hang

• Tested and documented support for encrypted private RSA keys

• Replaced "cryptography" with "pycryptodomex" as compatibility is better

• Re-implemented support for encrypted private RSA keys through the "cryptography" Python package
• Accounting for Splunk sometimes randomly removing \n or spaces for no apparent reason when saving keys/salts
• Updated Splunk Python SDK

• Fixed support for distributed environments

• Ensured and tested Splunk 8 compatibility
• Ensured and tested Python2/3 cross-compatibility for crypt command
• Ensured and tested Python2/3 cross-compatibility for hash command
• Added a logo

• Updated Splunk SDK for Python

• Using custom search command protocol v2 now

• Updated README and docs with soon-to-come changes

• Implemented proper key/salt management on per-user and per-role basis

• Added licenses to used Python modules and gave credit

• Removed keyencryption parameter from crypt command and replaced by automatic detection

• Removed randpadding parameter and support for non-random padding

• Implemented basic review checks for private/public RSA key usage during encryption/decryption attempts

• Implemented PKCS#1 v2 (OAEP) support

• Implemented possibility for AES-128/192/256-CBC/OFB field/event encryption & decryption

• AES-CBC and AES-OFB are now the only parameter values and modes for AES encryption/decryption

• Ensured Splunk 8 and Python 2/3 cross-compatilibity for hash command

• Hashes will now be written to a new field with the name of the used algorithm

• Renamed saltfile parameter to salt

• Implemented SHA3 and Blake2 support for environments using Python 3

See https://github.com/my2ndhead/SA-hypercrypto for previous, deprecated versions.

The add-on relies on the Splunk Add-On builder[1], developed by Splunk and licensed under the Apache License, Version 2.0 http://www.apache.org/licenses/LICENSE-2.0.

The crypt command uses a slightly modified version of Sybren A. Stuevel's (https://stuvel.eu/) pure-Python RSA implementation [2] which is licensed under the Apache License, Version 2.0 http://www.apache.org/licenses/LICENSE-2.0. This module itself relies on Ilya Etingof's pyasn1 Python module [2.2] which is licensed under the BSD 2-Clause License, https://spdx.org/licenses/BSD-2-Clause.html. Changes include the PKCS#1 v2 (OAEP) support suggested and implemented by Inada Naoki (https://twitter.com/methane) [2.3] which is licensed under the Apache License, Version 2.0 http://www.apache.org/licenses/LICENSE-2.0 as well.

The crypt command also uses Richard Moore's (http://www.ricmoo.com/) pure-Python AES implementation [3] which is licensed under the MIT License, https://opensource.org/licenses/MIT.

When including a system wide Python installation's package library, the add-on will make use of Python's "pycryptodomex" package [4] which is partially in the public domain and partially released under the BSD 2-Clause license https://www.pycryptodome.org/en/latest/src/license.html.

The encode command uses a pure-Python Base58 implementation based on David Keijser's code [5] which is licensed under the MIT License, https://opensource.org/licenses/MIT.

The encode command uses a pure-Python Base62 implementation based on Sumin Byeon's code [6] which is licensed under the BSD 2-Clause Simplified License, https://spdx.org/licenses/BSD-2-Clause.html.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. [7]

This roughly translates to

• You may freely use, share and adapt this work non-commercially as long as you give appropriate credit. When adapting or sharing, this needs to happen under the same license, changes should be indicated.
• You may freely use this work in a commercial environment as long as the use is not primarily intended for commercial advantage or monetary compensation. YOU MAY NOT SELL THIS WORK. Neither standalone nor within an application bundle.

As this paragraph is not a substitute for the license, please refer to References for completeness and correctness.

Feel free to contact me should you plan to use the add-on outside these terms.

[0] https://www.splunk.com/en_us/blog/security/storing-encrypted-credentials.html

[1] https://splunkbase.splunk.com/app/2962

[2] https://github.com/sybrenstuvel/python-rsa

[2.2] https://github.com/etingof/pyasn1

[2.3] sybrenstuvel/python-rsa#126 and https://github.com/methane/python-rsa/tree/rsa_oaep

[3] https://github.com/ricmoo/pyaes

[4] https://pypi.org/project/pycryptodomex/

[5] https://github.com/suminb/base62

[6] https://github.com/keis/base58

[7] http://creativecommons.org/licenses/by-nc-sa/4.0/