This is a Python destination driver for syslog-ng which writes messages to SQS in batches. Messages are flushed to SQS based on time and count, and batched to minimize cost and maximize throughput. Each SQS Message contains 1 or more Syslog messages, serialized as a JSON array of objects*.

* If the Flush.Single config flag is set, SQS Messages each contain exactly one Syslog message serialized as a JSON object

• Add a flag to disable grouping of messages to SQS
• Figure out what to do if a serialized group or batch exceeds SQS size limits
• Handle exceptions from calling SQS and retry

• syslog-ng
• syslog-ng-incubator (syslog-ng-mod-python)
• boto (python-boto)

python python_sqs.py

import python_sqs

python_sqs.init()
python_sqs.queue({'message': 'text', 'src_ip': '192.168.0.1'})
python_sqs.deinit()

[AWS]
Region=us-west-2
AccessKeyID=COOLESTKEYEVER
SecretAccessKey=MOSTSECRETKEYONEARTH
SQSQueueName=syslog-queue

[Flush]
Single=False
Seconds=1
Lines=100