Simple JSON wrapper protocol for ZeroMQ.

I want to make it trivial to broadcast and consume events that happen in the apartment. If you find a way to detect something in the apartment, create a daemon that notifies the rest of us about it.

Some examples of possible event publishers:

• IR reciever
• bro based network activity monitors
• voice recognition devices
• generic security sytems
• desktop apps
• web apps

Some examples of micro-daemons that we could do with these event publishers:

• Display events on the desktop using notify-send on Linux boxes, and Growl on OS X
• handle Wit.ai intents generated by the microphone on other systems
• automatically activate game mode when the TV is switched on

http://zguide.zeromq.org/page:all#Pub-Sub-Message-Envelopes

We send two-part messages over the ZeroMQ sockets:

PART1: (the origin)  STRING like 'myapp/user/login@host.name'
PART2: (the message) JSON of format {
    'content': JSON,
    'time'   : STRING like '%Y-%m-%d %H:%M',
    'origin' : STRING like 'myapp/user/login@host.name'
}

We send the message in two seperate parts to handle endpoint subscriptions neatly: ZeroMQ manages subscriptions by checking for byte equality between the subscription filters and every inbound message. More info. Byte-string equality is easy on strings, and hard on JSON coming from an unsorted hashmap ;)

we put the hostname origin at the end because who cares where its coming from? only what the API startpoint is.

All wire bytes are UTF8-encoded string, and start with the string 'all'.

You can use squidwork.quick.pub and squidwork.quick.sub to create the type of ZeroMQ socket you need to send or receive events, and never have to import zmq yourself, or worry about the other socket types. Here is a simple event publisher and a reciever:

# event emitter
from squidwork.quick import pub
from squidwork import Sender
from time import sleep

endpoint = Sender(pub('tcp://127.0.0.1:6696'))
while True:
    sleep(3)
    endpoint.send(['hello', 'world'])

# event listener
from squidwork.quick import pub
from squidwork import Sender

reciever = Reciever(sub('tcp://127.0.0.1:6696'))
while True:
    # this call blocks
    message = reciever.recieve()

    print message.origin
    print message.time
    print message.content

This module provides an implementation of the protocol over Websockets for Javascript clients. I'm only planning on implementing subscribing features, no publishing. Here's a somple example:

<script type="text/javascript" src="http://squidwork.internal/squidwork.js"></script>

# squidwork imported into `window` namespace already
sub = squidwork.subscribe 'tcp://192.168.0.1:9999', 'ear/active', (message) ->
    document.body.innerHTML = message.content

# re-assign sub.fn to change the handler
sub.fn = (message) ->
    console.log("Got message: ", message)

# we don't care anymore
sub.unsubscribe()

This is implemented on top of the Tornado event loop in python and ZeroMQ streams. To run the bridge, execute the following:

python -m squidwork.web -c ./config.yml

If you direct your browser to http://localhost:8888 you will see a mostly-blank page that directs you to open your eveloper console. From there you can inspect the configuration data and create new subscriptions!

At this point, we've got a full blown web micro-library going on in squidwork.web.handlers, and a pretty great Javascript squidwork API, so why not make a real front-end app of equal quality?

The monitor is a graphical log of recent squidwork messages. depending on configuration, it displays a list of the last N squidwork events. It can filter for uniqueness (squashing a bunch of duplicates), or just display things chronologically.

I feel like I might be pioneering strange application architectures at this point. Here's what we've got so far:

• ZeroMQ pub-sub data source
• data becomes JSON
  • some is stream over WebSockets to front-end
  • some is plain-jain JSON
• tornado webserver to deliver templates and websockets
  • templates mostly static, only dynamic so we can ship over config values
  • only 4 lines of HTML
• coffeescript front-end app consumer
• views rendered on the client by Mithril.js

I can't tell if Mithril.js is awesome or what. Look at the occlusion culling bit the creator does: we've got a full data binding application (or at least view-controller) in 43 lines. So refreshing compared to Ember.js.

I'm very satisified with my Mithril experiece. it's considerably less complex to use Mithril than Mustache or any other "logic-free" template system that then requires an aditional view layer, and it seems pretty fast. I'm just using it for rendering (no two-way data bindigns), but it's still very convinient. Hooray for immidiate-mode GUIs.

The class squidwork.config.Config is a unified command line argument and configuration file loader. Unified command line argument and configuration file loader. Abstracts over argparse and config file direct access. Uses argparse to create the command-line interface, and you may pass kwargs through to the ArgumentParser when instantiating a new Config.

Define options that your app requires using the option method, and then retrieve command line args and load the config file with the retrieve method.

After calling Config#retrieve you may access your config options as dot-properties of the config object. Retrieve the configuration will also populate the Service index, see squidwork.config.service for more information

Usage example:

config = Config('my app')
config.option('dog-name')
# Config#option passes args through to 
config.option('threshold', type=float, help="sound level before we  scold")
config.retrieve()

# Dashed option names are python-ized by replacing each - with a _
>>> dog = Dog(name=config.dog_name)

see test/beeper.py for a concrete usage example

1. pyzmq
2. pyyaml (for config)
3. tornado (for squidwork.websocket, which depends on config)

The tests in tests/ come in pairs, one sender and one receiver for each strategy.