Simple JSON wrapper protocol for ZeroMQ.
I want to make it trivial to broadcast and consume events that happen in the apartment. If you find a way to detect something in the apartment, create a daemon that notifies the rest of us about it.
Some examples of possible event publishers:
- IR reciever
bro
based network activity monitors- voice recognition devices
- generic security sytems
- desktop apps
- web apps
Some examples of micro-daemons that we could do with these event publishers:
- Display events on the desktop using
notify-send
on Linux boxes, and Growl on OS X - handle Wit.ai intents generated by the microphone on other systems
- automatically activate game mode when the TV is switched on
http://zguide.zeromq.org/page:all#Pub-Sub-Message-Envelopes
We send two-part messages over the ZeroMQ sockets:
PART1: (the origin) STRING like 'myapp/user/login@host.name'
PART2: (the message) JSON of format {
'content': JSON,
'time' : STRING like '%Y-%m-%d %H:%M',
'origin' : STRING like 'myapp/user/login@host.name'
}
We send the message in two seperate parts to handle endpoint subscriptions neatly: ZeroMQ manages subscriptions by checking for byte equality between the subscription filters and every inbound message. More info. Byte-string equality is easy on strings, and hard on JSON coming from an unsorted hashmap ;)
we put the hostname origin at the end because who cares where its coming from? only what the API startpoint is.
All wire bytes are UTF8-encoded string, and start with the string 'all'.
You can use squidwork.quick.pub
and squidwork.quick.sub
to
create the type of ZeroMQ socket you need to send or receive events,
and never have to import zmq yourself, or worry about the other
socket types. Here is a simple event publisher and a reciever:
# event emitter
from squidwork.quick import pub
from squidwork import Sender
from time import sleep
endpoint = Sender(pub('tcp://127.0.0.1:6696'))
while True:
sleep(3)
endpoint.send(['hello', 'world'])
# event listener
from squidwork.quick import pub
from squidwork import Sender
reciever = Reciever(sub('tcp://127.0.0.1:6696'))
while True:
# this call blocks
message = reciever.recieve()
print message.origin
print message.time
print message.content
This module provides an implementation of the protocol over Websockets for Javascript clients. I'm only planning on implementing subscribing features, no publishing. Here's a somple example:
<script type="text/javascript" src="http://squidwork.internal/squidwork.js"></script>
# squidwork imported into `window` namespace already
sub = squidwork.subscribe 'tcp://192.168.0.1:9999', 'ear/active', (message) ->
document.body.innerHTML = message.content
# re-assign sub.fn to change the handler
sub.fn = (message) ->
console.log("Got message: ", message)
# we don't care anymore
sub.unsubscribe()
This is implemented on top of the Tornado event loop in python and ZeroMQ streams. To run the bridge, execute the following:
python -m squidwork.web -c ./config.yml
If you direct your browser to http://localhost:8888
you will see a mostly-blank
page that directs you to open your eveloper console. From there you can inspect
the configuration data and create new subscriptions!
At this point, we've got a full blown web micro-library going on in
squidwork.web.handlers
, and a pretty great Javascript squidwork API, so why
not make a real front-end app of equal quality?
The monitor is a graphical log of recent squidwork messages. depending on configuration, it displays a list of the last N squidwork events. It can filter for uniqueness (squashing a bunch of duplicates), or just display things chronologically.
I feel like I might be pioneering strange application architectures at this point. Here's what we've got so far:
- ZeroMQ pub-sub data source
- data becomes JSON
- some is stream over WebSockets to front-end
- some is plain-jain JSON
- tornado webserver to deliver templates and websockets
- templates mostly static, only dynamic so we can ship over config values
- only 4 lines of HTML
- coffeescript front-end app consumer
- views rendered on the client by Mithril.js
I can't tell if Mithril.js is awesome or what. Look at the occlusion culling bit the creator does: we've got a full data binding application (or at least view-controller) in 43 lines. So refreshing compared to Ember.js.
I'm very satisified with my Mithril experiece. it's considerably less complex to use Mithril than Mustache or any other "logic-free" template system that then requires an aditional view layer, and it seems pretty fast. I'm just using it for rendering (no two-way data bindigns), but it's still very convinient. Hooray for immidiate-mode GUIs.
The class squidwork.config.Config
is a unified command line argument and
configuration file loader.
Unified command line argument and configuration file loader. Abstracts
over argparse and config file direct access. Uses argparse to create the
command-line interface, and you may pass kwargs through to the
ArgumentParser when instantiating a new Config.
Define options that your app requires using the option
method, and then
retrieve command line args and load the config file with the retrieve
method.
After calling Config#retrieve
you may access your config options as
dot-properties of the config object. Retrieve the configuration will also
populate the Service index, see squidwork.config.service for more
information
Usage example:
config = Config('my app')
config.option('dog-name')
# Config#option passes args through to
config.option('threshold', type=float, help="sound level before we scold")
config.retrieve()
# Dashed option names are python-ized by replacing each - with a _
>>> dog = Dog(name=config.dog_name)
see test/beeper.py for a concrete usage example
- pyzmq
- pyyaml (for config)
- tornado (for squidwork.websocket, which depends on config)
The tests in tests/
come in pairs, one sender and one receiver for each
strategy.