Project start: 2014/8/8
Distributed document-oriented database for small PCs (incl. Raspberry Pi), written in Python:
- Tornado-based
- CRUD/RPC supported
- Transaction supported
- JSON-oriented (not YANG)
- Schema-less
- Extensible with plugins
- NAT traversal (HTTP/WebSocket)
- Python and Go driver
- Easy to use
$ tega-cli
tega CLI (q: quit, h:help)
[tega: 0] put a.b.c
1
[tega: 1] put a.x
2
[tega: 2] get a
b: {c: 1}
x: 2
- Simplicity rather than high-performance and rigid schema
- NOSQL for non big data (small data)
- Embeddable data base
- Focus on config and operational data management
- Support various integration patterns: data change notification, messaging and rpc
- Easy-to-use APIs
- Concurrenty support with coroutine
- NAT/HTTP-Proxy traversal: REST and WebSocket
- In general, you had better use SQL (or SQL-like db) for storing config of a whole network.
- You define Nodes table, Links table, Subnets table... a lot of tables.
- A subset (a set of rows from those tables) of the network config should be stored in a local db at each router. In general, such data has a tree structure (e.g., SNMP MIB).
- There are two choices: SQL or graph(tree) database. It depends... In my case, I have chosen grap(tree) database, since I want version controll for the config (like "git").
- NLAN
- BBR remote config
- OpenWrt remote config
- Instant VPN (L2 or L3)
- IOT
You need to have python3.5 installed on your Debian/Ubuntu Linux.
$ pip install tornado
$ pip install httplib2
$ pip install pyyaml
$ pip install readline
$ git clone http://github.com/araobp/tega
$ cd tega
$ python setup.py install
Start tega server with sample options, like this:
$ cd tega/scripts
$ ./global
And start tega CLI to use CLI commands:
$ tega-cli
[cli.py]
|
[driver.py]
| |
+---------------------+ +-----------------------+
| |
Tornado Tornado
[server.py] ------------ REST/WebSocket ---------[server.py]
: :
[idb.py]...[tree.py] [idb.py]...[tree.py]
: :
-------- --------
/tega.db/ /tega.db/
--------- ---------
commit-log commit-log
- tree structure
- tree structure implemention: Cont class and its attributes
- YANG-tega comparisons
- tega message format
- tega message sequence diagram
- tega db commit-log format
- sync path and notifications
- subscription scope
- Guideline
For example,
- My neutron-lan project has taken the pattern [D]. The implementation of the agent is not simple.
- My nlan project has been taking the pattern [A], since Docker is for an immutable infrastructure. The implementation of the agent is quite simple.
- In general, commercial-grade network controllers take the pattern [E]. It requires a development team with a lot of software engineers.
- Signalling protocol developers think differently... My another drawing
You had better study how databases (incl. NOSQL ones) realize data synchronization in a cluster.
- Tega db is still in alpha release and part of the documentation here has not been implemented yet.
it is a schema-less database and the performance (especially write-performance) of this database is not good:
- dynamic typing
- nested hash map (hash collisions, rehashing...)
- it makes use of for-loop a lot (Python's for-loop performance is not good)
- it saves every commit log onto a storage