A benchmark project of Avro, JSON, Protocol Buffer created at Insight.
Slides can be checked at http://www.slideshare.net/XuechaoWu/xuechao-serial-war-63753460
The dashboard currently has no data feed. Leave an issue for a live demo.
-
Metrics
- Use raw JSON, data serialized in avro and data serialized in protocol buffer to benchmark their performance under
- High volume throughput
- Narrow/Wide/Nested data type
- real-time MapReduce
- Use raw JSON, data serialized in avro and data serialized in protocol buffer to benchmark their performance under
-
Data Source
- JSON formatted
- real-time data from Twitter Streaming API
- Synthesized person info data
-
Ingestion
- 3 different producers into 3 topics
- avro
- json
- protobuf
- 3 different producers into 3 topics
-
Stream Processing
- Spark Streaming API
-
Cache
- Redis cluster
-
API
- Written in Flask
-
Front-End
- Bootstrap, d3.js and highcharts.js