Kafka-HUE is a HUE application to admin and manage a pool of Apache Kafka clusters.
- Multi cluster support
- Cluster Overview (Brokers, Topics, Consumers, etc.)
- Kafka topics administration (add)
- Custom Dashboards based on Ganglia metrics. Configure Kafka to export JMX metrics to Ganglia and Kafka-HUE will be able to show all of those metrics specifiying them in a config file (metrics.ini).
- View offset ranges by topic.
- Config validator at HUE start.
- HUE 3.8 or higher.
- Ganglia
- Kazoo 2.2.1
- ReportLab 2.X for Python 2.5 or 2.6. ReportLab 3.X requires Python versions 2.7 or higher.
- Apache Kafka local installation is needed. It is not necessary that kafka services are up. It only needs kafka scripts
- Python
- Django
- Mako
- jQuery
- Knockout.js
- Bootstrap
To get the Kafka-HUE app integrated and running in your HUE deployment:
$ sudo $HUE_HOME/build/env/bin/python $HUE_HOME/build/env/bin/pip install kazoo
For Python 2.5 or 2.6
$ sudo $HUE_HOME/build/env/bin/python $HUE_HOME/build/env/bin/pip install "reportlab<3.0"
For Python 2.7 or higher.
$ sudo $HUE_HOME/build/env/bin/python $HUE_HOME/build/env/bin/pip install reportlab
$ git clone https://github.com/keedio/kafka-hue.git
$ mv kafka-hue/kafka $HUE_HOME/apps
$ cd $HUE_HOME/apps
$ sudo ../tools/app_reg/app_reg.py --install kafka --relative-paths
$ chown -R hue: kafka/
Modify the hue.ini config file as follows and restart HUE.
Configs needed in hue.ini config file.
[kafka]
[[clusters]]
[[[default]]]
# Zookeeper ensemble. Comma separated list of Host/Port.
# e.g. localhost:2181,localhost:2182,localhost:2183
zk_host_ports=localhost:2181,localhost:2182,localhost:2183
# Path to brokers info in Zookeeper Znode hierarchy
brokers_path=/brokers/ids
# Path to consumers info in Zookeeper Znode hierarchy
consumers_path=/consumers
# Ganglia Server
# e.g. http://localhost
ganglia_server=http://localhost
# Ganglia Data Source
# e.g. GangliaCluster
ganglia_data_source=GangliaCluster
Metrics example
[BrokerTopicMetrics.BytesInPerSec]
key = Count,OneMinuteRate,FiveMinuteRate,FifteenMinuteRate,MeanRate
[BrokerTopicMetrics.BytesOutPerSec]
key = Count,OneMinuteRate,FiveMinuteRate,FifteenMinuteRate,MeanRate
[BrokerTopicMetrics.FailedFetchRequestPerSec]
key = Count,OneMinuteRate,FiveMinuteRate,FifteenMinuteRate,MeanRate
[BrokerTopicMetrics.FailedProduceRequestPerSec]
key = Count,OneMinuteRate,FiveMinuteRate,FifteenMinuteRate,MeanRate
[BrokerTopicMetrics.MessagesInPerSec]
key = Count,OneMinuteRate,FiveMinuteRate,FifteenMinuteRate,MeanRate
To compile the locales:
Set the ROOT variable in the Makefile file pointing to the HUE installation path.
Compile with make.
$ cd $HUE_HOME/apps/kafka
$ sudo make compile-locale
Restart HUE.
Apache License, Version 2.0 http://www.apache.org/licenses/LICENSE-2.0
-- Daniel Tardón dtardon@keedio.org