Skip to content

gRPC server and client for HBase as an alternative to Thrift

Notifications You must be signed in to change notification settings

cyberbot97/gRPC-HBase

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

gRPC HBase

This repository contains a gRPC server which emulates the HBase Java API and clients for this server. The intent is to make this a kubernetes-first application which will run as a sidecar container in the pod of an application. The gRPC service is then exposed over localhost to your app.

Server

The server is written in Scala. It supports Kerberos authentication. Because this server will run alongside each container, each instance can manage its own keytab. Fetched results are streamed to the client cell-by-cell.

Clients

At this point only a python client is available. It has integration with pandas to load results into a DataFrame.

TODO

  • Add support for more operations to server (Put, Delete)
  • Test
  • Add documentation
  • Create Dockerfile for server
  • Create clients in more languages (Rust (with Tonic), ...)
  • Structure Python client as proper library
  • Create example k8s deployments with server and client in one pod (targeting Minikube)

Run example

Create a resource file named secrets.conf in server/src/main/resources and populate it with HBase (and Kerberos if wanted) config.

Then start the server and wait for it to be ready:

cd server
sbt run

Create a virtual python environment in your preferred way and activate it.

Now install dependencies:

pip install -r python-client/dev-requirements.txt

Then run the client:

python python-client/src/main.py

About

gRPC server and client for HBase as an alternative to Thrift

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 71.6%
  • Scala 27.6%
  • Shell 0.8%