FEDB is a NewSQL database optimised for realtime inference and decisioning applications. These applications put real-time features extracted from multiple time windows through a pre-trained model to evaluate new data to support decision making. Existing in-memory databases cost hundreds or even thousands of milliseconds so they cannot meet the requirements of inference and decisioning applications.
FEDB use a double-layered skiplist as the core data structure to store all the data in memory and optimize sql compilation to improve the execution performance.
-
High Performance
The benchmark shows that FEDB can be one to two orders of magnitude faster than SingleStore and SAP HANA.
-
SQL Compatible
FEDB is compatible with most of ANSI SQL syntax. You can implement your aplications with sqlalchemy or JDBC.
-
Online-offline Consistency
Machine learning applications developed by FEDB can be launched simply and ensure online and offline consistency, which greatly reduces the cost.
-
High Availability
Support auto failover and scaling horizontally.
Note: The latest released FEDB is unstable and not recommend to be used in production environment.
docker pull 4pdosc/centos6_gcc7_hybridsql:0.1.1
git clone --recurse-submodules https://github.com/4paradigm/fedb.git
cd fedb
docker run -v `pwd`:/fedb -it 4pdosc/centos6_gcc7_hybridsql:0.1.1
cd /fedb
sh steps/init_env.sh
sh steps/install_hybridse.sh
mkdir -p build && cd build && cmake ../ && make -j5 fedb
- Predict taxi trip duration
- Detect the healthy of online transaction and make an alert -oncoming
- Online real-time transaction fraud detection -oncoming
In AI scenarios most real-time features are time-related and required to be computed over multiple time windows. So we use computation TopN queries as benchmark scenario.
The server spec is as follows:
| Item | Spec |
|---|---|
| CPU | Intel Xeon Platinum 8280L Processor |
| Memory | 384 GB |
| OS | CentOS-7 with kernel 5.1.9-1.el7 |
The benchmark result shows that FEDB can be one to two orders of magnitude faster than SingleStore and SAP HANA.
FEDB is currently compatible with mainstream DDL and DML syntax, and will gradually enhances the compatibility of ANSI SQL syntax.
- [2021H1] Support the standard syntax of Window, Where, Group By and Join ect.
- [2021H1&H2] Expand AI-oriented SQL syntax and UDAF functions.
In order to meet the high performance requirements of realtime inference and decisioning scenarios, FEDB chooses memory as the storage engine medium. At present, the memory storage engine used in the industry has memory fragmentation and recovery efficiency problems. FEDB plans to optimize the memory allocation algorithm to reduce fragmentation and accelerate data recovery with PMEM(Intel Optane DC Persistent Memory Module).
- [2021H1]Provide a new strategy of memory allocation to reduce memory fragmentation.
- [2021H2]Support PMEM-based storage engine.
FEDB has python client and java client which support most of JDBC API. FEDB will make a connection with big data ecosystem for integrating with Flink/Kafka/Spark simplily.
- [2021H1&H2]Support Flink/Kafka/Spark connector.
- Report bugs, ask questions or give suggestions by Github Issues.
- Cannot find what you are looking for? Have a question or idea? Please post your questions or comments on our slack.
Apache License 2.0