Visit nuclio.io for product information and news and a friendly web presentation of the nuclio documentation.
Check out the global nuclio hackathon for a some serverless fun and a chance to win a Phantom 4 drone!
- Overview
- Why another "serverless" project?
- Quick-start steps
- High-level architecture
- Function examples
- Global nuclio hackathon
- Further reading
nuclio is a new "serverless" project, derived from iguazio's elastic data life-cycle management service for high-performance events and data processing. You can use nuclio as a standalone binary (for example, for IoT devices), package it within a Docker container, or integrate it with a container orchestrator like Kubernetes.
nuclio is extremely fast. A single function instance can process hundreds of thousands of HTTP requests or data records per second. This is 10-100 times faster than some other frameworks. To learn more about how nuclio works, see the nuclio architecture documentation and watch the technical CNCF nuclio presentation and demo (slides can be found here).
Note: nuclio is still under active development and is not recommended for production use.
We considered existing cloud and open-source serverless solutions, but none addressed our needs:
- Real-time processing with minimal CPU and I/O overhead and maximum parallelism
- Native integration with a large variety of data sources, triggers, and processing models
- Abstraction of data resources from the function code - to support code portability, simplicity and data-path acceleration
- Simple debugging, regression testing, and multi-versioned CI/CD pipelines
- Portability across low-power devices, laptops, on-prem clusters and public clouds
We designed nuclio to be extendable, using a modular and layered approach that supports constant addition of triggers and data sources. We hope many will join us in developing new modules, developer tools, and platforms.
The simplest way to explore nuclio is to run its graphical user interface (GUI) of the nuclio playground. All you need in order to run the playground is Docker:
docker run -p 8070:8070 -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp nuclio/playground:stable-amd64
Browse to http://localhost:8070 and deploy one of the example functions, or write your own function. When run outside of an orchestration platform (for example, Kubernetes or Swarm), the playground will simply deploy to the local Docker daemon.
For a complete step-by-step guide to using nuclio over Kubernetes, either with the playground UI or the nuclio command-line interface (nuctl), see Getting Started with nuclio on Kubernetes or Getting Started with nuclio on Google Kubernetes Engine (GKE), or Getting started with nuclio on Azure Container Services (AKS).
The following image illustrates nuclio's high-level architecture:
Following is an outline of the main architecture components. For more information about the nuclio architecture, see Architecture.
A processor listens on one or more triggers (for example, HTTP, Message Queue, or Stream), and executes user functions with one or more parallel workers.
The workers use language-specific runtimes to execute the function (via native calls, shared memory, or shell). Processors use abstract interfaces to integrate with platform facilities for logging, monitoring and configuration, allowing for greater portability and extensibility (such as logging to a screen, file, or log stream).
A controller accepts function and event-source specifications, invokes builders and processors through an orchestration platform (such as Kubernetes), and manages function elasticity, life cycle, and versions.
The playground is a standalone container microservice that is accessed through HTTP and includes a code-editor GUI for editing, deploying, and testing functions. This is the most user-friendly way to work with nuclio. The playground container comes packaged with a version of the nuclio builder.
A builder receives raw code and optional build instructions and dependencies, and generates the function artifact - a binary file or a Docker container image that the builder can also push to a specified image repository. The builder can run in the context of the CLI or as a separate service, for automated development pipelines.
A dealer is used with streaming and batch jobs to distribute a set of tasks or data partitions/shards among the available function instances, and to guarantee that all tasks are completed successfully. For example, if a function reads from a message stream with 20 partitions, the dealer will guarantee that the partitions are distributed evenly across workers, taking into account the number of function instances and failures.
Functions can be invoked through a variety of event sources that are defined in the function (such as HTTP, RabbitMQ, Kafka, Kinesis, NATS, DynamoDB, iguazio v3io, or schedule). Event sources are divided into several event classes (req/rep, async, stream, pooling), which define the sources' behavior. Different event sources can plug seamlessly into the same function without sacrificing performance, allowing for portability, code reuse, and flexibility.
Data-binding rules allow users to specify persistent input/output data resources to be used by the function. (Data connections are preserved between executions.) Bound data can be in the form of files, objects, records, messages, etc. The function specification may include an array of data-binding rules, each specifying the data resource and its credentials and usage parameters. Data-binding abstraction allows using the same function with different data sources of the same type, and enables function portability.
The nuclio SDK is used by function developers to write, test, and submit their code, without the need for the entire nuclio source tree.
The following sample function implementation uses the Event and Context interfaces to handle inputs and logs, returning a structured HTTP response; (it's also possible to use a simple string as the returned value).
In Go
package handler
import (
"github.com/nuclio/nuclio-sdk-go"
)
func Handler(context *nuclio.Context, event nuclio.Event) (interface{}, error) {
context.Logger.Info("Request received: %s", event.GetPath())
return nuclio.Response{
StatusCode: 200,
ContentType: "application/text",
Body: []byte("Response from handler"),
}, nil
}In Python
def handler(context, event):
response_body = f'Got {event.method} to {event.path} with "{event.body}"'
# log with debug severity
context.logger.debug('This is a debug level message')
# just return a response instance
return context.Response(body=response_body,
headers=None,
content_type='text/plain',
status_code=201)More examples can be found in the hack/examples nuclio GitHub directory.
Sign up for the global online nuclio hackathon, build the greatest serverless application, and you just might win a Phantom 4 drone!
Judges include
-
Brendan Burns
Kubernetes Co-Founder -
Lee Calcote
Head of Technology Strategy, SolarWinds -
Yaron Haviv
CTO, iguazio -
Orit Nissan-Messing
Chief Architect, iguazio
Submissions will be accepted until Jan 29 2018. Register now.
- Setup
- Getting Started with nuclio on Kubernetes
- Getting Started with nuclio on Azure Kubernetes Service (AKS)
- Getting Started with nuclio on Google Kubernetes Engine (GKE)
- Getting Started with nuclio On Raspberry Pi (coming soon)
- Concepts
- Architecture details
- Configuring a Function
- Build Process (coming soon)
- Deploy Process (coming soon)
- Kubernetes
- Examples
- Roadmap
- Contributing
- Reference
- Media
For support and additional product information, join the active nuclio Slack workspace.