Introduction
Welcome to the documentation of the project RIK, we hope you will enjoy your journey here. In case of mistakes or typos, please open an issue or a pull request on the GitHub repository.
Getting started
Prerequisites
Our worker component, Riklet, is currently only available for Linux systems with a x86_64 architecture.
Install packages required to build the project:
- Ubuntu:
sudo apt update && sudo apt install libssl-dev protobuf-compiler - Fedora:
sudo dnf update && sudo dnf install openssl-devel protobuf-compiler protobuf-devel
Clone the project
Start by cloning the project using Git:
git clone https://github.com/rik-org/rik.git
Start a cluster
Using Docker
docker compose up
Build from source
Be aware that each component of the project is a separate binary, and that you need to execute them in a specific order.
Build all components of the project
cargo build --release
Start the scheduler in a terminal
cargo run --release --bin scheduler
Start the controller in a terminal
DATABASE_LOCATION=~/.rik/data cargo run --release --bin controller
Start the worker in a terminal
sudo ./target/release/riklet
If you experience any issue, please refer to the troubleshooting, if you can't find a solution, please open an issue.
Create your first workload
Create a definition of your workload
Create a workload using example in examples/workload-1.json:
# Create an alpine container workload
RIKCONFIG=docs/src/examples/config.json cargo run \
--bin rikctl -- create workload \
--file docs/src/examples/workloads/workload-1.json
# The ID of the workload is returned, it will be useful next
# Workload alpine has been successfully created with ID : "0e4c1da4-0277-4088-9f37-8f445cbe8e46"
Deploy an instance
Based on your workload ID you can now deploy an instance:
# Please replace the following value with the ID of your workload
export WORKLOAD_ID=0e4c1da4-0277-4088-9f37-8f445cbe8e46
RIKCONFIG=docs/src/examples/config.json cargo run \
--bin rikctl -- create instance \
--workload-id ${WORKLOAD_ID}
Check your instance
You should now see your instance running:
RIKCONFIG=examples/config.json cargo run \
--bin rikctl -- get instances
Configuration
You can configure a remote cluster by setting the RIKCONFIG environment variable
to the path of a configuration file. Here is an example of configuration:
{
"cluster": {
"name": "default",
"server": "http://127.0.0.1:5000"
}
}
Workloads
A workload is a definition of a piece of software that should be run on a cluster. It can be seen as a collection of containers, VM and other resources that are required to run a software. Defining a workload won't deploy an instance immediately, but it will be used as a definition to deploy an instance.
The implementation only support JSON format, but the API is designed to be extensible to support other formats in the future. In case you expect to have a specific format, please open an issue to discuss it.
RIK supports two types of workload:
- Pod: A pod is a collection of containers that should be run together. As you might imagine in Kubernetes, for each instance, they will be packed together and share the same resource limits. They are the most basic workload type, and they are used to run a single application.
- Function1: A function is a definition for a microVM that should be running in a more isolated way than a pod. This kind of workload supports network capabilities, it means it can be exposed easily on the network.
Lifecycle
Workloads have a common lifecycle which goes through various states. Each time the state is updated at any step of the lifecycle, the scheduler and the controller are aware of it and will act accordingly. In the case a failure is detected, the controller won't try to recover the workload, but it will give the error message to the user.
sequenceDiagram
actor Bob
participant Primary as RIK Primary Node
participant Scheduler as RIK Scheduler
participant Secondary as RIK Secondary Node
Bob-->>Primary: Request new instance <br/>of a workload
note over Primary: Status: PENDING
Primary-->>Scheduler: Request Schedule
note over Scheduler: Status: PENDING
Scheduler-->>Secondary: Determine appropriate node and<br/> order scheduler
note over Secondary: Status: CREATING
Secondary-->>Secondary: Download and run <br/> necessary components
note over Secondary: Status: RUNNING
Secondary-->>Scheduler: Return instance running OK
Scheduler-->>Primary: Return instance running OK
Primary-->>Bob: Return instance running OK
JSON Schema Reference
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://rik-org.github.io/rik/workloads/schema-v1.json",
"title": "Workload-v1",
"description": "Reference for workload in RIK with apiVersion v1",
"type": "object",
"properties": {
"apiVersion": {
"description": "The version associated to the current schema",
"type": "string",
"default": "v1"
},
"kind": {
"description": "The kind of the resource",
"type": "string",
"enum": [ "Pod", "Function" ]
},
"name": {
"description": "Unique name of the workload",
"type": "string"
},
"replicas": {
"description": "Number of replicas expected (won't be deployed)",
"type": "integer",
"minimum": 1
},
"spec": {
"description": "Full specification of the workload",
"type": "object",
"required": [ "containers", "function" ],
"properties": {
"containers": {
"description": "List of containers to be deployed",
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name of the container"
},
"image": {
"type": "string",
"description": "Image to be used for the container"
},
}
}
},
"function": {
"description": "Function to be deployed",
"type": "object",
"properties": {
"execution": {
"type": "object",
"properties": {
"rootfs": {
"type": "string",
"description": "Rootfs to be used for the container, must a be URL that can be publicly accesed"
}
}
}
}
}
}
}
},
"required": [ "apiVersion", "kind", "name", "replicas", "spec" ]
}
This workload will probably be renamed in the future, as it is not
strictly related to functions.
Troubleshooting
cargo build fails because cannot build openssl-sys
This is due to missing packages in your system, install libssl-dev to fix this.
- Ubuntu:
sudo apt update && sudo apt install libssl-dev protobuf-compiler - Fedora:
sudo dnf update && sudo dnf install -y openssl-devel protobuf-compiler protobuf-devel
protoc failed: Explicit 'optional' labels are disallowed in the Proto3 Syntax
This is due to the version of protoc you are using, you need to use version
3.14 or later.
controller failed to run: panic, Permission denied
Controller component tries to create a folder in /var/lib/rik/data to store
your cluster data. You can either run the controller as root or change the saved
directory by setting DATABASE_LOCATION to another folder location.
Controller
Configuration
| Environment variable | Default | Description |
|---|---|---|
DATABASE_LOCATION | /var/lib/rik/data/ | Database data location |
SCHEDULER_URL | http://localhost:4996 | Host location of the scheduler |
PORT | 5000 | Port to listen on |
Database structure
Workloads:
-
element_type:/workload -
element_id:/workload/${WORKLOAD_KIND}/${NAMESPACE}/${WORKLOAD_NAME}- WORKLOAD_KIND: One of
pods,function - NAMESPACE: Static
default - WORKLOAD_NAME: Dynamically defined
- WORKLOAD_KIND: One of
Instances:
-
element_type:/instance -
element_id:/instance/${WORKLOAD_KIND}/${NAMESPACE}/${INSTANCE_NAME}- WORKLOAD_KIND: One of
pods,function - NAMESPACE: Static
default - INSTANCE_NAME: Dynamically defined
- WORKLOAD_KIND: One of
Network
This project has network features, it's state is unstable and should be used with caution as it is managing your network interfaces and routing. We are doing our best not to break your system's network!
Workloads
Current workloads cannot be configured with network implementation, however
Function workload implement a first version of network configuration which
can't be configured yet.
Riklet SDN
This component onboard a network component which will manage network exposure
and routing. Depending on the workload, it will be configured to use a specific
network implementation. For now, only Function workload have an implementation
of network configuration. This implementation is based on iptables and
rtnetlink.
Function network implementation
This network feature allows you to forward traffic from a specific port to a
Function instance port. We achieve this using iptables, a widely used linux
tool for managing network traffic. The translation of IP and port is targetting
a TAP interface on the
machine that is communicating with the Function instance (microVM).
┌──────────────────────────────────────────────────────────────────┐
│ Host Machine (riklet) │
│ │
│ │
│ ┌─────────────────────────────┐ ┌─────────────────────────┐ │
│ │Iptables │ │ Function Instance │ │
│ │ │ │ │ │
│ │ ┌─────────────────────────┐ │ │ │ │
│ │ │APPLY NAT ON │ │ │┌───────────────────────┐│ │
│ │ │host:${port} │ │ ││ Guest_veth ││ │
│ │ │ │ │ ││ ││ │
│ │ │TO │─┼┐ │└───────────────────────┘│ │
│ │ │host_tap:${service_port} │ ││ │ ▲ │ │
│ │ │ │ ││ └────────────┼────────────┘ │
│ │ └─────────────────────────┘ ││ │ │
│ │ ▲ ││ │ │
│ └──────────────┼──────────────┘│ ┌───────────────────────┐ │
│ │ │ │ Host_tap │ │
│ │ └──▶│ │ │
│ │ └───────────────────────┘ │
│ ┌───────────────────────┐ │
│ │Host Ethernet Interface│ │
│ │ │ │
│ └───────────────────────┘ │
│ ▲ │
└─────────────────┼────────────────────────────────────────────────┘
│
This is what the network configuration looks like when you deploy a Function
instance with a port mapping, please not it is very specific to Function. The
host_tap interface is created by the
riklet and is used to communicate with the Function instance. The Guest_veth
interface is created by the firecracker microVM and is used to communicate
with the host_tap interface. The host_tap is connecteed to the internet and
is not restricted in bandwidth.
Iptables
Riklet will use a custom chain called RIKLET on the table nat to do DNAT (Destination NAT), it
matches two use cases:
- Local processes: when another workload wants to communicate with a Function instance
- Internet: when the workload needs to be exposed externally on the worker node
.─────────────────. .─────────────────.
,─' '─. ,─' '─.
( Local processes ) ( Internet )
`──. _.─' `──. _.─'
`───────────────' `───────────────'
│ │
│ │
│ │
▼ ▼
┌────────────────────┐ ┌────────────────────┐
│ OUTPUT (nat) │ │ PREROUTING (nat) │
└────────────────────┘ └────────────────────┘
│ │
│ │
│ ┌──────────────────────────┤
│ │ │
▼ ▼ ▼
┌────────────────────┐ ┌────────────────────┐
│ RIKLET (nat) │───────────▶│ FORWARD │
└────────────────────┘ └────────────────────┘
│
│
│
│
▼
┌────────────────────┐
│ POSTROUTING │
└────────────────────┘
│
│
│
▼