Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction to Yeet

Yeet is an PULL-based deployment server for Nix closures. Yeet only acts as an intermediary and does leave you open to choose your own nix cache and build-systems.

The indent of Yeet is to:

  • Provide an easy way to manage your whole fleet of devices - from homelab to enterprise
  • Not pose any restrictions onto how your nix derivation is built
  • Allow for clients to be offline when the updates is created
  • Secure defaults to protect your fleet

Why do I want Yeet?

Wheter you are a person who manages their homelab server or an business which manages a large fleet of devices. Installing, configuring and maintaing your devices imperatively will bite you back in the long run.

Push based deployment system do work really well if you only deploy to servers. Once you also want to provision devices that may not be online this gets a whole more complicated. Now you have devices that miss updates, fly under the radar and clog up your pipelines.

This is where Yeet comes into play. By reversing the role with an on-device agent, the clients now signal to the server that they are ready to receiver their update.

This allows to offload the heavy lifting of the build process to build-machines. Especially in infrastractures that include hundreds of devices the savings get noticeable.

Architecture

architecture-beta


    group clients[Fleet]

    service yeet-server(game-icons:catapult)[Yeet Server]

    service laptop(mdi:laptop)[Laptop] in clients
    service desktop(mdi:desktop-tower)[Desktop] in clients
    service server(mdi:server)[Server] in clients

    service cache(devicon:nixos)[Nix Cache]

    service source(logos:git-icon)[Fleet Definition]
    service build-machine(mdi:build)[Build Machine]
    service pipeline(logos:github-actions)[Build Pipeline]

    junction tl
    junction tlb
    junction tr
    junction trb
    source:B -- > T:pipeline
    pipeline:L -- R:tl
    build-machine:R -- R:tr
    pipeline:R --> L:build-machine
    tr:B -- T:trb
    tl:B -- T:tlb
    tlb:B --> T:yeet-server
    trb:B --> T:cache

    junction y_top
    y_top:B -- T:y_middle
    junction y_middle
    y_middle:B -- T:y_bottom
    junction y_bottom
    y_middle:L --> R:yeet-server

    laptop:L <-- R:y_top
    desktop:L <-- R:y_middle
    server:L <-- R:y_bottom


    junction c_top
    c_top:B -- T:c_middle
    junction c_middle
    c_middle:B -- T:c_bottom
    junction c_bottom
    c_middle:R -- L:cache

    laptop:R <-- L:c_top
    desktop:R <-- L:c_middle
    server:R <-- L:c_bottom

Quickstart

nix flake init --template github:srylax/yeet

Go to https://app.cachix.org/cache. Register if you have no account. Create a new binary cache - note the name you are going to need it later. Create a new authtoken on cachix. cachix authtoken <token>

yeet-server

You check out your current build with yeet vm my-nixos (only for nixos not for darwin) yeet publish --cachix <cache> -> client are now listed as unverified Get the code from your vm via yeet approve aegis <code> -> Client is now listed as verified Edit the file my-nixos/configuration.nix and add ripgrep: yeet publish --cachix <cache> -> client are now listed as unverified Client automatically get the update yeet log mynixos [TODO]

Endpoints

The Lifecycle of an client is as following:

stateDiagram-v2
    state if_state <<choice>>
    [*] --> New

    New --> Standby: Register
    Standby --> Updating: Update Available
    Updating --> if_state
    if_state --> Update: Successful
    if_state --> Rollback: Failure
    Rollback --> Standby: Mark update as Failure
    Update --> Standby: Update remote version

Varlink

Varlink allows the user facing cli to perform actions that would otherwise require configuration or authentication. The idea is that even if the user is not an admin, if the yeet server allows it, the user can perform system modifying action by leveragin the running yeet agent. Furthermore if the user wants to execute commands on the yeet server, instead of the cli executing the command itself it will ask the daemon to do it (TBD)

This is different for yeet server commands as these are for non interactive use or for debugging.

Security

The yeet-daemon will at startup create a socket at /run/yeet/agent.varlink. This socket is only read+write for the yeet group. Even for basic status queries the user has to be in the yeet group.

TBD: If the user whishes to execute a varlink method that requires more permissions e.g. detach the daemon, a polkit request is created to authenticate the user. This allows administrators to fine-tune the actions a user can take on the yeet daemon.

An example configuration includes the user always in the yeet group (to allow the status query) but disallows any higher functions.

Design

The varlink thread is not coupled with the agent thread at all. There is no communication that exists between the different thread. Meaning that all information the daemon requires, besides configuration, he has to get by himself. This includes querying the yeet server for information about its state. Because the whole yeet agent does not contain any state this will not inconsistencies. But there is still room for race conditions: If the user performs a modifying action like switching to a new system while the agent thread is currently switching systems themself this can lead to race conditions between the two threads (To be verified). A simple fix is to implement a simple Thread Lock when any modifying actions are executed (TBD).

Detach aka Taskforce mode (TBD)

Sometimes we want to allow to switch to a new system locally. This mean ignoring the version that is provided by the server and instead using a locally supplied one.

This process is called Detaching. Once a client decides to start using the server supplied version again the client gets re-Attached. Detaching is usefull in multiple different scenarios:

  • Testing a configuration before rolling configuration
  • Installing time critical services (e.g. if you are in an organization where modifying the system is not the norm)

Design

Because the daemon does by design not store any of its state this also includes the current state of attachment. Meaning that if the agent wants to detach he has to consult the server first. This parts also allows for custom policy and rules to apply because the server can refuse the detachment.

Once the server allowed the detachment the server is always replying with the AgentAction::Detach this signals the agent to allow any detach requests. The agent has to never store information about its state because the server will tell him. AgentAction::Detach only changes the actions taken by the Varlink service. The agent part of the daemon still continues to report the current system in the specified time interval, providing the server with information about the current closure.

Local only detach

There may be times were no connection to the yeet server is possible or event wanted. For this event the --force option exists, signaling the daemon to not contact the server and instead force-switch to a new derivation. The implication is, that once the daemon regains connection to the yeet server he will automatically switch to provisioned state and use the server provided version, rolling back any changes made by the force switch.

Server

The server should still save updates for a detached client. Once the agent attaches he should get the latest version instead of the version when he detached.

Yeet secret management

GOAL

As a NixOS admin I want to be able to include nix secrets in my os configuration. It is not advised to include the secrets directly in the nix config because this will publicly expose the secrets in the nix-store.

Existing solutions

Widely used solutions to this problems are tools like agenix or sops-nix. These tools encrypt the secret with the public key of the target and then decrypt them once the system is enrolled on the target system with the sshd identity key. This ensures that only the intended host can decrypt the secrets.

This solution has a relative high initial configuration cost and also has the chicken-egg problem: Before we can encrypt a secret we have to get its public key but before we can get the system we have to setup the system.

Furthermore, if we have n hosts, s secrets and a hosts we have to encrypt the secret with (n*s*a) keys. Because administrators will have to be able to decrypt and re-encrypt the secret once a new host needs access to the secret.

These tools require a lot of manual work, it is possible to automate this workflow.

Morph

Morph is another nix deployment tool, it implements secret managing. Morphs secret management is realtively stupid. It simply copies file to the target. See https://github.com/DBCDK/morph/blob/master/examples/secrets.nix. This make it trivial for new users who do not have to manage keys. The chicken-egg problem is also resolved.

Proposed Solution

In Yeet we can make use of the best of both worlds. Yeet has information of the public keys of all hosts. This make it possible for the nix server to manage secrets in a secure way. The flow is best described by a quick example:

yeet secret add nix-cache-signing-key --from-file ../secrets/very-secret.txt Sequence:

  1. Yeet cli downloads the public server key
  2. Content from ../secrets/very-secret.txt is age encrypted with public server key
  3. The Yeet server creates a new secret with the name nix-cache-signing-key that is stored at rest encrypted

The admin them would reference this secret in the configuration something like this (pseudo syntax):

yeet.secrets."nix-cache-signing-key" = {
  path = "/var/secrets/very-secret.txt";
  mode = "0400";
  owner = "nginx";
  group = "nginx";
};

services.mysuperservices.settings.signingKeyFile = config.yeet.secrets."nix-cache-signing-key".path;

When building the nix derivation at build time, all yeet.secrets are stored in the output derivation.

To support secrets the yeet-agent would have to make changes to the update process:

  1. Agent receives SwitchTo->RemoteStore RemoteStore would now include a field fetch_secrets: Vec<String>
  2. Agent requests all secrets from fetch_secrets (this only includes netrc at the moment)
  3. Agent creates temporary netrc
  4. Agent fetches the RemoteStorePath
  5. Agent reads all the secrets that are required by the RemoteStorePath
  6. Agent tries to request all required secrets (Activation fails if not all secrets exist on the Yeet server)
  7. Secrets are created by the Agent.
  8. System Activation begins
  9. If System activation is successfull delete old generation of secrets else we delete the new generation

On the server the following happens:

  1. Server receives an Update for a host with some fetch_secrets set
  2. Server receives agent SystemCheck responds with RemoteStorePath
  3. Server receives secret request for netrc secret.
    1. Check access for $HOST for netrc
    2. Encrypt netrc with pub key of $HOST and respond
  4. Server receives request for multiple secrets
    1. Check access for all request secrets
    2. Encrypt all secrets with pub key of $HOST and respond

Advantages

A further advantage is that we can use ephemeral age keys used only for this single transaction.

Alternatives considered

Using agenix with yeet

An alternative would be that when building the derivation yeet could fetch the secrets at build time and age encrypt them with agenix. The main point against this is that yeet would just be an agenix wrapper and create dependencies at build time. With the proposed solution, secrets would be able to change without re-deploying the derivation. Which is a double edged sword because on the one side now derivations are no longer hermetic but on the other side secrets are not meant to be hermetic, they always rely on some state bound secret for unlocking even in the case of agenix.

Future extension

This proposal does not exactly define how the secrets are created on the host. This is because there are many possible solutions:

  • create a tmpfs so that they are destroy if powerloss happens
  • bind them to the tpm
  • plain write them to disk (simplest)

Especially the tpm solution can be interesting for critical secrets as this would protect a device against hardware attacks. Once a device is stolen you can revoke the access on the yeet server and then if the device is tampered with it will lose its secrets and will not be able to fetch them again from the yeet-server.

osquery

Intro to osquery

Osquery (https://osquery.io/) is a tool that allows to query information about an device in an relational-data model way. Osquery is however only the tool / daemon that runs on the device. Extraction of this information is possible in multiple ways:

  • logs forwarder to e.g. splunk
  • osquery extension
  • osquery remote api (tls extension)

The logs forwarding option the the most secure one. With the huge downside that you are not able to run distributed queries and have to make an config deployment each time you want to change a scheduled query.

If you want to use osquery as an investigation tool / information collection in an interactive way you either have to develop a new extension or use the remote api. To orchestrate all the nodes (hosts) you need a central server which implements the remote api. This design document describes this remote api implementation in yeet. To see possible alternative implementations see Alternative remote api implementations

Possible use-cases

The integration of osquery into yeet opens many possibilities for yeet:

  • Displaying more information about a host (in yeet)
  • Investigate a host with yeet query
  • Create warnings on queries e.g. if space is running out | as alternative to e.g. checkmk

The biggest improvement will however be for your specififc use-case. The osquery information can be forwarded to your SIEM and then used to enrich your detection / alerts:

  • “real”-time sofware inventory
  • integrate osquery data into checkmk
  • CVE scanning on installed packages
  • Windows Registry tests
  • Windows GPO tests

Alternative remote api implementations

Note: this is at the time of writing (03.2026). This is in no way a representative sample nor the opinion of yeet. This is a highly subjective view and is not meant derogatory.

TrendAI integration

TrendAI has a basic integration altough it is hard to use in the UI and currently 14! versions out of date. There is no easy

osctrl

Good initial idea and also has gained some more traction recently but is still very WIP. Deployment is a nightmare and not good documented.

Wazuh

Untested Documentation could be improved. If you already use Wazuh definitely go with it. If not this may be a a big dependency to use for osquery. Like shooting on pigeons with canons.

Elastic Security

Untested Seems to be a very good integration. If you already have Elastic go for it. Unfortunately, not open source.

Goals

  • Implement the osquery remote deployment https://osquery.readthedocs.io/en/latest/deployment/remote/
  • Be able to run yeet only as a osquery remote
  • Run distributed osquery
  • Support file carving
  • Support scheduled queries

Integration into yeet

All Api request / responses are implemented in an independant osquery-rs crate to allow other crates to easy implement the api.

Permissions

API Design

The following apis map to the osquery flags:

enroll_tls_endpoint = "/osquery/enroll";
config_tls_endpoint = "/osquery/config";
logger_tls_endpoint = "/osquery/log";
distributed_tls_read_endpoint = "/osquery/query/read";
distributed_tls_write_endpoint = "/osquery/query/write";
carver_start_endpoint = "/osquery/carver/init";
carver_continue_endpoint = "/osquery/carver/block";

Enrollment

The osquery client provides an enroll secret set with enroll_secret_path. Yeet expects the client to supply the content defined in the yeet secret called osquery-enroll. This secret has to be create via yeet secret add. The content is completely arbitrary. The node is free to use any host_identifier as long as it is UNIQUE for all nodes.

The response from the server responds with an UUIDv4 node_key.

Client Configuration

Future Work

  • integrate an cve scanning into yeet so that you can find cves in osquery
  • splunk forwarder
  • osquery extension