Daemon

The VM Registry Daemon (vm-registry-daemon) is the central orchestrator running on each host machine. It exposes a gRPC service over a Unix domain socket and manages all local VM operations — image storage, VM lifecycle through libvirt, virtual networking, VMCompose orchestration, and communication with the remote registry and auth servers.

Overview

The daemon is the bridge between the user-facing CLI and the underlying infrastructure. It receives gRPC requests from the CLI, translates them into libvirt API calls, filesystem operations, and HTTP requests to remote servers, then returns structured responses.

text

CLI ──gRPC/Unix Socket──► Daemon ──libvirt API──► QEMU/KVM
                              │
                              ├──HTTP──► Registry Server
                              ├──HTTP──► Auth Server
                              └──Filesystem/S3──► Image Store

Running the Daemon

For production installation, see the Installation page. The commands below are for development:

bash

cd vm-registry-daemon
go run ./cmd/vm-registry-daemon [FLAGS]

The daemon uses cobra for CLI argument parsing. All flags use double dashes (e.g., --socket).

Configuration Flags

Flag	Env Variable	Default	Description
`--socket <PATH>`	`VM_REGISTRY_SOCKET_PATH`	`/var/run/vm-registry.sock`	Unix socket path for gRPC (mutually exclusive w/ `--port`)
`--port <PORT>`	`VM_REGISTRY_SOCKET_PORT`	—	TCP port to listen on at `127.0.0.1` (mutually exclusive w/ `--socket`)
`--registry-url <URL>`	`VM_REGISTRY_SERVER_URL`	—	Remote registry server URL
`--auth-url <URL>`	`VM_AUTH_SERVER_URL`	—	Remote auth server URL
`--storage <BACKEND>`	—	`filesystem`	Storage backend: `filesystem` or `s3`
`--fs-path <PATH>`	`VM_REGISTRY_STORAGE_PATH`	`/var/lib/vm-registry`	Local filesystem storage path
`--log-level <LEVEL>`	`VM_REGISTRY_LOG_LEVEL`	`info`	Log level: `debug`, `info`, `warn`, or `error`
`--s3-bucket <BUCKET>`	—	—	S3 bucket name
`--s3-region <REGION>`	—	`us-east-1`	S3 region
`--s3-endpoint <URL>`	—	—	S3-compatible endpoint URL
`--s3-access-key <KEY>`	`AWS_ACCESS_KEY_ID`	—	S3 access key
`--s3-secret-key <SECRET>`	`AWS_SECRET_ACCESS_KEY`	—	S3 secret key
`--s3-skip-bucket-creation`	—	`false`	Skip automatic bucket creation

TIP

--socket and --port are mutually exclusive. Use --socket for Unix domain sockets (default) or --port for TCP listening. When neither is specified, the daemon checks the VM_REGISTRY_SOCKET_PORT env var for TCP, then VM_REGISTRY_SOCKET_PATH for Unix, falling back to /var/run/vm-registry.sock.

systemd Socket Activation

The daemon supports systemd socket activation via go-systemd. When launched by systemd with a passed file descriptor, the daemon uses the inherited listener instead of creating its own. This is the recommended approach for production deployments — the NixOS module configures this automatically.

The listener resolution order is:

systemd-passed listener (if present)
--socket or --port flag
VM_REGISTRY_SOCKET_PORT env var (TCP)
VM_REGISTRY_SOCKET_PATH env var (Unix)
Default: /var/run/vm-registry.sock

Examples

bash

# Unix socket (default)
go run ./cmd/vm-registry-daemon \
  --socket /tmp/vm-registry-daemon.sock \
  --registry-url http://127.0.0.1:8080 \
  --auth-url http://127.0.0.1:4078

# TCP listening
go run ./cmd/vm-registry-daemon \
  --port 50051 \
  --registry-url http://127.0.0.1:8080 \
  --auth-url http://127.0.0.1:4078

# S3 storage backend
go run ./cmd/vm-registry-daemon \
  --storage s3 \
  --s3-bucket vm-images \
  --s3-endpoint http://127.0.0.1:9000 \
  --s3-access-key admin \
  --s3-secret-key admin

# Debug logging
go run ./cmd/vm-registry-daemon --log-level debug

Internal Structure

The daemon is organized into the following internal packages:

Package	Responsibility
`cmd/vm-registry-daemon`	CLI entry point — cobra command definition, flag parsing, listener setup, and gRPC server initialization
`internal/service`	gRPC service implementations — one file per domain (auth, compose, context, images, vm, network, gc, console, etc.)
`internal/libvirt`	Libvirt wrapper — domain management, network management, bridge configuration, disk handling, cloud-init ISO generation, and helper utilities
`internal/vmfile`	VMFile parser and validator — YAML parsing, path resolution, schema validation, and image config generation
`internal/vmcompose`	VMCompose parser and validator — parses multi-VM compose files, validates structure and dependency graphs
`internal/storage`	Storage interface definition with `filesystem` and `s3` sub-packages for pluggable backends
`internal/config`	Configuration struct and validation — loads from flags and environment variables
`internal/client`	HTTP client for communicating with the registry and auth servers (push, pull, auth, context)
`internal/proto`	Generated gRPC/protobuf Go code from `vm-registry-proto`
`internal/logs`	Structured logging with gRPC interceptors for request/response tracing
`internal/utils`	Shared utility functions

gRPC Service

The daemon implements the VMService gRPC service defined in vm-registry-proto. The service is registered on a Unix domain socket with reflection enabled for debugging with tools like grpcurl.

The gRPC server is configured with:

4 GiB max message size for both send and receive — necessary for streaming large disk images during import
Unary interceptor for request/response logging
Stream interceptor for streaming call logging

Libvirt Integration

The daemon uses the libvirt.org/go/libvirt and libvirt.org/go/libvirtxml Go bindings for direct interaction with the libvirt hypervisor API. Key capabilities include:

Domain Management

Creating libvirt domains from image specs (CPU, memory, bootloader, disk)
Starting, stopping (graceful ACPI shutdown or forced destroy), restarting, and undefining domains
Listing running and defined domains with resource and status information

Network Management

Creating virtual networks in all supported modes: NAT, isolated, routed, bridged, open, and macvtap
Activating, deactivating, and deleting networks
Bridge interface configuration for bridged networking
DHCP range configuration for NAT and isolated networks

Cloud-Init

Generating cloud-init ISO images from user-data, meta-data, and network-config sources
Attaching cloud-init ISOs to VM domains for guest provisioning at first boot

Disk Management

Creating overlay disks (copy-on-write) for ephemeral VM storage
Resizing disk images when diskSize overrides are specified in VMCompose services

Image Storage

The daemon stores images locally in a content-addressable layout:

text

/var/lib/vm-registry/
├── manifests/
│   └── <repository>/
│       └── <tag>/
│           └── manifest.json
├── blobs/
│   └── sha256/
│       └── <digest>
└── runtime/
    └── <vm-name>/

manifests/ — Image manifests indexed by repository and tag, containing references to content-addressed blobs
blobs/ — Content-addressed storage for disk images and configuration blobs, keyed by SHA-256 digest
runtime/ — Ephemeral state for running VMs (cloud-init ISOs, overlay disks)

VMFile Processing

When a vmr import command is received, the daemon:

Receives the VMFile metadata and disk image chunks over a gRPC stream
Parses and validates the VMFile YAML structure
Computes the SHA-256 digest of the disk image
Stores the disk image blob in content-addressable storage
Creates an image configuration blob from the VMFile spec
Generates a manifest referencing both blobs
Stores the manifest under the specified repository and tags

VMCompose Orchestration

When a vmr compose up command is received, the daemon:

Parses and validates the VMCompose YAML
Creates any networks defined in the networks section that don't already exist
Resolves the service dependency graph using topological sorting
For each service in dependency order:
- Resolves the image reference to a local manifest
- Applies resource overrides (CPU, memory, disk size)
- Loads cloud-init configuration (inline or from files)
- Creates and starts a libvirt domain with the appropriate network attachments
Returns the status of all services

Dependencies

Dependency	Purpose
`github.com/spf13/cobra`	CLI framework for flag parsing and command structure
`github.com/coreos/go-systemd/v22`	systemd socket activation support
`google.golang.org/grpc`	gRPC server framework
`google.golang.org/protobuf`	Protocol Buffer runtime
`gopkg.in/yaml.v3`	YAML parsing for VMFile and VMCompose
`libvirt.org/go/libvirt`	Libvirt C API bindings
`libvirt.org/go/libvirtxml`	Libvirt XML domain/network generation
`github.com/aws/aws-sdk-go-v2`	S3 storage backend

NixOS Module

The daemon's flake.nix exports a NixOS module for declarative system configuration. It manages the vm-registry user, group, systemd socket activation, and service dependencies.

Options

Option	Type	Default	Description
`services.vm-registry-daemon.enable`	boolean	`false`	Enable the VM Registry daemon service
`services.vm-registry-daemon.package`	package	`self.packages.<system>.default`	The daemon package to use
`services.vm-registry-daemon.dataDir`	string	`/var/lib/vm-registry`	Data directory for the daemon
`services.vm-registry-daemon.socketPath`	string	`/run/vm-registry-daemon.sock`	Unix socket path for the daemon

Usage

nix

{
  inputs.vm-registry.url = "github:VM-Registry/vm-registry";

  outputs = { vm-registry, nixpkgs, ... }: {
    nixosConfigurations.myhost = nixpkgs.lib.nixosSystem {
      modules = [
        vm-registry.nixosModules.default
        {
          services.vm-registry-daemon = {
            enable = true;
            dataDir = "/var/lib/vm-registry";
            socketPath = "/run/vm-registry-daemon.sock";
          };
        }
      ];
    };
  };
}

The module automatically:

Creates a vm-registry system user and group
Adds the user to libvirtd and kvm groups
Configures systemd socket activation with 0660 root:vm-registry permissions
Sets up the service to require libvirtd.service and the socket
Applies tmpfiles rules for the data directory

Docker Image

The flake builds a Docker image at ghcr.io/vm-registry/daemon:latest containing the daemon binary, QEMU/KVM, and cdrkit (for ISO generation). Build it with:

bash

nix build .#docker-image --impure

Daemon ​

Overview ​

Running the Daemon ​

Configuration Flags ​

systemd Socket Activation ​

Examples ​

Internal Structure ​

gRPC Service ​

Libvirt Integration ​

Domain Management ​

Network Management ​

Cloud-Init ​

Disk Management ​

Image Storage ​

VMFile Processing ​

VMCompose Orchestration ​

Dependencies ​

NixOS Module ​

Options ​

Usage ​

Docker Image ​

Daemon

Overview

Running the Daemon

Configuration Flags

systemd Socket Activation

Examples

Internal Structure

gRPC Service

Libvirt Integration

Domain Management

Network Management

Cloud-Init

Disk Management

Image Storage

VMFile Processing

VMCompose Orchestration

Dependencies

NixOS Module

Options

Usage

Docker Image