Architecture

VM Registry follows a modular, multi-process architecture where each component has a well-defined responsibility and communicates through standard protocols. The system is split across two languages — Go for infrastructure and server components, Rust for user-facing tooling — unified by Protocol Buffers for type-safe communication.

System Overview

                         ┌──────────────────────────────────────┐
                         │            User Workstation           │
                         │                                      │
                         │  ┌──────────┐       ┌─────────────┐  │
                         │  │ CLI      │       │ Editor +    │  │
                         │  │ (Rust)   │       │ LSP (Rust)  │  │
                         │  └────┬─────┘       └─────────────┘  │
                         │       │ gRPC over Unix Socket         │
                         │  ┌────▼──────────────────────────┐   │
                         │  │       Daemon (Go)              │   │
                         │  │  ┌──────────┐ ┌────────────┐  │   │
                         │  │  │ libvirt  │ │ Image      │  │   │
                         │  │  │ Manager  │ │ Storage    │  │   │
                         │  │  └──────────┘ └────────────┘  │   │
                         │  └────────────┬──────────────────┘   │
                         └───────────────┼──────────────────────┘
                                         │ HTTP / HTTPS
                         ┌───────────────┼──────────────────────┐
                         │               │   Remote Servers      │
                         │  ┌────────────▼─────────────────┐    │
                         │  │   Registry Server (Go)       │    │
                         │  │   OCI-compatible HTTP API     │    │
                         │  │   ┌────────┐  ┌───────────┐  │    │
                         │  │   │ FS     │  │ S3        │  │    │
                         │  │   │ Store  │  │ Store     │  │    │
                         │  │   └────────┘  └───────────┘  │    │
                         │  └────────────┬─────────────────┘    │
                         │               │ JWT Verification      │
                         │  ┌────────────▼─────────────────┐    │
                         │  │   Auth Server (Go)           │    │
                         │  │   ┌──────────────────────┐   │    │
                         │  │   │     PostgreSQL        │   │    │
                         │  │   └──────────────────────┘   │    │
                         │  └──────────────────────────────┘    │
                         └──────────────────────────────────────┘

Component Responsibilities

CLI Rust

The CLI (vm-registry-cli) is the primary user interface. It is a statically compiled Rust binary built with clap for argument parsing and tonic for gRPC communication. The CLI does not manage VMs directly — every operation is delegated to the daemon over a Unix domain socket.

Key responsibilities:

• Parsing user commands and translating them into gRPC requests
• Managing local authentication state (storing JWT tokens on disk)
• Context management for switching between daemon instances and registries
• Streaming large image files to the daemon during import via chunked gRPC streams

Daemon Go

The daemon (vm-registry-daemon) is the central orchestrator running on each host. It exposes a gRPC service over a Unix socket and manages all local VM operations.

Key responsibilities:

• Image management — Importing, storing, tagging, and garbage-collecting VM images in a content-addressable store (filesystem or S3)
• VM lifecycle — Creating, starting, stopping, restarting, and deleting libvirt domains
• Networking — Creating and managing libvirt virtual networks (NAT, isolated, routed, bridged, open, macvtap)
• VMFile parsing — Validating and processing VMFile manifests that describe VM images
• VMCompose orchestration — Parsing VMCompose files and coordinating multi-VM deployments with dependency ordering
• Cloud-init — Generating cloud-init ISO images for guest provisioning
• Registry interaction — Pulling images from and pushing images to the remote registry server
• Authentication proxying — Forwarding login, logout, and registration requests to the auth server

The daemon uses the libvirt.org/go/libvirt and libvirt.org/go/libvirtxml bindings for direct interaction with the libvirt hypervisor API.

Registry Server Go

The registry server (vm-registry-server) stores and serves VM images over an OCI-inspired HTTP API. It is intentionally simple — a stateless HTTP server that delegates all persistence to the configured storage backend.

Key responsibilities:

• Blob storage — Upload and download content-addressed blobs (disk images, config layers)
• Manifest management — Store and retrieve image manifests that reference blobs by digest
• Storage backends — Pluggable storage with filesystem and S3-compatible implementations
• Authentication enforcement — Validates JWT tokens issued by the auth server using the shared RSA public key

API versioning follows the /v1 prefix. The registry supports standard blob operations (HEAD, GET, PUT, PATCH, DELETE) and manifest operations.

Auth Server Go

The auth server (vm-registry-auth) handles user identity and access control. It is deployed alongside PostgreSQL and uses RSA key pairs for JWT signing.

Key responsibilities:

• User registration — Creating accounts with bcrypt-hashed passwords
• Token issuance — Generating JWT access tokens and refresh tokens on login
• Token endpoint — Issuing scoped tokens for registry operations (follows the Docker token auth spec pattern)
• ACL enforcement — Repository-level access control with pattern matching (e.g., user/*, *)
• Token lifecycle — Background cleanup of expired and revoked refresh tokens

Database schema:

Table	Purpose
users	Usernames and bcrypt password hashes
acl_rules	Per-user repository patterns and allowed actions (pull, push, *)
refresh_tokens	Refresh token tracking with expiry and revocation state

Proto Protobuf

The proto repository (vm-registry-proto) contains the shared Protocol Buffer definitions consumed by both the Go daemon and the Rust CLI. The single VMService gRPC service defines all RPCs grouped into domains:

Domain	RPCs	Description
Authentication	Login, Logout, Register, HealthCheck	User auth proxied through the daemon
Images	PullImage, PushImage, ImportImage, ImportImageStream, ExportImage, ListLocalImages, DeleteImage, GetImageManifest, ValidateVMFile, TagImage, GarbageCollect	Full image lifecycle
Contexts	ContextCreate, ContextUpdate, ContextUse, ContextList, ContextRemove, ContextCurrent	Multi-environment switching
Actions	RunVM, StartVM, StopVM, RestartVM, DeleteVM, PsVMs, ListVMs	VM lifecycle operations
Networks	NetworkList, NetworkGet, NetworkCreate, NetworkActivate, NetworkDeactivate, NetworkDelete	Libvirt network management
Compose	ComposeUp, ComposeDown, ComposePs, ComposeRestart, ComposeValidate	Multi-VM orchestration

Message definitions are split across focused files in defs/: action.proto, auth.proto, compose.proto, context.proto, image.proto, and network.proto.

LSP Rust

The language server (vm-registry-lsp) provides editor intelligence for VMFile and VMCompose YAML files. It implements the Language Server Protocol using the lsp-server and lsp-types crates.

Key capabilities:

• Diagnostics — Real-time validation of document structure, required fields, value types, enum constraints, memory format patterns, IP/CIDR notation, and cross-references (e.g., service network references matching defined networks)
• Completions — Context-aware field name and value completions with snippet support
• Hover — Documentation tooltips showing field descriptions, valid values, and format requirements
• Schema-driven — All validation and intelligence is derived from a centralized schema definition that mirrors the daemon's parser expectations

Communication Flow

Image Import

1. User runs vmr import ./VMFile -t latest
2. CLI parses the VMFile locally and opens a gRPC stream to the daemon (ImportImageStream)
3. CLI sends metadata (VMFile content, tags, options) as the first stream message
4. CLI reads the disk image file in chunks and streams each chunk to the daemon
5. Daemon reassembles the image, computes content digests, and stores blobs in the local image store
6. Daemon creates a manifest referencing the image config and disk blob
7. Daemon responds with the repository name, digest, and assigned tags

Image Push/Pull

1. Push: Daemon reads blobs from local storage, uploads them to the registry server via HTTP PUT, then uploads the manifest
2. Pull: Daemon fetches the manifest from the registry, then downloads each referenced blob and stores them locally

Both operations require a valid JWT token obtained through vmr login.

VM Run

1. User runs vmr run user/image:latest --vm-name my-vm
2. CLI sends a RunVM gRPC request to the daemon
3. Daemon resolves the image reference to a local manifest
4. Daemon creates a libvirt domain XML from the image spec (CPU, memory, bootloader, disk)
5. If cloud-init config is provided, daemon generates an ISO and attaches it
6. If a network is specified, daemon attaches the VM to the libvirt network
7. Daemon defines and starts the libvirt domain
8. Daemon responds with the VM ID and status

Compose Up

1. User runs vmr compose up -f cluster.VMCompose
2. CLI reads the compose file and sends the full YAML content to the daemon (ComposeUp)
3. Daemon parses and validates the VMCompose structure
4. Daemon creates any defined networks that don't already exist
5. Daemon resolves service dependency ordering
6. Daemon starts each service's VM in dependency order, applying resource overrides, network attachments, and cloud-init configuration
7. Daemon responds with the status of each service

Storage Architecture

Local Image Store

The daemon stores images in a content-addressable layout under its configured storage path:

/var/lib/vm-registry/
├── manifests/
│   └── <repository>/
│       └── <tag>/
│           └── manifest.json
├── blobs/
│   └── sha256/
│       └── <digest>          # disk images and config blobs
└── runtime/
    └── <vm-name>/            # running VM state (cloud-init ISOs, etc.)

Registry Storage

The registry server supports two backends:

• Filesystem — Direct file I/O to a local directory. Suitable for single-node deployments.
• S3 — Any S3-compatible object store (AWS S3, MinIO, RustFS). Suitable for distributed deployments.

Both backends implement the same Storage interface, making them interchangeable through configuration.

Security Model

Authentication flows through three layers:

1. User credentials — Bcrypt-hashed passwords stored in PostgreSQL
2. JWT tokens — RSA-signed access tokens with scoped claims (repository pattern + actions)
3. Registry middleware — The registry server validates JWT signatures using the auth server's public key and enforces scope requirements per API endpoint

The auth server and registry server share only the RSA public key — the private key never leaves the auth server. Tokens are short-lived, and refresh tokens support revocation.