model-hoster

…

Add a machine

Machines waiting to join

Each of these generated its own credential and is asking to be let in. Compare the fingerprint with the one the machine is showing — in its console, on its own page, or in the Mac menu bar — then give it a name. Nothing needs copying back.

Fingerprint	Machine	Hardware	Waiting since

Catalog

Model	Role	Residency	Sizing	Use 24h

Add a model from Hugging Face

A model can be several repositories. Look up the main one here — the diffusion build for a video model — and add its other components below. A single repository that holds everything in subfolders needs none of that: it is one component already.

Other components — a text encoder, a VAE, an audio model

Every component is fetched and needed, and each states its own VRAM: an assembled model is sized by adding them up, so a missing figure makes the total unknown rather than smaller. The name is how a render mode's pipeline finds the component, so pick it from the list — those are the names the shipped modes actually read, and a spelling of your own is a part no pipeline will look up.

Name	Repository	VRAM

Reset the hand-authored catalog

Empties catalog.yaml, keeping a timestamped copy beside it. Models added here live in models.d/ and are untouched. Use it when a config volume outlived the image that filled it and is still serving entries nobody asked for.

Render modes

A job picks its mode by the name it asked for, so a mode is named after a catalog entry, one of its aliases, or a personality. Several modes over one entry is the point: five names, five pipelines, one set of weights loaded once.

Name	A client can ask for it	What it takes

Modes are deployed with this server — a mode is {name}.pipeline.json plus its {name}.inputs.json declaration in config/workflows/ — so adding or changing one is a change to the repository, reviewed like any other. There is no graph and nothing to bind: the runner loads only the components this pool fetched for the model.

In flight

Model	Waiting	Serving	Longest wait

Usage by key 24h

Key	Requests	Prompt	Completion	Mean	Last used

Memory

Recent

Device	Size	Committed	In use	Free	Load	Recent

Nodes

Node	Platform	Engines	Host RAM	Processor	Paging	Cached	Held back

Resident

Model	Node	Engine	State	Reserved	Loaded as	Leases

New personality

The name is what clients ask for and it is the half that should stay still. Change the model behind it whenever you like — nothing that uses the name needs reconfiguring. Saving an existing name replaces it.

Personalities

Name	Answers with	System prompt

Run a benchmark case

A case is a prompt and the frame it starts from, shipped with this server so a figure measured here means the same as one measured elsewhere. Running one submits an ordinary render — the same path, the same checks — and records what the machine did while it ran.

Every render

Every render this pool has run, newest first — a benchmark, one submitted from a script, or one a client asked for — and a render still going appears here while it runs, with its figures filled in when it ends. Benchmarks only asks a narrower question rather than filtering this one: the full list is bounded by recency, so a busy week of client renders would push a month-old benchmark off it.

Case	Kind	Model	Requested by	Node	State	Render	Peak VRAM	Shape	Peak RAM	CPU	GPU	Action

What is on the disks

A store is a name the owning side resolves to a directory of its own. Rows say what a thing is — which repository and revision, whether an engine is reading it, and which catalog entries still point at it. Nothing pointing at it is the answer worth looking for: those weights will never be loaded again.

Name	Size	What it is	Named by	Modified

Bring a build in

Files land on this server first and are then handed to nodes on command — agents never fetch on their own. What arrives on a node is an ordinary cached revision, so a catalog entry points at it with the same repository and revision as anything else. Upload the config.json beside a single-file build: the render runner refuses weights with no architecture stated next to them, so the upload would otherwise succeed and the first load would fail.

Waiting to be handed over

Held as	Files	Size	Uploaded

LoRAs

A render mode's LoRA is declared in its pipeline.json and fetched to a machine the first time that machine renders the mode. Renders counts only renders whose runner said it applied the LoRA. A LoRA's disk is freed on the Storage tab, like any other weights.

LoRA	Kind	Pinned to	Used by	Downloaded on	Renders

Keys

Label	Scope	Models	Limit	Created	Last used	Expires

Copy this now — it is not stored and cannot be shown again.

Issue a key

Nodes need nothing from here. A machine generates its own credential, appears above as waiting to join, and is let in by being approved — so removing one machine means rejecting it under Machines, not revoking a key. Pick a lifetime to mint a key that dies on its own, or leave it at "never" for a permanent one. "Custom…" takes an ISO 8601 duration such as PT1H.

Engine arguments

Context

Parallel

GPU layers

Speculation

MTP

also draft with this model's own MTP block

Only for weights that carry one — this pool's 27B coding entry does, and loads it unused today. Measured there at about 2.4x for roughly 950 MiB of draft context, which grows with the context length. It changes what the model writes: unlike the n-gram speculators, its output is not identical to not speculating, so it is a per-model decision rather than a speed setting. A model with no MTP block simply runs without it.

KV cache

A list tries each type in turn and each machine takes the first that fits it — a Mac with room keeps f16, a card where f16 does not fit takes q8_0 — with nothing set per machine. q8_0 answered like f16 on this pool's tests and found text 120k tokens back; both quantized types decode at about half speed once the context is long, so f16 stays first where it fits.

Declared size

Blank sizes the model from its weights, context and KV cache type. A figure here is used as it is for every machine and every cache type, so it has to be blank for a list of cache types to be chosen.

Projector

A vision model needs its projector named here, or the engine starts and refuses every image. Use the filename as the repository publishes it; it is added to the files this model fetches if it was not already among them.

…

Weights, by component

Component	Builds

Group another repository into this model

A component is fetched and needed, unlike a build, which is one of several ways to serve the same component. Every build states its own size, because an assembled model is sized by adding its parts up and a missing figure makes the total unknown rather than smaller.

Serving

What renders have cost

Shape	Renders	Peak	Weights	Work	Typical

How fast each machine renders it

What placement's speed term has learnt from this model's own renders, per unit of work. A node that has never rendered it is absent — its speed is the accelerator profile's, not a measured zero.

Node	Speed	Renders

Residency

Keep loaded never evicted, and loaded without waiting to be asked

On machine only meaningful when kept loaded; one machine going away then means not resident

Runs beside must load on the same machine as this model, which has to be resident already. Keep that one loaded, or this one has nowhere to go

Priority higher survives pressure longer; the lowest is evicted first

Idle timeout seconds idle before it is unloaded on its own. 0 never unloads it

Names

Aliases other names a client may ask for, comma separated. A whole list, so removing one means deleting it here

Weights, repository, and removing this model

Files one pattern per line — what the inline repository fetches. Trimming makes cached copies incomplete, and the next load pulls only what is missing; * means everything