CLI reference

Ollaya is a single binary: the CLI, the server and the model runners. If you have used Ollama, the commands will feel familiar.

Every command except serve talks to the server at OLLAYA_HOST. When nothing answers there and the address is local, the CLI (except ollaya stop without a model) starts ollaya serve in the background, logging to ~/.ollaya/logs/server.log.

Model names

Models are referenced as name:tag; without a tag, latest is used. Names are case-insensitive.

ollaya run laya                 # same as laya:latest, a router
ollaya run laya:multilingual
ollaya pull laya:en-fp32

A model such as laya:en carries an fp16 and an fp32 graph that share one weights file. The precision is picked when the model loads: fp16 on a CUDA GPU, fp32 on the CPU. The -fp16 and -fp32 tags pin one.

ollaya run

ollaya run laya --preset triage "I was charged twice for my subscription this month and want a refund."

run connects to the server (starting it if needed), pulls the model if it is not on this machine, loads it and prints one row per question: the answer, a bar and its probability.

Where the questions come from, first match wins: --questions, then --preset, then questions built into the model with a Modelfile. A model with none needs one of the flags.

The state is the rest of the command line. Without one, run reads piped stdin:

cat ticket.txt | ollaya run laya --preset triage

On a terminal without a state, run opens a prompt. Type a state and press Enter; wrap several lines in """. Commands:

ollaya serve

Starts the server that the CLI and your applications talk to. It serves the native API (/api/*) and the TypeSafe-compatible API (/v1/*); see the API reference. The Linux installer runs it as the ollaya systemd service.

ollaya serve

You rarely need it: the other commands start the server in the background when it is not running. If a server already runs at OLLAYA_HOST, ollaya serve says so and exits; stop that one with ollaya stop first to run the server in the foreground.

It is configured with environment variables:

For the systemd service, change them with sudo systemctl edit ollaya and Environment= lines.

ollaya pull

Downloads a model and verifies every layer against its sha256. Pulling a router also pulls every model it routes to. Only the layers this machine needs are downloaded, and interrupted downloads resume.

ollaya pull laya

ollaya list and ollaya ps

list shows the models on this machine with their ID, size and when they were pulled. ps shows the loaded models, the device they run on (cpu, cuda:0), the precision and when they will be unloaded.

NAME                ID             SIZE     MODIFIED
laya:latest         b87ca1631b11   11 KB    13 seconds ago
laya:multilingual   ba7a334675b4   684 MB   13 seconds ago
laya:en             bf30e4654e94   854 MB   30 seconds ago

ollaya show

Prints a model's architecture, parameters, context length, precisions, languages, capabilities and license. For a router, it prints the routes.

ollaya stop

ollaya stop MODEL unloads a running model once its in-flight requests finish, instead of waiting for the keep-alive to run out.

ollaya stop without a model stops the server at OLLAYA_HOST, which unloads every model. It stops only a server that you started, with ollaya serve or by running another command. It never stops another user's server, such as the Linux systemd service; stop that with sudo systemctl stop ollaya.

ollaya rm and ollaya cp

ollaya cp laya:en my-guardrail
ollaya rm my-guardrail

rm also deletes the blobs no other model uses. Removing a router keeps the models it routes to. cp overwrites an existing destination.

ollaya create

Builds a model from a Modelfile, for example to bake in a question set, a refit calibration or a pinned precision. -f defaults to ./Modelfile.

ollaya create triage -f Modelfile