System 1 Server

Repository: github.com/andrewcampi/system1_server. License: Apache-2.0.

What it does

System 1 Server is a local decision server. It loads Laya multilingual once and answers Jev-shaped questions: choice, score, and noul. Each answer includes a probability. It does not generate text.

The client is the official OpenAI Python package. Point OPENAI_BASE_URL at the process and the same code can make a live call, submit a batch, or do both. Requests stay on the machine that is running the model.

Calling it

The process speaks the OpenAI HTTP shape well enough that the stock client can target it. A bearer token is required. The example environment file in the repo documents the local defaults, including the bind address and a development token. Treat those defaults as local-only. The published example binds 0.0.0.0:8000 so a machine you control can reach it. Do not expose that development token on a public network.

Live calls are admitted up to MAX_CONCURRENT_REQUESTS (default 4). Further live calls wait in order. Uploaded files and batch records go in DATA_DIR.

The runtime uses CUDA when it is present, then Apple MPS, then CPU. Each microbatch is sized from 92% of free GPU memory, or 92% of available RAM when there is no GPU. The only model name the process serves is laya-multilingual.

Throughput

The project README reports a measurement on an RTX 5080. A warm live call through the OpenAI client returned in 9.5 ms. A batch of 10,000 different requests, 23,333 decisions, finished in 13.2 seconds: 1,773 decisions per second.

Running it

From a checkout of the repository:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python main.py

The first start downloads the multilingual checkpoint into models/. main.py reads .env, then example.env for anything still unset. A separate client_example/ directory shows a live call and a batch through the OpenAI client.