# System 1 Server

Repository: [github.com/andrewcampi/system1_server](https://github.com/andrewcampi/system1_server). License: Apache-2.0.

## What it does

System 1 Server is a local decision server. It loads [Laya multilingual](https://huggingface.co/convaiinnovations/laya) once and answers Jev-shaped questions: `choice`, `score`, and `noul`. Each answer includes a probability. It does not generate text.

The client is the official OpenAI Python package. Point `OPENAI_BASE_URL` at the process and the same code can make a live call, submit a batch, or do both. Requests stay on the machine that is running the model.

## Calling it

The process speaks the OpenAI HTTP shape well enough that the stock client can target it. A bearer token is required. The example environment file in the repo documents the local defaults, including the bind address and a development token. Treat those defaults as local-only. The published example binds `0.0.0.0:8000` so a machine you control can reach it. Do not expose that development token on a public network.

Live calls are admitted up to `MAX_CONCURRENT_REQUESTS` (default 4). Further live calls wait in order. Uploaded files and batch records go in `DATA_DIR`.

The runtime uses CUDA when it is present, then Apple MPS, then CPU. Each microbatch is sized from 92% of free GPU memory, or 92% of available RAM when there is no GPU. The only model name the process serves is `laya-multilingual`.

## Throughput

The project README reports a measurement on an RTX 5080. A warm live call through the OpenAI client returned in 9.5 ms. A batch of 10,000 different requests, 23,333 decisions, finished in 13.2 seconds: 1,773 decisions per second.

## Running it

From a checkout of the repository:

```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python main.py
```

The first start downloads the multilingual checkpoint into `models/`. `main.py` reads `.env`, then `example.env` for anything still unset. A separate `client_example/` directory shows a live call and a batch through the OpenAI client.