The node
The node is the program that takes jobs from the queue and runs them. It is a TypeScript worker that ships with the backend. Everything about how it runs is set by environment variables, so the same worker runs on a laptop or a server with nothing changed but its settings.
| Variable | What it sets | Default |
|---|---|---|
AI_NODE_BACKEND_URL |
The backend to ask for jobs | |
AI_NODE_API_TOKEN |
The worker token | |
AI_NODE_OLLAMA_URL |
The local model server | |
AI_NODE_MODEL |
The model | qwen2.5:7b-instruct |
AI_NODE_POLL_MS |
How often to ask for batch jobs | 2000 |
AI_NODE_INTERACTIVE_POLL_MS |
How often to ask for interactive jobs | 1000 |
AI_NODE_BATCH |
Jobs taken per claim | 1 |
AI_NODE_LEASE_MS |
The lease it asks for | 30000 |
AI_NODE_WORKER_ID |
The name it gives itself in claims | |
AI_NODE_WEBHOOK_PORT |
The port its webhook receiver listens on | |
AI_NODE_WEBHOOK_SECRET |
The Standard Webhooks secret, starting whsec_ |
How it picks up a job
The node asks for jobs in two loops. A batch loop asks at the batch interval. An interactive loop asks at the interactive interval, and asks only for jobs someone is waiting on, those at priority 100 or above. Once it holds a job, the node extends the lease every third of the lease's length, so a job it is still working on is never given to someone else.
The node also listens for a push. Its webhook receiver checks each delivery it gets, using the Standard Webhooks signature, allowing a 300-second difference in clocks and refusing a body over 64 KiB. When the delivery says a job was enqueued, the node wakes its interactive loop at once, so an interactive job is picked up without waiting for the next poll.
Running a job
For a categorize_and_tag job, the node sends the document to the model server's chat completions endpoint with a JSON schema that says what shape the answer must take. The model answers with a category, tags and a confidence. When the node reports the job complete, it also reports what the model used, so the job's cost is metered like any other model call.
- Languages
- TypeScript