AI worker (local LLM)

Qwen2.5-1.5B sidecar for staff AI Chat — install, env vars, health checks, and troubleshooting.

This deploy & admin guide covers school ERP implementation for K-12 day schools. Whether you are evaluating cloud school management software or configuring Schoolyi after go-live, you will find practical steps for school software deployment, MySQL school database backup - written for administrators and staff, not developers.

Last updated August 27, 2026

Schoolyi runs a separate systemd service (schoolyi-ai-worker) that loads Qwen2.5-1.5B-Instruct locally. The Next.js app calls it over HTTP on localhost — no cloud LLM API. Help mode retrieves verified FAQs and docs (RAG) then asks the model to summarize with paths.

Architecture

Caddy → schoolyi (Next.js) → schoolyi-ai-worker (127.0.0.1:11435)
Model: Qwen2.5-1.5B-Instruct Q4_K_M (~940MB) under AI_MODEL_PATH
Knowledge: lib/ai/staff-help/ — FAQ bank from /docs, navigation, and curated manual chunks
UI: /ai-tools/chat and floating widget; API: /api/ai/status, /api/ai/chat

Environment variables

Variable	Purpose
AI_WORKER_ENABLED=1	Turn worker on (required for LLM modes)
AI_WORKER_URL	URL Next app calls (default http://127.0.0.1:11435)
AI_WORKER_TOKEN	Shared secret for worker HTTP + /api/ai/*
AI_MODEL_PATH	Full path to GGUF file
AI_CONTEXT_SIZE	Context window (default 4096)
AI_CHAT_HELP_STYLE	llm (default) or faq for verbatim FAQ help
AI_CHAT_ALLOWED_ROLES	Comma list of roles that can use AI Tools

Operations

Health: curl -s http://127.0.0.1:11435/health on the server
Status: systemctl status schoolyi-ai-worker; journalctl -u schoolyi-ai-worker -n 50
Re-download model: npm run ai:model:download (as app user, with .env sourced)
Benchmark: npm run ai:eval (requires running worker)
Retrieval-only check: npm run ai:gap-report (no worker needed)

Symptom	Check	Fix
/api/ai/status not ready	Service and model file	systemctl restart schoolyi-ai-worker; confirm AI_MODEL_PATH exists on disk.
Help works but draft/polish fails	FAQ-only mode	AI_CHAT_HELP_STYLE=faq skips model for help only; draft/polish always need worker.
Worker OOM killed	MemoryMax	Unit caps at 1536M; use Q4_K_M quant only; do not upgrade to 3B on this host.
Wrong help answers	Knowledge not model	Improve /docs troubleshooting tables and manual FAQs; thumbs down logs feedback.

Common questions

Quick answers in plain language.

How do I fix AI chat worker not ready?+

Enable AI_WORKER_ENABLED=1, download the model (npm run ai:model:download), start schoolyi-ai-worker, and confirm GET /api/ai/status shows ready. Local dev: npm run dev starts the worker when enabled in .env.local.