> ## Content Index > Fetch the complete content index at: https://www.notatechguy.com/llms.txt > Use this file to discover other available public pages before exploring further. # CourseChat AI tutor picks 8B model over bigger rivals - URL: https://www.notatechguy.com/coursechat-ai-tutor-picks-8b-model-over-bigger-rivals/ - Published: 2026-10-05T15:27:29.000Z - Updated: 2026-10-05T15:27:29.000Z - Description: Lethbridge researchers built CourseChat, an on-premises RAG tutor for six business courses, where bigger AI models failed the classroom speed test. - Author: Marcello Babbili - Tags: Technology & AI, AI Agents, AI Models Sidney Shapiro and Joshua Lindemann at the University of Lethbridge built CourseChat, an on-premises AI tutor serving six business courses from two edge machines [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com)[P²](https://arxiv.org/html/2610.02510?ref=notatechguy.com). They kept an 8-billion-parameter language model in production after larger rivals failed a classroom speed test [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The finding cuts against the grain of current AI practice, where the biggest available model usually wins. A 12B model and a 7B alternative passed the speed gate, and a mixture-of-experts candidate fixed some errors while introducing new factual and continuity mistakes [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). **My read:** This is the first campus-deployed RAG tutor I have seen that treats model choice as a hardware constraint rather than a quality dial. The authors are honest about what they do not know: they explicitly disclaim learning gains, and they flag faculty ratings and peak-load capacity as open questions. The 8B retention is provisional, not a verdict. What makes this worth reading is the engineering discipline. They ran two bake-off rounds, a separate fidelity audit, and conversation and quiz audits, then still said the alternatives were not good enough to switch. That kind of restraint is rare in AI education deployments, where the default move is to chase the newest model. ## On-premises RAG with prebuilt questions CourseChat is a RAG tutor, a system that pulls relevant passages from course materials and feeds them to a language model to generate answers grounded in those texts [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). It runs entirely on-premises, behind a campus web gateway, with no external API calls [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The authors intend it to be embedded in Moodle, the learning management system many universities already use [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). Six isolated course offerings, each identified by its own course reference number, share two edge AI hosts [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). Those hosts run a FastAPI web service, a local vector database for finding relevant course passages, and a local large language model served through Ollama, an open-source tool for running models on hardware the institution owns [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The system includes 435 prebuilt questions across 65 modules, which decouple student practice from live generation [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). A student working through a quiz gets a fixed, pre-validated question rather than a freshly generated one, reducing the risk of a model fabricating a problem on the fly. ## Larger models failed the speed test The authors ran two rounds of generation-model comparisons, a separate fixed-evidence source-fidelity test, and conversation and quiz audits [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). Several larger models failed the classroom speed gate, the latency threshold below which a tutor feels responsive enough for live use [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). A 12B model and a 7B alternative passed. A mixture-of-experts candidate, an architecture that activates only a subset of its parameters per query to save compute, improved some corrections while introducing new factual and continuity errors [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The authors retained the 8B production model, pending a demonstrated overall improvement [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). They do not claim the 8B model is universally optimal. Their argument is that model choice, evidence selection, serving compatibility, and product design should be treated as a single engineering decision, not four separate ones [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). This matters because the dominant pattern in AI tutoring is to call a cloud API running the largest model available. CourseChat inverts that. The constraint is not model quality in isolation, but model quality at classroom speed, on hardware the university owns, with no data leaving campus. The paper is an arXiv preprint and has not undergone peer review [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The authors explicitly state that their results do not establish learning gains [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). Faculty ratings, peak-load capacity, and complete public-gateway acceptance remain separate evaluation needs [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). The self-reported improvements to follow-up topic resolution lack independent validation [S¹](https://arxiv.org/abs/2610.02510?ref=notatechguy.com). Note: the latency measurements and model comparison outcomes have not been independently verified — no error bars or confidence intervals are reported, no evaluation code has been released, and the speed-test thresholds are the authors' own. The trade-offs described are specific to this campus deployment and may not generalise to institutions with different course loads or hardware. ## A reference design for campus IT directors A business school IT director evaluating on-premises AI tutoring could use CourseChat's architecture as a reference design: twin edge hosts running Ollama, a FastAPI service layer, and a local vector database, all behind a campus gateway. The 435 prebuilt questions across 65 modules offer a template for decoupling practice content from live generation, which reduces both latency and hallucination risk during quizzes. For that IT director, the desk shifts from managing cloud API keys and per-query billing to provisioning two edge hosts and maintaining a local vector database — the procurement question becomes a one-time hardware spend rather than an ongoing subscription. The next checkpoint for CourseChat is the evaluation the authors themselves flag as missing: faculty ratings and peak-load testing, which would show whether the system holds up when all six courses are active simultaneously. --- *Sources: [S1 — On-Premises Multi-Course RAG Tutoring for Business Education: Hardware](https://arxiv.org/abs/2610.02510?ref=notatechguy.com) · [P2 — On-Premises Multi-Course RAG Tutoring for Business Education:Hardware–](https://arxiv.org/html/2610.02510?ref=notatechguy.com) · [P3 — abinthomas9322/study-rag-tutor](https://github.com/abinthomas9322/study-rag-tutor?ref=notatechguy.com) · [P4 — h-lu/bussiness-data-analysis-agentic-textbook](https://github.com/h-lu/bussiness-data-analysis-agentic-textbook?ref=notatechguy.com) · [P5 — CHIA: An open-source framework for principled, agenticAI-driven hardwa](https://arxiv.org/html/2606.27350v3?ref=notatechguy.com)* --- *Written from 5 sourced items, 4 of them primary.* ![Model size vs classroom speed gate result](https://storage.ghost.io/c/6e/89/6e896869-22ef-4281-a213-b4c462c17cff/content/images/2026/10/chart_5bd588abd086e939d972.png) ## More from Not A Tech Guy - [MCP Python SDK trends on GitHub with 24,000 stars](https://www.notatechguy.com/mcp-python-sdk-trends-on-github-with-24-000-stars/) - [Multimodal AI models merge vision and text via two distinct pathways](https://www.notatechguy.com/multimodal-ai-models-merge-vision-and-text-via-two-distinct-pathways/) - [AI agent skill scanners evaded 97% by Huawei researchers](https://www.notatechguy.com/ai-agent-skill-scanners-evaded-97-by-huawei-researchers/)