---

name: offline-ai-toolkit description: >- Build offline-capable AI systems with local models, embedded knowledge bases, and no internet dependency. Use when: building AI tools for offline use, creating self-contained knowledge systems, deploying AI in air-gapped environments. license: MIT compatibility: "Node.js 18+ or Python 3.10+, Ollama" metadata: author: terminal-skills version: "1.1.0" repository: https://github.com/ollama/ollama category: development tags: [offline, local-ai, ollama, knowledge-base, edge-ai] use-cases: - "Build a self-contained AI assistant that works without internet" - "Create an offline knowledge base with local LLM for field work" - "Deploy AI tools in air-gapped or low-connectivity environments" agents: [claude-code, openai-codex, gemini-cli, cursor]

Offline AI Toolkit — Self-Contained AI Systems

Overview

Build AI systems that work completely offline — local LLMs via Ollama, embedded vector search with SQLite, knowledge bases from pre-downloaded content, and a PWA interface that runs without connectivity. Ideal for field work, air-gapped environments, or privacy-first deployments.

Instructions

Step 1: Install Ollama for Local LLMs

brew install ollama            # macOS (Linux: tarball below; Windows: winget install Ollama.Ollama)
ollama serve &                 # skip if the desktop app or systemd service already runs it
ollama pull llama3.1:8b        # General purpose (about 5GB)
ollama pull nomic-embed-text   # Embeddings (274MB, 2K context)
ollama list                    # confirm both are on disk

On Linux, download ollama-linux-amd64.tar.zst (or the arm64 build) from the Ollama GitHub releases page, check it against the sha256sum.txt published there, then extract it under /usr. For an air-gapped target, do the downloads and ollama pull on a connected machine and copy the ~/.ollama/models directory (or the whole Docker volume of the ollama/ollama image) to the offline machine. Set OLLAMA_HOST=127.0.0.1:11434 so the API is not exposed on the network.

Use Case	Model	RAM Needed
General Q&A	llama3.1:8b	8GB
Quick answers	phi3:mini	4GB
Code help	qwen2.5-coder:7b	8GB
Embeddings	nomic-embed-text	2GB

Step 2: Build the Offline Knowledge Base

import os
import sqlite3
from urllib.parse import quote

import requests

HEADERS = {'User-Agent': 'offline-kb-builder (https://terminalskills.io)'}  # Wikipedia requires one

def init_knowledge_db(db_path='knowledge.db'):
    """Initialize SQLite database for knowledge storage."""
    conn = sqlite3.connect(db_path)
    conn.execute('''CREATE TABLE IF NOT EXISTS documents (
        id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT,
        source TEXT, content TEXT, category TEXT,
        created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
    )''')
    conn.execute('''CREATE TABLE IF NOT EXISTS embeddings (
        id INTEGER PRIMARY KEY, doc_id INTEGER REFERENCES documents(id),
        chunk_text TEXT, embedding BLOB, chunk_index INTEGER
    )''')
    # External-content FTS index; run the 'rebuild' command after loading documents
    conn.execute('''CREATE VIRTUAL TABLE IF NOT EXISTS fts_documents
        USING fts5(title, content, category, content='documents', content_rowid='id')''')
    return conn

def download_wikipedia_articles(topics, conn):
    """Download Wikipedia articles for offline knowledge."""
    for topic in topics:
        url = f"https://en.wikipedia.org/api/rest_v1/page/summary/{quote(topic)}"
        try:
            r = requests.get(url, headers=HEADERS, timeout=10)
            r.raise_for_status()
            data = r.json()
            content = data.get('extract', '')
            if content:
                conn.execute(
                    'INSERT INTO documents (title, source, content, category) VALUES (?, ?, ?, ?)',
                    (data.get('title', topic), f'wikipedia:{topic}', content, 'encyclopedia'))
        except Exception as e:
            print(f"Failed to download {topic}: {e}")
    conn.commit()
    conn.execute("INSERT INTO fts_documents(fts_documents) VALUES ('rebuild')")
    conn.commit()

Wikipedia summaries are only a paragraph each. For full articles bundle a Kiwix ZIM file and serve it with kiwix-serve, or ingest your own Markdown/PDF text.

Step 3: Generate Embeddings with Ollama

import struct

def get_embedding(text, model='nomic-embed-text'):
    """Get embedding vector from Ollama (/api/embed; /api/embeddings is the legacy endpoint)."""
    response = requests.post('http://localhost:11434/api/embed',
                             json={'model': model, 'input': text}, timeout=60)
    response.raise_for_status()
    return response.json()['embeddings'][0]

def embedding_to_blob(embedding):
    return struct.pack(f'{len(embedding)}f', *embedding)

def blob_to_embedding(blob):
    n = len(blob) // 4
    return list(struct.unpack(f'{n}f', blob))

def embed_all_documents(conn, chunk_size=500):
    """Generate embeddings for all documents in the database."""
    cursor = conn.execute('SELECT id, content FROM documents')
    for doc_id, content in cursor.fetchall():
        words = content.split()
        for i in range(0, len(words), chunk_size):
            chunk = ' '.join(words[i:i + chunk_size])
            if len(chunk.strip()) < 20:
                continue
            emb = get_embedding(chunk)
            conn.execute(
                'INSERT INTO embeddings (doc_id, chunk_text, embedding, chunk_index) VALUES (?, ?, ?, ?)',
                (doc_id, chunk, embedding_to_blob(emb), i // chunk_size))
    conn.commit()

Step 4: Offline Vector Search

import math

def cosine_similarity(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    norm_a = math.sqrt(sum(x * x for x in a))
    norm_b = math.sqrt(sum(x * x for x in b))
    return dot / (norm_a * norm_b) if norm_a and norm_b else 0

def search_knowledge(query, conn, top_k=5):
    """Search the knowledge base using vector similarity."""
    query_emb = get_embedding(query)
    cursor = conn.execute('SELECT doc_id, chunk_text, embedding FROM embeddings')
    results = []
    for row in cursor.fetchall():
        sim = cosine_similarity(query_emb, blob_to_embedding(row[2]))
        results.append({'chunk_text': row[1], 'doc_id': row[0], 'similarity': sim})
    results.sort(key=lambda x: x['similarity'], reverse=True)
    return results[:top_k]

Step 5: RAG with Local LLM

def ask_offline(question, conn, model='llama3.1:8b'):
    """Answer questions using local RAG pipeline."""
    results = search_knowledge(question, conn, top_k=3)
    context = '\n\n'.join([r['chunk_text'] for r in results])
    response = requests.post('http://localhost:11434/api/generate', json={
        'model': model,
        'prompt': f"""Answer using ONLY the context provided.
If the context doesn't contain the answer, say "I don't have information about that."

Context:
{context}

Question: {question}
Answer:""",
        'stream': False
    }, timeout=300)
    return {
        'answer': response.json()['response'],
        'sources': [r['chunk_text'][:100] for r in results],
        'model': model
    }

Examples

Example 1: Build an Offline Field Research Assistant

A wildlife researcher prepares an offline AI assistant before a 2-week trip to a remote area with no connectivity:

# While online: download knowledge and build embeddings
conn = init_knowledge_db('field_research.db')

# Load species identification guides and park documentation
download_wikipedia_articles([
    'Grizzly_bear', 'Gray_wolf', 'Elk', 'Moose',
    'Yellowstone_National_Park', 'Wildlife_tracking',
    'Bear_safety', 'GPS_navigation'
], conn)

# Ingest local field manuals (markdown files on laptop)
for root, _, files in os.walk('./field-manuals'):
    for f in files:
        if f.endswith('.md'):
            with open(os.path.join(root, f)) as fh:
                conn.execute('INSERT INTO documents (title, source, content, category) VALUES (?,?,?,?)',
                             (f, os.path.join(root, f), fh.read(), 'field-manual'))
conn.commit()
embed_all_documents(conn)

# In the field (fully offline):
result = ask_offline("What are the signs of a nearby grizzly bear den?", conn)
# Answer: "Look for excavated hillside entrances, claw marks on nearby trees,
#  matted vegetation, and a strong musky odor. Dens are typically on north-facing
#  slopes at elevations above 6,000 feet..."

Example 2: Air-Gapped Developer Documentation Server

A defense contractor sets up an offline coding assistant for a secure facility with no internet:

conn = init_knowledge_db('dev_docs.db')

# Pre-load language and framework documentation
import os
for doc_dir in ['./docs/python-stdlib', './docs/react-docs', './docs/kubernetes']:
    for root, _, files in os.walk(doc_dir):
        for f in files:
            if f.endswith(('.md', '.txt', '.rst')):
                path = os.path.join(root, f)
                with open(path, 'r', errors='ignore') as fh:
                    conn.execute('INSERT INTO documents (title,source,content,category) VALUES (?,?,?,?)',
                                 (f, path, fh.read(), 'dev-docs'))
conn.commit()
embed_all_documents(conn)

# Developer queries the system (no internet needed):
result = ask_offline("How do I create a Kubernetes CronJob that runs every 6 hours?", conn)
# Answer: "Create a CronJob manifest with schedule '0 */6 * * *' and specify
#  your container image in the jobTemplate spec. Set restartPolicy to OnFailure..."
print(result['sources'])  # Shows which doc chunks were used as context

Guidelines

• Download everything while online — models, knowledge content, and embeddings must be prepared beforehand
• Test offline before deploying — disconnect WiFi and verify the full pipeline works end-to-end
• Choose models by hardware — phi3:mini for 4GB RAM devices, llama3.1:8b for 8GB+, a 70B model only on workstations with 48GB+ memory
• Use FTS as fallback — SELECT title FROM fts_documents WHERE fts_documents MATCH 'grizzly den' ORDER BY rank LIMIT 5 works when embeddings are unavailable or for exact matches; rebuild the index after inserting documents
• Scale — the brute-force cosine loop is fine up to roughly 50k chunks; beyond that use the sqlite-vec extension
• Keep the same embedding model — changing models means re-embedding everything
• Keep Ollama local — never expose port 11434 beyond localhost
• Package for portability — bundle everything on a USB drive or Docker image for easy deployment
• Keep knowledge fresh — sync new content and re-embed when connectivity returns

References

• Ollama — local LLM runtime
• SQLite FTS5 — full-text search
• PWA docs — offline-first web apps