BYOB AI¶

Bring Your Own AI allows enterprises to use their own language models and AI infrastructure with Archivus, maintaining full control over AI processing.

Why BYOB AI?¶

Data Privacy¶

Sensitive documents never leave your infrastructure for AI processing. Process everything locally with no external API calls.

Model Control¶

Use models fine-tuned for your industry, terminology, or compliance requirements.

Cost Optimization¶

High-volume users can reduce costs by running models on their own infrastructure.

Air-Gapped Compliance¶

Required for defense, government, and highly regulated industries where external AI services are prohibited.

Supported AI Backends¶

Local LLM Providers¶

Cloud AI Providers¶

Custom Models¶

Deploy your own fine-tuned models:

• Custom document classification models
• Industry-specific entity extraction
• Domain terminology embeddings
• Compliance-trained summarization

Architecture¶

graph LR
    subgraph "Your Infrastructure"
        M[Your LLM Server]
        E[Your Embedding Server]
    end
    subgraph "Archivus"
        A[Document Processor]
        B[AI Router]
        C[Knowledge Graph]
    end

    A --> B
    B --> M
    B --> E
    M --> C
    E --> C

Processing Flow¶

1. Document Upload - New document enters the system
2. AI Router - Archivus routes AI requests to your configured backend
3. Your Infrastructure - Processing happens on your models
4. Results Return - AI outputs returned to Archivus for storage

What Your AI Handles¶

Configuration¶

Ollama (Recommended for On-Premises)¶

ai:
  provider: ollama
  endpoint: http://localhost:11434
  models:
    chat: llama3.2
    embedding: nomic-embed-text
  timeout: 300s
  max_tokens: 4096

OpenAI (Your Key)¶

ai:
  provider: openai
  api_key: ${OPENAI_API_KEY}  # Your organization's key
  models:
    chat: gpt-4-turbo
    embedding: text-embedding-3-large
  organization: org-xxxxx  # Optional

Azure OpenAI¶

ai:
  provider: azure-openai
  endpoint: https://your-instance.openai.azure.com
  api_key: ${AZURE_OPENAI_KEY}
  api_version: "2024-02-01"
  deployments:
    chat: gpt-4-deployment
    embedding: embedding-deployment

Custom Endpoint¶

For any OpenAI-compatible API:

ai:
  provider: openai-compatible
  endpoint: https://your-llm-server.internal/v1
  api_key: ${YOUR_API_KEY}
  models:
    chat: your-model-name
    embedding: your-embedding-model

Recommended Models¶

For Document Intelligence¶

Hardware Requirements¶

Performance Tuning¶

Ollama Optimization¶

# Environment variables for Ollama
OLLAMA_NUM_PARALLEL=4      # Concurrent requests
OLLAMA_MAX_LOADED_MODELS=2 # Models kept in memory
OLLAMA_KEEP_ALIVE=24h      # Keep models loaded
OLLAMA_GPU_LAYERS=33       # GPU offloading (adjust for VRAM)

Batch Processing¶

For high-volume document processing:

• Enable request batching for embeddings
• Use async processing for non-blocking operations
• Configure queue priorities for different document types

Caching¶

AI responses are cached to reduce redundant processing:

• Identical queries return cached results
• Embeddings cached by content hash
• Classification results cached per document

Fallback Configuration¶

Configure fallback providers for resilience:

ai:
  primary:
    provider: ollama
    endpoint: http://primary-gpu:11434
  fallback:
    provider: openai
    api_key: ${OPENAI_API_KEY}
  fallback_on:
    - timeout
    - server_error
    - rate_limit

When the primary provider fails, Archivus automatically falls back to the secondary.

Security Considerations¶

Network Isolation¶

• Run AI servers on private networks
• Use internal DNS for service discovery
• No internet access required for local LLMs

API Key Security¶

• Store keys in secret management (Vault, AWS Secrets Manager)
• Rotate keys periodically
• Audit API key usage

Model Security¶

• Verify model checksums before deployment
• Use official model sources
• Monitor for model drift or tampering

Cost Comparison¶

Example: 10,000 Documents/Month¶

Break-even for local LLM hardware typically occurs at 20,000-50,000 documents/month depending on infrastructure costs.

Getting Started¶

1. Choose Your Backend¶

• Quick Start: Ollama with Llama 3.2
• Enterprise Cloud: Azure OpenAI or AWS Bedrock
• Maximum Control: Self-hosted vLLM

2. Deploy and Test¶

# Ollama quick start
ollama pull llama3.2
ollama pull nomic-embed-text
curl http://localhost:11434/api/generate -d '{"model":"llama3.2","prompt":"Hello"}'

3. Configure Archivus¶

Update your Archivus configuration with AI backend settings.

4. Validate¶

Run test documents through the pipeline to verify AI processing.

Next Steps¶

• BYOB Storage - Bring your own storage
• Deployment Options - Deployment models
• Compliance - Security certifications