Intelligent Rules Engine

Self-Learning Document Organization with Hierarchical Rules (All Tiers)

---

Overview

The Rules Engine automates document organization through intelligent, hierarchical rules that learn from your behavior. Create rules once, and watch as Archivus automatically routes documents, applies tags, sets metadata, triggers workflows, and sends notifications based on 30+ document conditions.

Unlike traditional rule engines, Archivus learns from your manual corrections and suggests new rules based on detected patterns, continuously improving organization accuracy over time.

---

Key Features

• Hierarchical Rules: 5 scope levels (Tenant → Organization → Workspace → Project → User)
• 30+ Condition Fields: Match on document type, vendor, amount, AI categories, extracted text, and more
• 15 Operators: From simple equality to regex patterns and temporal conditions
• 7 Action Types: Route, tag, assign, notify, trigger workflows, and request reviews
• AI Learning: Detects patterns from your corrections and suggests rules automatically
• Full Audit Trail: Every rule execution logged with explainability
• Tier-Based Scaling: Free (5 rules) to Enterprise (unlimited)

---

How It Works

┌─────────────────────────────────────────┐
│       Document Upload/Move Event        │
└───────────────────┬─────────────────────┘
                    │
           ┌────────▼─────────┐
           │  Load Applicable  │
           │  Rules (Cached)   │
           └────────┬─────────┘
                    │
           ┌────────▼─────────┐
           │ Resolve Hierarchy │
           │  (Inherit/Override/│
           │   Extend/Enforce) │
           └────────┬─────────┘
                    │
           ┌────────▼─────────┐
           │ Evaluate Conditions│
           │ (AND/OR Groups)   │
           └────────┬─────────┘
                    │
              ┌─────┴─────┐
              │           │
         ┌────▼────┐ ┌────▼────┐
         │ Matched │ │ Skipped │
         └────┬────┘ └─────────┘
              │
    ┌─────────┼─────────┐
    │         │         │
┌───▼───┐ ┌───▼───┐ ┌───▼────┐
│ Route │ │  Tag  │ │ Notify │
│Folder │ │Document│ │ Team  │
└───┬───┘ └───┬───┘ └───┬────┘
    │         │         │
    └─────────┼─────────┘
              │
         ┌────▼────┐
         │  Audit  │
         │   Log   │
         └─────────┘

---

Rule Structure

Conditions

Match documents based on 30+ fields with 15 powerful operators:

Document Fields:

• document_type - Document classification (invoice, receipt, contract, etc.)
• content_type - MIME type (application/pdf, image/jpeg, etc.)
• file_name - Current filename
• file_size - File size in bytes

Metadata Fields:

• vendor_name - Vendor/supplier name
• customer_name - Customer name
• amount - Document amount (numeric)
• currency - Currency code (USD, EUR, etc.)
• document_date - Document date
• due_date - Due date

AI-Generated Fields:

• ai_summary - AI-generated summary
• ai_primary_topic - Primary topic classification
• ai_confidence - AI confidence score (0-1)
• ai_categories - AI-detected categories (array)
• extracted_text - OCR/extracted text content

Temporal Fields (Pro+):

• created_at - Document creation timestamp
• hour_of_day - Hour (0-23)
• day_of_week - Day (0=Sunday, 6=Saturday)
• month_of_year - Month (1-12)

Operators:

Equality:     eq, neq
String:       contains, not_contains, starts_with, ends_with
Regex (Pro+): matches, not_matches
Numeric:      gt, gte, lt, lte
Array:        in, not_in
Null:         is_empty, is_not_empty, exists, not_exists
Temporal:     before, after, between (Pro+)

Example Condition:

{
  "id": "cond1",
  "field": "document_type",
  "operator": "eq",
  "value": "invoice",
  "mandatory": true
}

Actions

Execute 7 powerful actions when rules match:

1. Route to Folder

Move documents to specific folders with dynamic path variables:

{
  "type": "route_to_folder",
  "config": {
    "folder_path": "/Accounting/Inbox/{YEAR}/{VENDOR}"
  }
}

Dynamic Variables: {YEAR}, {MONTH}, {DAY}, {VENDOR}, {PROJECT}, {TYPE}, {USER}

2. Apply Tags

Add static and dynamic tags:

{
  "type": "apply_tags",
  "config": {
    "tags": ["invoice", "accounting"],
    "dynamic_tags": ["{VENDOR}", "{YEAR}"]
  }
}

3. Set Metadata

Update document metadata fields:

{
  "type": "set_metadata",
  "config": {
    "metadata": {
      "category": "financial",
      "priority": "high",
      "processed_by": "rules_engine"
    }
  }
}

4. Assign to Project

Associate document with project:

{
  "type": "assign_to_project",
  "config": {
    "project_id": "project-uuid-1234"
  }
}

5. Trigger Workflow

Start a DAG workflow:

{
  "type": "trigger_workflow",
  "config": {
    "workflow_id": "dag-uuid-5678",
    "input_data": {
      "priority": "high"
    }
  }
}

6. Send Notification

Notify team members:

{
  "type": "send_notification",
  "config": {
    "channel": "email",
    "recipients": ["accounting@example.com"],
    "message": "New invoice:  - $"
  }
}

Supported channels: email, websocket, slack

7. Request Review

Queue for human review:

{
  "type": "request_review",
  "config": {
    "reviewer_id": "user-uuid-9012",
    "reason": "Large amount requires approval"
  }
}

---

Hierarchical Rules

Scope Levels

Rules can be defined at 5 levels of granularity:

Tenant (applies to entire organization)
  └─ Organization (specific legal entity)
      └─ Workspace (department/team)
          └─ Project (specific initiative)
              └─ User (personal preferences)

More specific rules take precedence, allowing organization-wide policies while enabling team customization.

Inheritance Modes

Control how child rules interact with parent rules:

1. Inherit (Default)

• Merges child with parent
• Combines conditions (AND logic)
• Appends actions
• Best for: Adding specificity to broad rules

2. Override

• Replaces parent rule entirely
• No parent conditions or actions inherited
• Best for: Exceptions to general rules

3. Extend

• Adds to parent’s conditions (OR logic for optional conditions)
• Appends additional actions
• Best for: Building upon parent rules

4. Enforce (Enterprise Only)

• Locks rule from child overrides
• Ensures organization-wide policies are respected
• Best for: Compliance and security policies

Example: Hierarchical Rule System

Tenant-Level Rule (applies to all documents):

{
  "scope_type": "tenant",
  "name": "Tag All Invoices",
  "conditions": [
    {"field": "document_type", "operator": "eq", "value": "invoice"}
  ],
  "actions": [
    {"type": "apply_tags", "config": {"tags": ["invoice"]}}
  ]
}

Organization-Level Rule (extends tenant rule):

{
  "scope_type": "organization",
  "organization_id": "org-uuid-1234",
  "name": "Route Large Invoices",
  "inheritance_mode": "extend",
  "conditions": [
    {"field": "amount", "operator": "gte", "value": 1000}
  ],
  "actions": [
    {"type": "route_to_folder", "config": {"folder_path": "/High-Value-Invoices"}},
    {"type": "send_notification", "config": {"channel": "email", "recipients": ["cfo@example.com"]}}
  ]
}

Result: Invoices in this organization get:

1. Tagged with “invoice” (tenant rule)
2. Routed to /High-Value-Invoices if amount >= $1,000 (org rule)
3. CFO notified if amount >= $1,000 (org rule)

---

AI Learning System

Pattern Detection

The Rules Engine learns from your manual corrections:

1. User moves document from folder A to folder B
2. System records correction with document context
3. AI analyzes corrections for patterns (minimum 5 occurrences)
4. System suggests rule with confidence score
5. User approves/rejects suggestion

Example Detection:

User moved 7 receipts under $50 to "Personal Expenses" folder
→ AI suggests rule: "Route receipts < $50 to Personal Expenses"
→ Confidence: 87% (7 corrections / 8 total receipts)

Correction Types

• moved - Document moved to different folder
• retagged - Tags changed
• reclassified - Document type changed
• undone - Rule action manually reversed

Feedback Types

• implicit - System infers from user actions (moving documents, changing tags)
• explicit - User provides direct feedback (thumbs up/down, comments)

Suggestion Sources

• pattern_detection - Identified from user corrections
• user_correction - Single high-confidence correction
• similar_tenant - Learned from similar organizations (Enterprise)

---

Tier Limits

Feature	Free	Starter	Pro	Team	Enterprise
Max Rules	5	25	100	500	Unlimited
Org-Level Rules	-	-	-	50	Unlimited
Workspace Rules	-	5	20	100	Unlimited
Project Rules	-	-	10	50	Unlimited
Regex Conditions	-	-	✓	✓	✓
Temporal Conditions	-	-	✓	✓	✓
Enforce Mode	-	-	-	-	✓
AI Suggestions	✓	✓	✓	✓	✓

---

Use Cases

1. Invoice Processing Automation

Rule: Route invoices to accounting by vendor and year

{
  "name": "Organize Invoices by Vendor",
  "conditions": [
    {"field": "document_type", "operator": "eq", "value": "invoice"}
  ],
  "actions": [
    {
      "type": "route_to_folder",
      "config": {
        "folder_path": "/Accounting/Invoices/{YEAR}/{VENDOR}"
      }
    },
    {
      "type": "apply_tags",
      "config": {
        "tags": ["invoice"],
        "dynamic_tags": ["{VENDOR}", "{YEAR}"]
      }
    },
    {
      "type": "send_notification",
      "config": {
        "channel": "email",
        "recipients": ["accounting@example.com"],
        "message": "New invoice from : $"
      }
    }
  ]
}

2. High-Value Contract Review

Rule: Flag large contracts for legal review

{
  "name": "Legal Review for Large Contracts",
  "conditions": [
    {"field": "document_type", "operator": "eq", "value": "contract"},
    {"field": "amount", "operator": "gte", "value": 50000}
  ],
  "actions": [
    {
      "type": "route_to_folder",
      "config": {"folder_path": "/Legal/Contracts/High-Value"}
    },
    {
      "type": "apply_tags",
      "config": {"tags": ["contract", "high-value", "requires-review"]}
    },
    {
      "type": "request_review",
      "config": {
        "reviewer_id": "legal-team-uuid",
        "reason": "Contract value exceeds $50,000"
      }
    },
    {
      "type": "send_notification",
      "config": {
        "channel": "email",
        "recipients": ["legal@example.com"],
        "message": "High-value contract requires review:  - $"
      }
    }
  ]
}

3. Time-Based Document Routing (Pro+)

Rule: Route documents differently based on business hours

{
  "name": "After-Hours Document Queue",
  "conditions": [
    {"field": "hour_of_day", "operator": "gt", "value": 17},  // After 5 PM
    {"field": "document_type", "operator": "in", "value": ["invoice", "receipt"]}
  ],
  "actions": [
    {
      "type": "route_to_folder",
      "config": {"folder_path": "/Processing Queue/After-Hours"}
    },
    {
      "type": "apply_tags",
      "config": {"tags": ["after-hours", "pending-review"]}
    }
  ]
}

4. AI-Based Categorization with Confidence Threshold

Rule: Auto-organize high-confidence AI classifications

{
  "name": "Auto-Organize High-Confidence Documents",
  "conditions": [
    {"field": "ai_confidence", "operator": "gte", "value": 0.9},
    {"field": "ai_primary_topic", "operator": "exists", "value": true}
  ],
  "actions": [
    {
      "type": "route_to_folder",
      "config": {"folder_path": "/Auto-Organized/{TYPE}"}
    },
    {
      "type": "apply_tags",
      "config": {"dynamic_tags": ["{TYPE}", "auto-organized"]}
    }
  ]
}

5. Compliance-Enforced Organization Policy (Enterprise)

Rule: Enforce PCI compliance for payment documents

{
  "name": "PCI Compliance Policy",
  "scope_type": "tenant",
  "inheritance_mode": "enforce",  // Cannot be overridden
  "conditions": [
    {"field": "ai_categories", "operator": "in", "value": ["payment", "credit-card"]},
    {"field": "extracted_text", "operator": "matches", "value": "\\d{4}[- ]?\\d{4}[- ]?\\d{4}[- ]?\\d{4}"}
  ],
  "actions": [
    {
      "type": "route_to_folder",
      "config": {"folder_path": "/Secure/PCI-Compliance"}
    },
    {
      "type": "apply_tags",
      "config": {"tags": ["pci-sensitive", "encrypted"]}
    },
    {
      "type": "set_metadata",
      "config": {
        "metadata": {
          "compliance_category": "pci",
          "security_level": "high"
        }
      }
    },
    {
      "type": "send_notification",
      "config": {
        "channel": "email",
        "recipients": ["compliance@example.com"],
        "message": "PCI-sensitive document detected and secured"
      }
    }
  ]
}

---

Testing & Validation

Dry-Run Testing

Test rules against documents without executing actions:

POST /api/v1/rules/:id/test
{
  "document_id": "doc-123-uuid"
}

Response:

{
  "matched": true,
  "explanation": "All conditions matched: document_type=invoice, amount >= 100",
  "matched_fields": ["document_type", "amount"],
  "actions_to_execute": [
    {"type": "route_to_folder", "config": {...}},
    {"type": "apply_tags", "config": {...}}
  ]
}

Rule Explainability

Understand which rules apply to a document and why:

GET /api/v1/rules/explain/:document_id

Response:

{
  "document_id": "doc-123-uuid",
  "rules": [
    {
      "rule_id": "rule-abc-123",
      "rule_name": "Route Invoices to Accounting",
      "matched": true,
      "explanation": "All mandatory conditions matched",
      "matched_conditions": ["document_type", "amount"],
      "actions_to_execute": [...]
    },
    {
      "rule_id": "rule-def-456",
      "rule_name": "Tag Large Invoices",
      "matched": false,
      "explanation": "Condition failed: amount < 1000 (required >= 1000)"
    }
  ]
}

---

Performance

• Redis Cache: 5-minute TTL for active rules (80-90% cache hit rate)
• 14 Database Indexes: Optimized for hierarchy lookup and execution history
• DuckDB Analytics: Historical executions offloaded to S3 Parquet for fast reporting
• Async Execution: Rule evaluation doesn’t block document upload
• Parallel Actions: Independent actions execute concurrently

---

Security

• Tenant Isolation: All rules and executions scoped by tenant_id with RLS
• System Rules: Protected from modification/deletion (compliance policies)
• Tier-Based Gating: Feature flags enforced at API level
• Audit Trail: Every execution logged with full context
• Input Validation: Regex patterns validated, malformed JSONB rejected

---

Best Practices

1. Start Broad, Refine Specific

Create tenant-level rules for general organization, then add workspace/project rules for specific needs:

Tenant Rule:    Tag all invoices
   ↓
Org Rule:       Route invoices by department
   ↓
Workspace Rule: Route large invoices to CFO review

2. Use Mandatory Conditions Wisely

Mark critical conditions as mandatory: true to ensure they must match:

{
  "field": "document_type",
  "operator": "eq",
  "value": "invoice",
  "mandatory": true  // Document MUST be an invoice
}

3. Leverage AI Suggestions

Review AI-suggested rules weekly:

• Approve high-confidence suggestions (>85%)
• Modify medium-confidence suggestions (60-85%)
• Reject low-confidence suggestions (<60%)

4. Test Before Activating

Always test new rules against sample documents:

# Create rule as inactive
POST /api/v1/rules
{ "is_active": false, ... }

# Test against documents
POST /api/v1/rules/:id/test
{ "document_id": "sample-doc-uuid" }

# Activate when confident
POST /api/v1/rules/:id/activate

5. Monitor Execution History

Track rule performance:

GET /api/v1/rules/:id/executions?status=failed

Review failed executions to refine conditions.

6. Use Priority for Execution Order

Lower priority numbers execute first:

{ "priority": 10, "name": "Critical Security Rule" }
{ "priority": 50, "name": "General Organization Rule" }
{ "priority": 100, "name": "Nice-to-Have Tagging" }

---

API Reference

See API Reference - Rules Engine for complete endpoint documentation.

Quick Links:

• Create Rule: POST /api/v1/rules
• List Rules: GET /api/v1/rules
• Test Rule: POST /api/v1/rules/:id/test
• AI Suggestions: GET /api/v1/rules/suggestions
• Execution History: GET /api/v1/rules/:id/executions

---

Migration Guide

From Manual Organization

1. Observe Patterns: Use Archivus for 1-2 weeks with manual organization
2. Review AI Suggestions: Check suggested rules based on your corrections
3. Start Simple: Approve 2-3 high-confidence suggestions
4. Expand Gradually: Add custom rules for specific needs
5. Monitor & Refine: Track execution history and adjust conditions

From Other Document Management Systems

1. Map Existing Rules: Document your current organization logic
2. Create Tenant Rules: Translate organization-wide policies first
3. Add Workspace Rules: Department-specific rules next
4. Test Extensively: Dry-run against sample documents
5. Activate Incrementally: Roll out in phases (critical rules first)

---

Roadmap

Q2 2026:

• Visual rule builder UI
• Bulk rule creation from templates
• Rule performance analytics dashboard
• Multi-language support for AI suggestions

Q3 2026:

• Cross-tenant learning (Enterprise)
• Advanced ML-based pattern detection
• Rule recommendations based on industry best practices
• Custom action plugins

Q4 2026:

• Natural language rule creation (“Move all invoices from Acme Corp to the Accounting folder”)
• Rule versioning and rollback
• A/B testing for rule effectiveness
• Integration with external workflow engines

---

Support

• Documentation: API Reference

  Handler Architecture

• Examples: See Use Cases above
• Community: Discord
• Enterprise: Contact your account manager for advanced configurations