Overview
Loom is an open-source document search engine built for rapid deployment in trusted, task-specific environments. It automates indexing of configured data sources, performs OCR, extracts content and metadata from a wide range of file formats, and provides full-text search with AI-powered retrieval and summarization capabilities.
Loom is designed around a clear operational model: deploy a fresh instance, ingest a dataset, analyze and search the results, shut it down. It is not a long-running SaaS platform. It makes deliberate trade-offs — no user management, no upgrade path guarantees, no hardening for public exposure — in exchange for low deployment friction, full local control, and a transparent, extensible architecture.
Version 1.0.0 represents the first stable, publicly supported release of the project.
What is Included in 1.0.0
Search and Retrieval
Loom exposes a flexible query interface backed by Elasticsearch. Supported query modes include:
- Full-text search across extracted document content and OCR output
- Phrase search using double-quoted terms
- Fuzzy search with configurable edit distance (e.g.,
term~2)
- Metadata filters:
author:,filename:,tags:,extension:,when:,size>,hidden:
- Wildcard matching on filenames and paths
- Saved queries stored in browser-local cache for reuse
- RAG chatbot for conversational interaction grounded in indexed document content
- AI-powered auto tagging — apply a tag to one or more documents and Loom will automatically find and tag semantically similar documents across the entire index
Data Source Integration
Files can be introduced to Loom through several paths:
- Frontend upload — direct file selection or drag-and-drop via the web interface
- S3-compatible API — upload directly to the SeaweedFS object store
The crawler monitors configured sources continuously and schedules indexing for new and updated files.
Content Extraction and OCR
Content extraction is handled by Apache Tika and Gotenberg, supporting:
- Archives: ZIP, PST (mail archives)
- Microsoft Office: Word, Excel, PowerPoint, Visio, Publisher
- PDF: including OCR of scanned pages and embedded images
- Images: OCR across common raster formats
- Email: EML files with full attachment extraction
- Other formats: OpenDocument, RTF, plain text, HTML, XHTML, and additional formats supported by Tika
Large files are handled efficiently via multi-threaded processing. Metadata (author, creation date, MIME type, and format-specific fields) is extracted automatically for every indexed file.
Indexing Pipeline
The indexing pipeline is built on Celery with RabbitMQ as the message broker, deployed as microservices on Kubernetes. Key characteristics:
- Asynchronous, queue-driven processing with configurable parallelism
- Dead-letter and graveyard queues for failed task isolation and retry (up to 5 attempts before permanent failure)
- AI-powered document summarization via Ollama (optional, GPU-accelerated)
- Translation to English via LibreTranslate
- End-to-end encrypted archive export for result sets
Deployment and Operations
Single-node deployment requires only Docker, minikube, helm, kubectl, and skaffold. Running ./up.sh starts the full stack, including all services, on a single machine (minimum: 8 CPU cores, 25 GiB RAM, 200 GiB disk).
Multi-node deployment is available via the official Helm chart published to the GitLab package registry, suitable for existing Kubernetes clusters.
Offline operation is fully supported. Run ./up.sh --offline --delete once while online to cache all images, then disconnect and operate without internet access indefinitely. Offline mode requires deploying from a specific git tag.
GPU support can be enabled with ./up.sh --gpus all for accelerated AI inference via Ollama.
Operational visibility is provided through Flower (Celery task monitor), Grafana dashboards, Prometheus metrics, RabbitMQ management UI, Elasticvue, and RedisInsight — all accessible within the deployment.
Documentation and Configuration
- Getting Started Guide covering upload, search, tagging, archiving, and translation workflows
- Development environment setup with Nix-based reproducible tooling via devenv
- REST API with full OpenAPI/Swagger documentation at
https://api.loom/docs
- Helm chart values documented in
charts/values.yaml; local single-node customization viacharts/values-overwrites.yaml
- Contributing Guidelines and Code of Conduct
Stability and Support
The 1.0.0 release reflects a stable, tested codebase with a complete CI/CD pipeline covering linting, type checking, unit tests, and integration tests. The REST API is documented and generated TypeScript client types are kept in sync with the backend schema.
The project does not guarantee upgrade paths between versions. Operators should treat each deployment as a fresh instance. The 1.0.0 tag is the recommended baseline for offline and reproducible deployments going forward.
Getting Started
Clone the repository and run ./up.sh to start a single-node instance. The frontend is accessible at https://frontend.loom after startup.
Full installation instructions, system requirements, and usage guidance are available in the README and the Getting Started Guide.
Acknowledgements
Loom is developed within the Swiss Armed Forces Cyber Command (CEA). Thank you to all contributors who authored code, wrote tests, reported issues, and shaped the project's architecture and tooling. Thank you also to the maintainers of the open-source components Loom depends on — Elasticsearch, Apache Tika, Ollama, Celery, RabbitMQ, SeaweedFS, LibreTranslate, Gotenberg, and the broader ecosystem that makes a project like this possible.