Knowledge Base

The knowledge base allows you to upload documents and connect external sources that workers can reference when processing tasks. Workers use retrieval-augmented generation (RAG) to find relevant information and cite sources in their proposals.

The knowledge base supports both manual document uploads and automated ingestion from connected integrations, keeping your workers informed with up-to-date information from across your organization.

Upload documents directly to your knowledge base. Supported formats include:

• PDF documents
• Plain text files (.txt)
• Markdown files (.md)
• Word documents (.docx)

Documents are automatically processed, chunked, and indexed using vector embeddings for fast semantic search.

Connect integrations to automatically ingest knowledge from external sources. Supported integration sources:

• Notion: Import pages and databases from your Notion workspace, including subpages
• GitHub: Ingest README files, issues, and pull requests from repositories
• Jira: Import project issues with descriptions, comments, and metadata
• RSS Feeds: Subscribe to RSS/Atom feeds to keep workers informed of updates
• Stripe: Import product catalog, pricing, and customer information

Each source maintains its external URL, allowing workers to cite the original source in their proposals.

Ingest content from public websites using the built-in web crawler. The crawler automatically discovers and processes pages from your specified starting URL.

Configuration options:

• Start URL: The URL to begin crawling from
• Allowed Domains: Restrict crawling to specific domains
• Include/Exclude Patterns: URL patterns to include or exclude from crawling
• Max Pages: Maximum number of pages to crawl (limit protects against large sites)
• Max Depth: How many links deep to follow from the starting page
• Respect robots.txt: Honor website crawling rules (enabled by default)
• Rate Limiting: Requests per second limit (default: 1/sec) to avoid overloading target sites

The crawler extracts readable content and converts it to markdown, making it easy for workers to search and cite.

Configure how often knowledge sources sync with their external sources:

• Manual: Only refresh when triggered manually
• Hourly: Sync every hour to capture frequent updates
• Daily: Sync once per day for moderately changing content
• Weekly: Sync once per week for stable content

Automatic refresh is processed by the system job scheduler. Documents are deduplicated by content hash to avoid creating unnecessary duplicate entries.

Control which workers can access specific knowledge sources. By default, all workers in an organization can search the full knowledge base, but you can restrict access to specific workers per source.

When a worker processes a task, it searches relevant knowledge sources for context. Retrieved chunks are included in the worker's prompt, and citations reference the original source URL when available.

1. Upload documents or connect knowledge sources from integrations
2. Content is processed, chunked, and indexed using vector embeddings
3. When processing tasks, workers search for relevant content
4. Retrieved content is included in the worker's context with source attribution
5. Workers cite sources (including external URLs) in their proposals

• Documentation home
• Getting started guide