Architecture Overview

Architecture Overview

Querri is built as a modern microservices application with a focus on scalability, maintainability, and developer experience.

High-Level Architecture

┌─────────────────────────────────────────────────────────────┐
│                         Traefik Proxy                        │
│                  (Routing, JWT Validation)                   │
└─────────────────────────────────────────────────────────────┘
                            │
                ┌───────────┴───────────┐
                │                       │
        ┌───────▼───────┐      ┌───────▼────────┐
        │   SvelteKit   │      │  FastAPI       │
        │   Frontend    │      │  Backend       │
        │   (Port 5173) │      │  (Port 8000)   │
        └───────────────┘      └────────┬───────┘
                                        │
                        ┌───────────────┼────────────┐
                        │               │            │
                ┌───────▼──────┐  ┌────▼─────┐  ┌──▼───────┐
                │ Hub Service  │  │ MongoDB  │  │  Redis   │
                │    (Auth)    │  │          │  │          │
                └──────────────┘  └──────────┘  └──────────┘

Core Services

Frontend (SvelteKit)

Location: /Querri/web-app

The frontend is a SvelteKit application that provides:

• Interactive UI for building data workflows
• Dashboard builder with drag-and-drop
• Real-time updates via WebSockets
• AI chat interface for natural language queries

Key Technologies:

• SvelteKit 2.x
• TypeScript
• Tailwind CSS
• Vite for bundling
• Svelte stores for state management

Routes:

• / - Home dashboard
• /project/[uuid] - Project editor
• /dashboard/[uuid] - Dashboard viewer
• /settings - User settings

Backend (FastAPI)

Location: /Querri/server-api

The backend API handles:

• Project management and execution
• Data processing with QDF (Querri Data Frame)
• Step execution and result storage
• File uploads and management
• Permissions and sharing
• AI integration with OpenAI

Key Technologies:

• FastAPI
• Python 3.11+
• Motor (async MongoDB driver)
• Pydantic for validation
• Redis for caching

API Structure:

/api
├── /projects          # Project CRUD
├── /steps             # Step execution
├── /files             # File management
├── /dashboards        # Dashboard operations
├── /chat              # AI chat interface
└── /permissions       # Access control

Hub Service

Location: /Querri/hub

Authentication and organization management service:

• WorkOS integration for SSO
• User management
• Organization setup
• Session handling

Traefik (Reverse Proxy)

Handles:

• Request routing to appropriate services
• JWT validation for authenticated requests
• SSL/TLS termination in production
• Load balancing across service instances

Data Flow

Project Execution Flow

1. User creates/modifies project in frontend
2. Frontend sends request to /api/projects/{uuid}/steps/{id}/execute
3. Backend loads project graph from MongoDB
4. Step executor runs the requested step
5. Results stored as QDF in MongoDB
6. Frontend polls for completion
7. Results displayed to user

Real-Time Updates Flow

1. WebSocket connection established from frontend
2. Backend subscribes to Redis pub/sub for project updates
3. Step execution completes and publishes event to Redis
4. Backend forwards event to connected WebSocket clients
5. Frontend updates UI in real-time

File Upload Flow

1. User uploads file via frontend
2. Frontend creates presigned upload URL from backend
3. File uploaded to storage (S3 or local filesystem)
4. Backend creates file metadata in MongoDB
5. File processed into QDF format for analysis
6. User can access file in their projects

Database Schema

MongoDB Collections

Projects

{
  _id: ObjectId,
  uuid: String,
  name: String,
  description: String,
  owner_id: String,
  organization_id: String,
  steps: Array,
  dataflows: Array,
  created_at: DateTime,
  updated_at: DateTime
}

Steps

{
  _id: ObjectId,
  uuid: String,
  project_id: String,
  step_type: String,
  configuration: Object,
  result: Object,  // QDF data
  status: String,
  executed_at: DateTime
}

Users

{
  _id: ObjectId,
  workos_id: String,
  email: String,
  name: String,
  organization_id: String,
  preferences: Object,
  created_at: DateTime
}

Redis Data Structures

• Session storage: session:{token}
• Cache: cache:{key}
• Background tasks: queue:tasks
• Pub/sub channels: events:{project_uuid}

Security

Authentication

1. User logs in via WorkOS (SSO)
2. Hub service validates credentials
3. JWT token issued with user claims
4. Frontend stores token in memory/localStorage
5. All API requests include JWT in Authorization header
6. Traefik validates JWT before routing
7. Backend extracts user context from validated JWT

Authorization

Querri uses FGA (Fine-Grained Authorization) for permissions:

• Resource-based permissions (project, dashboard, file)
• Role-based access (viewer, editor, admin)
• Organization-level and user-level permissions
• Public sharing with time-limited tokens

Data Protection

• Input validation using Pydantic models
• SQL injection prevention through parameterized queries
• File upload validation for size and type
• Rate limiting on API endpoints
• Audit logging for sensitive operations

Performance Optimizations

Frontend

• Code splitting for faster initial load
• Lazy loading of dashboard components
• Virtual scrolling for large datasets
• Debounced user inputs
• Optimistic UI updates

Backend

• Async I/O for all database operations
• Connection pooling for MongoDB
• Redis caching for expensive queries
• Streaming for large file downloads
• Background tasks for long-running operations

Database

• Indexes on frequently queried fields
• Aggregation pipelines for complex queries
• Projection to limit returned fields
• TTL indexes for automatic cleanup

Deployment Architecture

Development

docker compose -f docker-compose.dev.yml up

All services run locally with hot reloading.

Production

docker compose up

Services run behind Traefik with:

• Health checks for all containers
• Auto-restart on failure
• Resource limits to prevent overconsumption
• Logging to stdout/stderr for centralized collection

Scalability Considerations

Horizontal Scaling

• Frontend: Multiple instances behind load balancer
• Backend: Stateless API servers can scale independently
• MongoDB: Replica sets for high availability
• Redis: Sentinel for failover

Vertical Scaling

• Memory: QDF operations can use significant RAM for large datasets
• CPU: AI operations benefit from more cores
• Storage: File uploads and database growth

Monitoring & Observability

Health Checks

• Frontend: GET /
• Backend: GET /api/healthz
• Hub: GET /hub/healthz
• MongoDB: Connection check
• Redis: PING command

Logging

• Structured JSON logs for parsing
• Correlation IDs for request tracing
• Error tracking with stack traces
• Performance metrics for slow queries

Next Steps

• Authentication Details
• Data Processing Pipeline
• WebSocket Architecture
• Development Setup