Skip to content

Architecture Overview

This document is the top-level system map for quant-app.

Use it to understand how the main modules fit together before diving into the live pipeline or the test suites.

Design Goals

These goals explain the architectural tradeoffs reflected in this document. When a decision seems unusual, it is likely here.

  • Fail-fast over partial readiness. Repositories use suspendable factories so a partially initialized object can never become observable to the rest of the application. A crashed pod that Kubernetes restarts is always preferable to a zombie pod that appears healthy but cannot serve. This principle applies at every layer: startup tasks, concurrent modules, and the DataBridge readiness reporter.
  • Runtime isolation between the trading engine and the API tier. backend-app and backend-server are separate deployables. The API tier can stay up while a trading session restarts, and a server-side bug cannot corrupt live strategy state. DataBridge and AppDataBridge are the only sanctioned crossing points.
  • Single source of truth per data domain. QuestDB owns raw time-series market data. The main datastore (Gel) owns application state — strategies, feeds, orders, positions. The backend-sync Postgres instance owns sync-event lifecycle. These boundaries must not blur: historical data is never served from in-memory state, and application state is never written directly to QuestDB.
  • Reproducibility over convenience. All infrastructure is Terraform-managed. All Kubernetes resources are generated by JKube from Gradle DSL. No manual cloud console steps should be required to recreate any environment. Convenience shortcuts that bypass this are always technical debt.
  • Correctness over throughput. This is a personal research platform, not a high-frequency system. Where a tradeoff must be made between latency and correctness (e.g. dedup keys in QuestDB, guarded state transitions in backend-sync), prefer correctness.

System At A Glance

quant-app is split into two main runtime applications plus supporting storage, bridge, and frontend modules:

  • backend-app is the live trading runtime. It ingests live market data, runs strategies, executes trades, records market data, and publishes runtime state upstream.
  • backend-server is the API and aggregation runtime. It exposes GraphQL/SSE/HTTP-facing behavior, aggregates connected app instances, and brokers historical requests back to backend-app.
  • frontend-app is the user-facing Compose Multiplatform client.
  • backend-web-app packages the frontend web distribution into the deployable Caddy image.
  • backend-database, backend-recorder, and backend-sync provide the main storage and synchronization infrastructure around the two runtimes.

At a high level:

External market feeds
  -> backend-app
       -> Processor
       -> TradeExecutor
       -> DataRecorder / QuestDB
       -> DataBridge -> backend-server

frontend-app
  -> backend-server
       -> DataStream over DataBridge connections
       -> AppDataBridge back to backend-app for historical data

backend-database
  -> main application datastore

backend-sync
  -> sync-event persistence and data-source sync scheduling

Main Runtime Boundaries

backend-app

backend-app is responsible for:

  • restoring feed and strategy state at startup
  • collecting live market data from datasource integrations
  • running the processor pipeline
  • executing trades through the broker path
  • recording raw market data and reading historical backfill
  • streaming runtime state to backend-server

This is the module where the live strategy machinery actually runs.

backend-server

backend-server is responsible for:

  • hosting the inbound DataBridge server for connected app instances
  • exposing the downstream HTTP, SSE, and GraphQL surfaces
  • aggregating runtime state across active app connections through DataStream
  • calling back into backend-app through AppDataBridge for historical market data

backend-server is an aggregation and delivery boundary, not the place where strategy execution happens.

backend-data-bridge

backend-data-bridge is the shared contract module between the two runtimes.

It contains transport-agnostic APIs and DTOs for two distinct directions:

  • AppDataBridge: request/response calls from backend-server to backend-app, currently centered on historical market data lookup
  • DataBridge: long-lived streaming from backend-app to backend-server, carrying strategy output, strategy signals, market-feed events, trades, open positions, readiness, and recovery/error state

On the server side, DataStream is the aggregation surface over all active DataBridge connections.

Module Map

The repo contains many modules, but they cluster into a few architectural areas.

Area Main modules Responsibility
Runtime applications backend-app, backend-server, backend-web-app, frontend-app live execution runtime, API/runtime aggregation, web packaging, and UI
Strategy and execution engine backend-processor, backend-strategy, backend-trade-executor, backend-broker strategy execution, processor orchestration, trade execution, and broker integration
Market data and recovery backend-datasources, backend-instrument-map, backend-historical-data-provider, backend-recovery, backend-readiness live datasource integration, instrument/feed mapping, historical lookup interfaces, recovery wrappers, readiness reporting
Persistence and contracts backend-data-bridge, backend-recorder, backend-database, backend-repository, backend-storage, backend-sync runtime bridge contracts, QuestDB integration, main datastore deployment, repositories/storage access, and sync-event persistence
Frontend support frontend-api, frontend-graphql, frontend-websocket, frontend-charts, frontend-fts API clients, transport bindings, chart rendering, and frontend support features
Shared foundations shared-* common utilities such as core models, caching, logging, KV storage, HTTP/websocket helpers, and task scheduling
Build and deployment build-logic, terraform/edge/*, terraform/kubernetes Gradle convention plugins, Cloudflare/OCI edge infrastructure, and Kubernetes deployment infrastructure

You do not need to know every module in detail before working in the repo. The key starting point is understanding which layer owns the behavior you are changing.

Runtime Data Flows

Live Market Path

The live market path starts in backend-app:

  1. subscribed instruments are restored and mapped to datasource feeds
  2. live market data is collected from datasource integrations
  3. Processor fans ticks into StrategyExecutor and StrategyExecution
  4. TradeExecutor consumes signals and emits trade events
  5. raw market data is recorded to QuestDB and trade events are persisted through the orders repository
  6. runtime events are forwarded to backend-server through DataBridgeLauncher

This path is described in detail in Live Market Pipeline And Processor Machinery.

Historical Data Path

Historical data flows in the opposite direction:

  1. backend-server receives a historical request from downstream consumers
  2. AppDataBridgeManager calls backend-app over AppDataBridge
  3. backend-app serves the request from AppDataBridgeImpl
  4. the implementation reads historical data from HistoricalDataProvider
  5. in production, that provider is backed by recorder/QuestDB access and rolls the raw data up to the requested interval before returning it

Server Aggregation Path

backend-server does not own live processing state directly. Instead it:

  • accepts one or more DataBridge connections from app instances
  • aggregates them through DataStream
  • exposes the merged state through its delivery surfaces
  • reports bridge readiness based on active app connections

That separation matters when debugging bugs that look like "server state is stale" but are actually upstream bridge or app-runtime issues.

Bootstrap And Dependency Injection

Both backend-app and backend-server use the same startup model:

  • Ktor concurrent modules are enabled with startup: concurrent
  • suspendable startup dependencies are registered provider-first so Ktor can await them during startup
  • repositories such as OrdersRepository, StrategyRepository, and PositionsRepository initialize fail-fast through suspendable factories
  • Dagger 2 remains the compile-time safety boundary that assembles the final application graph
  • startup failures bubble to the engine and terminate the process, which lets Kubernetes treat initialization failure as a crash rather than leaving a partially alive pod

The practical outcome is that the system prefers hard startup failure over partial readiness.

Storage And Persistence Roles

The main persistence responsibilities are split intentionally:

  • backend-database packages the primary application datastore deployment and schema bootstrap
  • backend-recorder provides QuestDB integration for raw market data ingestion and historical lookup; its module also owns the recorder deployment/manifests
  • backend-sync owns sync-event persistence and instrument-sync recovery state using Postgres
  • application-level orders, positions, and strategy/feed configuration live behind the main repositories rather than in QuestDB

One useful mental model:

  • Gel/main datastore: application state
  • QuestDB: time-series market data
  • backend-sync Postgres: sync-event lifecycle

Frontend And Delivery Model

frontend-app is the main UI codebase and produces the web distribution consumed by backend-web-app.

backend-web-app does not build its own frontend. It resolves the frontend-app web artifact, stages it into the Caddy image, and becomes the deployable web surface.

Supporting frontend modules such as frontend-graphql, frontend-websocket, frontend-api, and frontend-charts keep transport and presentation concerns out of the main UI module.

Build And Deployment Model

The project uses Gradle build types and custom build logic:

  • DEV, STG, and PROD are controlled through -PbuildType=<type>
  • build-logic contains the convention plugins for build-type config, Kubernetes image naming, JKube test-fixture rendering, and JKube DSL extensions
  • modules that build images use conventions.kubernetes.image
  • modules with richer manifest customization use jkube.extension

Deployment is split between JKube and Terraform:

  • JKube renders Kubernetes resources from the application modules
  • terraform/edge/oci owns the OCI/Cloudflare production edge
  • terraform/edge/local owns the local dev Cloudflare edge
  • terraform/kubernetes owns in-cluster monitoring and app workloads behind that edge

How To Navigate The Docs

Use the docs in this order:

  1. Architecture Overview for the module and boundary map
  2. Live Market Pipeline And Processor Machinery for the live runtime path
  3. Testing Guide for how that behavior is verified
  4. Roadmap for current milestone status and what to work on next

For subsystem-specific operational details, keep using the module-local docs listed in docs/README.md.