Skip to content

Testing Guide

This document describes the current testing strategy for the project, with emphasis on the live-market processor path and the machinery around it.

For the top-level system map, see Architecture Overview.

For the runtime architecture that these tests exercise, see Live Market Pipeline And Processor Machinery.

Test Suite Types

Common suite names in this repo:

  • test: unit tests, in-process integration tests, and manifest-fixture coverage
  • dbIntegrationTest: database-backed integration tests; Docker is required
  • k8sIntegrationTest: real-cluster startup and runtime tests via Testcontainers; Docker is required

Testing Philosophy

The project uses layered verification rather than one giant end-to-end suite for every change.

The important split is:

  • pure strategy semantics live in backend-strategy
  • executor/backfill/channel-management behavior lives in backend-processor
  • processor-path timing, ordering, lifecycle, and resilience behavior lives in backend-app
  • persistence, bridge, and Kubernetes bootstrap behavior are tested in their own modules

That keeps failures localized while still preserving real-path coverage where timing and lifecycle matter.

Coverage Matrix

Layer Main suites What it proves Main seam
Pure strategy semantics backend-strategy:test strategy math, warm-up, output metadata, entries, exits, reversals no processor or transport
Executor boundary backend-processor:test backfill handoff, live handoff, dynamic per-instrument execution management real Strategy.execution(...), custom historical provider
Generic processor contract backend-app:test unit tests routing, flattening, fan-out, lifecycle wiring, readiness, failure handling mocked Strategy.executor(...)
Real processor path backend-app:test processor integration tests Processor -> StrategyExecutor -> StrategyExecution -> TradeExecutor behavior families real processor machinery
Recorder persistence backend-recorder:test and dbIntegrationTest QuestDB schema, retrieval, average-volume lookup, deduplication real QuestDB or JKube fixtures
Bridge and startup backend-app:test, backend-server:test, k8sIntegrationTest kRPC wiring, reconnect behavior, rendered manifests, cluster boot in-process Ktor or K3s

Processor-Focused Test Layout

backend-strategy:test

This is the source of truth for pure strategy behavior.

Current key suites:

  • StrategyExecutionTest
  • EmaStrategyExecutionTest
  • VolumeBreakoutStrategyExecutionTest
  • DecimalVolumeBarBuilderTest

What belongs here:

  • strategy-state transitions
  • bucket/volume-bar timing semantics
  • output metadata
  • entry, exit, and reversal ordering

If a behavior can be proven without Processor, this is the preferred layer.

backend-processor:test

Current key suite:

  • StrategyExecutorTest

What it covers:

  • historical backfill handoff
  • live handoff after backfill
  • dynamic instrument add/remove behavior
  • per-instrument channel management
  • preservation of strategy timing contracts through the executor boundary

Use this layer when the question is about executor behavior rather than app wiring.

backend-app:test: Generic Processor Contract

These tests keep Strategy.executor(...) mocked and verify the processor as infrastructure.

Current suites:

  • ProcessorTest
  • ProcessorReadinessTest
  • ProcessorSignalWiringTest
  • ProcessorFlatMapInstrumentUpdatesTest
  • ProcessorSingleStrategyRoutingTest
  • ProcessorMultiStrategyRoutingTest
  • ProcessorMultiFeedRoutingTest
  • ProcessorOverlappingInstrumentRoutingTest

What belongs here:

  • feed flattening
  • multi-feed routing
  • multi-strategy routing
  • signal/output fan-out
  • readiness and lifecycle wiring
  • failure handling that does not depend on concrete strategy timing

backend-app:test: Real Processor Path Integration

These tests run the real processor machinery end-to-end in process:

Processor -> StrategyExecutor -> StrategyExecution -> TradeExecutor

Current suites:

  • ProcessorStrategyEmaSignalIntegrationTest
  • ProcessorStrategyEmaTimingIntegrationTest
  • ProcessorStrategyVolumeBreakoutEntryIntegrationTest
  • ProcessorStrategyVolumeBreakoutExitIntegrationTest
  • ProcessorPipelineEmaTimingIntegrationTest
  • ProcessorPipelineEmaTradeExecutionIntegrationTest
  • ProcessorPipelineVolumeBreakoutEntryIntegrationTest
  • ProcessorPipelineVolumeBreakoutExitIntegrationTest
  • ProcessorPipelineEventContractIntegrationTest
  • ProcessorPipelineResilienceIntegrationTest
  • ProcessorPipelineRuntimeLifecycleIntegrationTest

What these suites collectively cover:

  • EMA intra-bucket behavior through the full processor path
  • volume_breakout close-gated entries and exits through the full processor path
  • signal-to-trade execution behavior
  • event ordering and linkage contracts
  • dynamic add/remove/re-add lifecycle behavior
  • retained execution behavior for open positions
  • resilience when broker calls fail or trade settings disappear

Current Helper Layout In backend-app

The processor integration helpers were intentionally split by concern. Do not recreate omnibus helper files.

Shared support

  • ProcessorPipelineIntegrationTestSupport.kt
  • ProcessorPipelineEventContractIntegrationHelpers.kt

EMA helper families

  • ProcessorPipelineEmaTimingIntegrationHelpers.kt
  • ProcessorPipelineEmaTradeExecutionIntegrationHelpers.kt
  • ProcessorPipelineEmaResilienceIntegrationHelpers.kt
  • ProcessorPipelineEmaRuntimeIntegrationHelpers.kt
  • ProcessorPipelineEmaRemovedInstrumentIntegrationHelpers.kt
  • ProcessorPipelineEmaIntegrationFixtures.kt

Volume breakout helper families

  • ProcessorPipelineVolumeBreakoutEntryIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutExitIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutResilienceIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutRuntimeIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutRemovedInstrumentIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutIntegrationFixtures.kt

Mixed-strategy helper families

  • ProcessorPipelineMixedStrategySharedIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyBrokerFailureIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyTradeSettingResilienceIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyEventContractIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyRuntimeRegressionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyLifecycleIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyVolumeRetentionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyEmaRetentionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyReadditionIntegrationHelpers.kt

Rule of thumb:

  • extend the nearest existing helper family
  • create a new helper file only when the behavior family is genuinely new
  • keep test helpers named by behavior, not by strategy count or ad hoc scenario names

Choosing The Right Layer For A New Test

Use backend-strategy when:

  • the change is pure strategy semantics
  • the question is about outputs, signals, or reversal ordering without processor concerns

Use backend-processor when:

  • the change is about StrategyExecutor
  • the question is about backfill/live handoff or per-instrument execution lifecycle

Use backend-app processor unit tests when:

  • the change is about generic processor routing or readiness
  • the concrete strategy behavior is irrelevant

Use backend-app real-path processor integration when:

  • timing semantics matter at the processor boundary
  • batching, fan-out, signal ordering, or lifecycle retention matter
  • you need to prove behavior through TradeExecutor

Do not add a processor-path integration test for every new strategy by default.

Add one only when the strategy introduces a new processor-visible family such as:

  • a new timing model
  • a different backfill model
  • a new signal-ordering contract
  • a new lifecycle or retention contract

The existing real-path strategy families are intentionally represented by EMA and volume_breakout.

Event-Time Assertions

The processor-path tests use two different time assertions on purpose:

  • StrategyOutput.occurrenceTime should match the live tick that caused the output
  • StrategySignalEvent.time should fall inside the local emission window, because signals currently use emission time rather than tick time

If you need source tick context for a signal, use:

  • signal.strategyOutputId
  • the linked StrategyOutput.occurrenceTime

Running The Useful Verification Sets

For processor-path timing or lifecycle changes, the useful default command is:

./gradlew :backend-strategy:test :backend-processor:test :backend-app:test

For focused processor-path verification inside backend-app:

./gradlew :backend-app:test \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineEventContractIntegrationTest" \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineResilienceIntegrationTest" \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineRuntimeLifecycleIntegrationTest"

For full backend-app verification, keep in mind that backend-app:test also includes:

  • DataBridgeLauncher in-process kRPC coverage
  • startup unit tests for concurrent DI and fail-fast behavior

Database and cluster suites:

  • dbIntegrationTest requires Docker
  • k8sIntegrationTest requires Docker and is intentionally slower and more operational

Other Important Project Test Areas

backend-app:test

Beyond processor tests, this module also contains:

  • seam-free DataBridgeLauncher tests using testApplication { externalServices { ... } }
  • startup tests for concurrent Ktor module initialization and failure propagation

backend-server:test

This module contains the symmetric bridge-side tests for AppDataBridgeManager, plus startup tests for concurrent initialization and dependency failure handling.

backend-recorder

Important coverage is split between:

  • manifest assertions in test
  • QuestDB-backed persistence assertions in dbIntegrationTest
  • real-cluster bootstrap assertions in k8sIntegrationTest

backend-sync

Important coverage is split between:

  • application logic and manifest assertions in test
  • Postgres-backed recorder assertions in dbIntegrationTest
  • real-cluster bootstrap assertions in k8sIntegrationTest

Current Coverage Posture And Gaps

Strongly covered today:

  • EMA timing families
  • volume_breakout timing families
  • processor event-contract behavior
  • processor runtime lifecycle behavior
  • resilience to broker failures and missing trade settings
  • recorder persistence against a real QuestDB instance
  • bridge reconnect and Kubernetes startup paths

Current gaps or intentionally deferred areas:

  • backend-trade-executor currently has no module-local test suite under src/test; its behavior is mostly exercised through backend-app processor-path tests
  • there is no single in-process suite that runs Processor -> TradeExecutor -> DataRecorder -> QuestDB as one cohesive test; processor-path behavior and recorder persistence are verified separately
  • bridge forwarding and recorder persistence are covered, but not as one monolithic full-stack test

Those are reasonable trade-offs today, but they are useful to keep in mind when adding new behavior.