Skip to content

Testing Guide

This document describes the current testing strategy for the project, with emphasis on the live-market processor path and the machinery around it.

For the top-level system map, see Architecture Overview.

For the runtime architecture that these tests exercise, see Live Market Pipeline And Processor Machinery.

Test Framework Conventions

  • Test runner — backend modules (backend-*) use JUnit 5 (@Test, @BeforeAll, @AfterAll, @TestInstance). Frontend/KMP modules (frontend-*, shared-*) use Kotest spec styles (StringSpec, etc.) to support KMP targets.
  • Assertions — all modules use Kotest assertions (shouldBe, shouldBeInstanceOf, shouldContain, etc.) regardless of which runner is used.
  • Coroutines in JUnit tests — wrap suspend calls with runTest { }. Use @TestInstance(Lifecycle.PER_CLASS) when the test class holds shared state (e.g. a Testcontainer) across all tests.

Test Suite Types

Common suite names in this repo:

  • test: unit tests, in-process integration tests, and manifest-fixture coverage
  • dbIntegrationTest: database-backed integration tests; Docker is required
  • k8sIntegrationTest: real-cluster startup and runtime tests via Testcontainers; Docker is required

Local container-backed suites under rootless Podman

On the current local rootless Podman setup, Testcontainers-backed suites need two explicit environment overrides before running Gradle:

export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock
export TESTCONTAINERS_RYUK_DISABLED=true

These overrides are required for the current Podman machine setup because Ryuk otherwise fails during startup while trying to create the Podman socket mountpoint under /var/folders/.../podman-machine-default-api.sock.

Additionally, Podman's API proxy has a default idle connection timeout that can cause Testcontainers operations to fail with Docker API timeouts (see testcontainers-java#7310). To fix this, set service_timeout = 0 in the Podman VM's containers.conf:

[engine]
service_timeout = 0

On macOS, this file must be edited inside the Podman VM (not on the host). To access the VM:

podman machine ssh
# Then edit /etc/containers/containers.conf
sudo vi /etc/containers/containers.conf
# Add or set: service_timeout = 0 under [engine]
# Exit and restart the machine
exit
podman machine stop
podman machine start

Setting service_timeout = 0 disables the idle timeout entirely, which prevents the API proxy from closing connections that Testcontainers expects to remain open.

This applies to local container-backed suites such as dbIntegrationTest, k8sIntegrationTest, and Gel-backed storage regressions that start containers through Testcontainers.

Testing Philosophy

The project uses layered verification rather than one giant end-to-end suite for every change.

The important split is:

  • pure strategy semantics live in backend-strategy
  • executor/backfill/channel-management behavior lives in backend-processor
  • processor-path timing, ordering, lifecycle, and resilience behavior lives in backend-app
  • persistence, bridge, and Kubernetes bootstrap behavior are tested in their own modules

That keeps failures localized while still preserving real-path coverage where timing and lifecycle matter.

Coverage Matrix

Layer Main suites What it proves Main seam
Pure strategy semantics backend-strategy:test strategy math, warm-up, output metadata, entries, exits, reversals no processor or transport
Executor boundary backend-processor:test backfill handoff, live handoff, dynamic per-instrument execution management real Strategy.execution(...), custom historical provider
Generic processor contract backend-app:test unit tests routing, flattening, fan-out, lifecycle wiring, readiness, failure handling mocked Strategy.executor(...)
Real processor path backend-app:test processor integration tests Processor -> StrategyExecutor -> StrategyExecution -> TradeExecutor behavior families real processor machinery
Recorder persistence backend-recorder:test and dbIntegrationTest QuestDB schema, retrieval, average-volume lookup, deduplication real QuestDB or JKube fixtures
Bridge and startup backend-app:test, backend-server:test, k8sIntegrationTest kRPC wiring, reconnect behavior, rendered manifests, cluster boot in-process Ktor or K3s

Processor-Focused Test Layout

backend-strategy:test

This is the source of truth for pure strategy behavior.

Current key suites:

  • StrategyExecutionTest
  • EmaStrategyExecutionTest
  • VolumeBreakoutStrategyExecutionTest
  • DecimalVolumeBarBuilderTest

What belongs here:

  • strategy-state transitions
  • bucket/volume-bar timing semantics
  • output metadata
  • entry, exit, and reversal ordering

If a behavior can be proven without Processor, this is the preferred layer.

backend-processor:test

Current key suite:

  • StrategyExecutorTest

What it covers:

  • historical backfill handoff
  • live handoff after backfill
  • dynamic instrument add/remove behavior
  • per-instrument channel management
  • preservation of strategy timing contracts through the executor boundary

Use this layer when the question is about executor behavior rather than app wiring.

backend-app:test: Generic Processor Contract

These tests keep Strategy.executor(...) mocked and verify the processor as infrastructure.

Current suites:

  • ProcessorTest
  • ProcessorReadinessTest
  • ProcessorSignalWiringTest
  • ProcessorFlatMapInstrumentUpdatesTest
  • ProcessorSingleStrategyRoutingTest
  • ProcessorMultiStrategyRoutingTest
  • ProcessorMultiFeedRoutingTest
  • ProcessorOverlappingInstrumentRoutingTest

What belongs here:

  • feed flattening
  • multi-feed routing
  • multi-strategy routing
  • signal/output fan-out
  • readiness and lifecycle wiring
  • failure handling that does not depend on concrete strategy timing

backend-app:test: Real Processor Path Integration

These tests run the real processor machinery end-to-end in process:

Processor -> StrategyExecutor -> StrategyExecution -> TradeExecutor

Current suites:

  • ProcessorStrategyEmaSignalIntegrationTest
  • ProcessorStrategyEmaTimingIntegrationTest
  • ProcessorStrategyVolumeBreakoutEntryIntegrationTest
  • ProcessorStrategyVolumeBreakoutExitIntegrationTest
  • ProcessorPipelineEmaTimingIntegrationTest
  • ProcessorPipelineEmaTradeExecutionIntegrationTest
  • ProcessorPipelineVolumeBreakoutEntryIntegrationTest
  • ProcessorPipelineVolumeBreakoutExitIntegrationTest
  • ProcessorPipelineEventContractIntegrationTest
  • ProcessorPipelineResilienceIntegrationTest
  • ProcessorPipelineRuntimeLifecycleIntegrationTest

What these suites collectively cover:

  • EMA intra-bucket behavior through the full processor path
  • volume_breakout close-gated entries and exits through the full processor path
  • signal-to-trade execution behavior
  • event ordering and linkage contracts
  • dynamic add/remove/re-add lifecycle behavior
  • retained execution behavior for open positions
  • resilience when broker calls fail or trade settings disappear

Current Helper Layout In backend-app

The processor integration helpers were intentionally split by concern. Do not recreate omnibus helper files.

Shared support

  • ProcessorPipelineIntegrationTestSupport.kt
  • ProcessorPipelineEventContractIntegrationHelpers.kt

EMA helper families

  • ProcessorPipelineEmaTimingIntegrationHelpers.kt
  • ProcessorPipelineEmaTradeExecutionIntegrationHelpers.kt
  • ProcessorPipelineEmaResilienceIntegrationHelpers.kt
  • ProcessorPipelineEmaRuntimeIntegrationHelpers.kt
  • ProcessorPipelineEmaRemovedInstrumentIntegrationHelpers.kt
  • ProcessorPipelineEmaIntegrationFixtures.kt

Volume breakout helper families

  • ProcessorPipelineVolumeBreakoutEntryIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutExitIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutResilienceIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutRuntimeIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutRemovedInstrumentIntegrationHelpers.kt
  • ProcessorPipelineVolumeBreakoutIntegrationFixtures.kt

Mixed-strategy helper families

  • ProcessorPipelineMixedStrategySharedIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyBrokerFailureIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyTradeSettingResilienceIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyEventContractIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyRuntimeRegressionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyLifecycleIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyVolumeRetentionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyEmaRetentionIntegrationHelpers.kt
  • ProcessorPipelineMixedStrategyReadditionIntegrationHelpers.kt

Rule of thumb:

  • extend the nearest existing helper family
  • create a new helper file only when the behavior family is genuinely new
  • keep test helpers named by behavior, not by strategy count or ad hoc scenario names

Choosing The Right Layer For A New Test

Use backend-strategy when:

  • the change is pure strategy semantics
  • the question is about outputs, signals, or reversal ordering without processor concerns

Use backend-processor when:

  • the change is about StrategyExecutor
  • the question is about backfill/live handoff or per-instrument execution lifecycle

Use backend-app processor unit tests when:

  • the change is about generic processor routing or readiness
  • the concrete strategy behavior is irrelevant

Use backend-app real-path processor integration when:

  • timing semantics matter at the processor boundary
  • batching, fan-out, signal ordering, or lifecycle retention matter
  • you need to prove behavior through TradeExecutor

Do not add a processor-path integration test for every new strategy by default.

Add one only when the strategy introduces a new processor-visible family such as:

  • a new timing model
  • a different backfill model
  • a new signal-ordering contract
  • a new lifecycle or retention contract

The existing real-path strategy families are intentionally represented by EMA and volume_breakout.

Event-Time Assertions

The processor-path tests use two different time assertions on purpose:

  • StrategyOutput.occurrenceTime should match the live tick that caused the output
  • StrategySignalEvent.time should fall inside the local emission window, because signals currently use emission time rather than tick time

If you need source tick context for a signal, use:

  • signal.strategyOutputId
  • the linked StrategyOutput.occurrenceTime

Running The Useful Verification Sets

For processor-path timing or lifecycle changes, the useful default command is:

./gradlew :backend-strategy:test :backend-processor:test :backend-app:test

For focused processor-path verification inside backend-app:

./gradlew :backend-app:test \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineEventContractIntegrationTest" \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineResilienceIntegrationTest" \
  --tests "com.timemanx.quant.server.app.processor.ProcessorPipelineRuntimeLifecycleIntegrationTest"

For full backend-app verification, keep in mind that backend-app:test also includes:

  • DataBridgeLauncher in-process kRPC coverage
  • startup unit tests for concurrent DI and fail-fast behavior

Database and cluster suites:

  • dbIntegrationTest requires Docker
  • k8sIntegrationTest requires Docker and is intentionally slower and more operational

Other Important Project Test Areas

backend-app:test

Beyond processor tests, this module also contains:

  • seam-free DataBridgeLauncher tests using testApplication { externalServices { ... } }
  • startup tests for concurrent Ktor module initialization and failure propagation
  • AuthStateCoordinatorTest — verifies recovery publication, obsolete-prompt cleanup, datasource resume on Valid transition, idempotent start, multi-broker support, and distinctUntilChanged filtering
  • ResilientTokenBrokerServiceTest — verifies single upstream subscription across multiple consumers (cold flow with subscription counter) and ZERODHA passthrough
  • ResilientProvisionedTokenBrokerServiceTest — verifies single upstream subscription across multiple consumers for provisioned-token handles and delegation of invalidateToken/reloadToken
  • OAuthTokenHealthReporterTest — verifies health status mapping for all OAuthTokenState variants, active/inactive broker partitioning, FAILED for non-recoverable failures, INACTIVE for no active brokers
  • ProvisionedTokenHealthReporterTest — verifies health status mapping for ProvisionedTokenState (Valid → HEALTHY, Failed → DEGRADED)
  • ResilientTokenStateFlowTest — verifies Arrow Schedule-based retry with backoff reset on successful emission
  • RequiredFeedReadinessReporterTest — verifies feed-health aggregation with HEALTHY/DEGRADED/BLOCKED precedence
  • ApplicationReadinessTest — verifies DEGRADED reporters still yield 200 OK on /readiness

backend-server:test

This module contains the symmetric bridge-side tests for AppDataBridgeManager, plus startup tests for concurrent initialization and dependency failure handling.

backend-token-broker:test

This module contains TokenManager and ProvisionedTokenManager tests covering:

TokenManager (OAuth):

  • suspend factory initialization (DataStore load before instance is returned)
  • concurrent invalidate()/submitAuthCode() serialization under initGuard mutex
  • _activeSubmitCount atomic counter preventing Exchanging state wedge after joiner completion
  • CancellationException propagation from exchange strategies without becoming OAuthTokenState.Failed
  • non-recoverable exchange failures (NonRecoverableTokenException) resulting in OAuthTokenState.Failed
  • write-through persistence with drop(1) skip for the init emission

ProvisionedTokenManager (provisioned tokens, runTest with virtual time):

  • startup with missing/expired/unparseable/valid token files
  • invalidate() with unchanged vs changed files
  • reload() with valid vs missing files
  • poll-loop recovery after file update and poll-loop stops after success
  • startup with missing/expired files starts poller and recovers after file update
  • concurrent invalidations do not create duplicate polling jobs
  • stale expiry timer does not override newer valid token (identity check)
  • transient unreadable file followed by same bytes recovers via poller

backend-broker:test

This module contains HttpClientAuthInvalidationTest covering auth-invalidation paths in shared-httpclient's HttpClient:

  • REST 4xx TokenError classification via UpstoxErrorParser triggers clearTokenCache() + onUnauthorized()
  • REST 401 fallback triggers both callbacks even without TokenError classification
  • REST 4xx non-token error does not trigger callbacks
  • onUnauthorized failure propagation (not swallowed by runCatching)
  • WebSocket upgrade 401 fallback triggers both callbacks
  • WebSocket upgrade 4xx TokenError classification triggers both callbacks
  • WebSocket 4xx non-token error does not trigger callbacks
  • WebSocket onUnauthorized failure propagation

These tests exercise the actual HttpClient.withConfig code path against a real Netty test server, verifying both REST (ClientRequestException) and WebSocket (WebSocketException) error branches.

backend-datasources:test

This module contains UpstoxDataSource tests covering:

  • terminal token failure handling (TokenUnavailableException stops retry, records terminal failure)
  • structured health reporting (DEGRADED after terminal failure, HEALTHY when stream is active, BLOCKED when not running)
  • resumeIfStopped() idempotency (no-op when stream already running or no subscribers)
  • new subscriber restarts upstream after terminal failure without resumeIfStopped()
  • reports HEALTHY after resumeIfStopped() when upstream reconnects without emitting ticks (quiet market)
  • day-boundary detection via vtt-decrease and time-delta heuristics
  • tick-to-MarketData conversion with per-tick volume delta computation

backend-readiness:test

This module contains DataSourceReadinessReporter tests covering:

  • HEALTHY component health when datasource is healthy
  • DEGRADED component health when datasource has a terminal failure
  • BLOCKED component health when datasource is not ready

backend-recovery:test

This module contains RecoveryManager tests covering:

  • logical recovery coalescing (same recoveryUrl produces one error)
  • targeted recovery (removes only the matching error)
  • clearLogicalRecovery() for obsolete prompts
  • clearLogicalRecovery() resumes waiting collectors

backend-recorder

Important coverage is split between:

  • manifest assertions in test
  • QuestDB-backed persistence assertions in dbIntegrationTest
  • real-cluster bootstrap assertions in k8sIntegrationTest

backend-sync

Important coverage is split between:

  • application logic, failure-propagation, and manifest assertions in test
  • Postgres-backed recorder assertions in dbIntegrationTest
  • real-cluster bootstrap assertions in k8sIntegrationTest

backend-storage

Important coverage is split between:

  • unit and integration tests in test
  • Gel-backed storage regression tests in dbIntegrationTest
  • stress tests in dbIntegrationTest tagged with @Tag("stress")

DataStoreImplTest is a Gel-backed integration suite:

  • Starts a Gel 7.1 Testcontainer, bootstraps the schema, and runs all storage operations against a real database.
  • Covers feeds, instruments, strategies, trade settings, orders, and positions with full round-trip assertions.
  • Uses DATA_RESET_QUERY to clear state between tests.

DataStoreStressTest is a Gel-backed stress suite:

  • Uses a local complete.json resource (~136k Upstox instruments) to exercise addInstruments at production scale.
  • Tagged with @Tag("stress") for selective execution.
  • Uses UpstoxInstrumentDeserializer from backend-broker for production-quality instrument parsing (same code path as UpstoxApiDecorator).
  • Tests: full sync timing, incremental sync with 10% churn, concurrent reads during writes, idempotent re-sync, and full read performance with data integrity spot-checks.
  • Timing-sensitive tests use @RepeatedTest(3) with measureTimedValue for throughput calculation and regression guards.
  • Requires a running Docker daemon (Testcontainers).

Current Coverage Posture And Gaps

Strongly covered today:

  • EMA timing families
  • volume_breakout timing families
  • processor event-contract behavior
  • processor runtime lifecycle behavior
  • resilience to broker failures and missing trade settings
  • recorder persistence against a real QuestDB instance
  • bridge reconnect and Kubernetes startup paths
  • terminal token failure handling and DEGRADED health reporting
  • datasource and feed-level health aggregation (HEALTHY/DEGRADED/BLOCKED)
  • RecoveryManager coalescing, targeted recovery, and obsolete-prompt cleanup
  • REST and WebSocket auth-invalidation paths in shared-httpclient (8 tests via Netty test server)
  • TokenManager concurrent invalidate()/submitAuthCode() serialization and suspend factory
  • ProvisionedTokenManager mutex-owned state machine, polling recovery, concurrent invalidations, stale expiry timers, and transient I/O recovery (16 tests)
  • ResilientTokenBrokerService and ResilientProvisionedTokenBrokerService single-subscription guarantee (cold-flow subscription counter)
  • OAuthTokenHealthReporter (HEALTHY/DEGRADED/BLOCKED/FAILED with active/inactive broker partitioning) and ProvisionedTokenHealthReporter (HEALTHY/DEGRADED) health status mapping
  • Arrow Schedule-based retry with backoff reset for token-state stream resilience
  • AuthStateCoordinator recovery publication, obsolete-prompt cleanup, and datasource resume

Current gaps or intentionally deferred areas:

  • backend-trade-executor currently has no module-local test suite under src/test; its behavior is mostly exercised through backend-app processor-path tests
  • there is no single in-process suite that runs Processor -> TradeExecutor -> DataRecorder -> QuestDB as one cohesive test; processor-path behavior and recorder persistence are verified separately
  • bridge forwarding and recorder persistence are covered, but not as one monolithic full-stack test
  • backend-sync still has no end-to-end broker-to-Gel instrument-sync regression that exercises a real datasource payload plus final Gel persistence assertions
  • backend-sync still has no dedicated test for the feed-bootstrap precondition it relies on, or for the current "log-and-skip malformed source row but still report success" behavior

Those are reasonable trade-offs today, but they are useful to keep in mind when adding new behavior.