Can this backtest be trusted?
Test decision-time causality, bar semantics, execution timing, and declared assumptions.
Independent, bounded, evidence-led
SCYLLA
Quantitative Trading Systems
Integrity & Validation
Independent verification of backtests, execution simulations, market-data pipelines, quantitative research workflows, reproducibility, and historical/live consistency.
We test whether system and research claims are supported by temporally correct, reproducible, and traceable evidence—within an explicitly frozen scope.
The questions behind the audit
What we verify
A convincing result can still rest on future information, invalid event order, optimistic fills, contaminated research splits, or an unreproducible build.
Does future information leak backward?
Use counterfactual histories that remain identical through the causal boundary.
Are fills and costs applied correctly?
Trace signal, order, eligibility, price source, fees, slippage, and latency assumptions.
Is market data temporally sound?
Check gaps, duplicates, replays, staleness, ordering, and availability semantics.
Does research respect validation boundaries?
Inspect temporal splits, labels, preprocessing, tuning, and final-test reuse.
Can another engineer reproduce it?
Freeze code, configuration, dependencies, inputs, lineage, and canonical output.
Core public offering
Integrity audits
Each engagement is bounded to explicit code, data, configuration, and claims. Untested components remain untested.
Backtest Integrity Audit
Look-ahead leakage, decision-time causality, same-bar semantics, path dependence, and bounded positive controls.
Execution & Fill Audit
Signal-to-order-to-fill sequencing, price eligibility, commissions, slippage, latency, and stated fill assumptions.
Market Data Integrity Audit
Timestamp meaning, availability, duplicates, gaps, replays, staleness, ordering, and unsafe filling.
Research Validation & ML Integrity Audit
Temporal splits, future-feature and label leakage, preprocessing boundaries, experiment identity, and test reuse.
Reproducibility & Data-Lineage Audit
Frozen versions, deterministic replay, raw-to-result traceability, dependency identity, and rebuild evidence.
Specialist extensions
Scoped assessments
Available only where venue, framework, evidence, and modelling boundaries can be defined precisely.
Historical / Live Parity Assessment
Compare clocks, event ordering, state transitions, connector normalization, and declared execution behaviour within the tested path.
Order-Book / Microstructure Assessment
Snapshot/diff sequencing, gaps, reconciliation, latency and queue assumptions—without claiming unknowable historical queue position.
Derivatives / Session-Rule Assessment
Contract identity, sessions, funding, expiry, rollover, forced closes, and venue-specific rules within an agreed scope.
A traceable engagement
How an audit works
The exact sequence is adapted to scope; no engagement is represented as testing components it did not examine.
- 01
Client intake
Start with one system, result, or integrity question.
- 02
Scope freeze
Define paths, claims, evidence, exclusions, and success criteria.
- 03
Version freeze
Pin repository commit, configuration, and dependencies where applicable.
- 04
Fixtures
Construct synthetic or counterfactual cases with explicit causal boundaries.
- 05
Bounded reproduction
Reproduce only supported behavior against the frozen target.
- 06
Evidence classification
Separate documentation, code trace, reproduction, controls, and unknowns.
- 07
Finding assessment
Record scope, causal impact, severity, limitations, and uncertainty.
- 08
Client-safe report
Deliver reproducible findings and verified controls without overclaiming.
- 09
Retest criteria
Define objective closure conditions for each actionable finding.
- 10
Corrected-version retest
When agreed, rerun the bounded fixture against the exact corrected version.
What the client receives
Evidence that can be reviewed
A clean result is a valid outcome. Findings and verified controls are both recorded.
- Frozen technical scope
- Finding and control report
- Reproducible evidence
- Explicit limitations
- Reproduction instructions or package where applicable
- Objective retest criteria
Visible boundaries
Integrity validation is bounded evidence—not a universal certificate.
An audit does not guarantee profitability, investment performance, total system correctness, absence of bugs, or correctness outside the tested scope.
- No trading signals or strategy development
- No alpha or profitability certification
- No investment advice
- No security certification unless separately scoped
- No universal framework or venue certification
- No claim without supporting evidence
Data minimization
Start with the smallest reviewable surface
- Public repository where possible
- Minimal reproducible subset
- No exchange or brokerage credentials
- No private keys or seed phrases
- No unnecessary customer data
- Private material only after scope agreement
Start with one testable question
Request a bounded integrity diagnostic
Mikhail | SCYLLA
scylla.audit@gmail.com