Survivorship Bias: Audit the Accounts and Instruments Missing from a Sample

Define who could have been selected then

Start with the population available at the beginning of the decision period. A list of today's visible accounts, funds or instruments may omit entities that closed, delisted or stopped reporting. Using that later list to evaluate an earlier selection process imports information about survival into the sample.

QuantConnect's research guide describes the problem with historical tests based only on surviving securities. The same sampling logic can be applied to account directories, provided the analysis distinguishes verified closures from merely missing data.

Work through a cohort example

Imagine six hypothetical accounts eligible on a fixed starting date, each with the same initial capital and no cash flows. Their period returns are +20%, +15%, +10%, −10%, −30% and −50%. The equally weighted arithmetic mean is −45% divided by six, or −7.5%.

Suppose only the first three are still shown in a directory when the review is written. Their mean is +15%. The 22.5-percentage-point difference results from changing the population, not from a calculation error in either average. A report on the three survivors may be numerically correct while failing to describe the original selection opportunity.

Build an inclusion and exclusion ledger

Record a stable identifier, eligibility start and end dates, data coverage, status changes and the reason each entity was included or excluded. Preserve evidence of the historical directory or instrument universe. If a broker account changed identifiers, determine whether it is the same economic history rather than counting its good and bad periods separately.

Missing is not synonymous with failed. An account can disappear for administrative reasons, and a failed account may remain public. Do not invent outcomes for absent histories. Instead state coverage, identify known omissions and, where useful, show transparent sensitivity scenarios under clearly labeled assumptions.

Separate weighting from survival

Even a complete cohort can produce different summaries under equal-account and capital-weighted methods. The six-account example uses equal initial capital so its weighting is unambiguous. If one account is much larger, an aggregate capital return answers a different question from the average experience of a randomly selected account.

Also preserve inactive periods and reporting delays. Backfilling an account only after a strong opening run can create selection even if no account is later removed. Requiring a long surviving history before inclusion changes the sample and should be justified against the original research question.

State what the evidence supports

A survivorship check cannot recover data that never existed, but it can prevent an incomplete sample from being presented as comprehensive. Report the denominator of eligible entities, observed coverage and unresolved missingness beside performance figures. For copy-trading research, avoid inferring that a current leaderboard represents all source accounts a user could have chosen historically. The goal is an honest description of the available evidence, not a promise that a corrected historical average predicts future results.