Point-in-Time Data Is Not a Boolean: What a Backtest Can and Cannot Know

In quantitative research, it is tempting to treat look-ahead as a yes-or-no question. The code does not read tomorrow's price, and financial data carries a disclosure date. The check turns green, and the experiment is described as point-in-time.

But what the code can see and what a researcher could have known historically are two questions that require separate evidence.

A historical record obtained today may incorporate a later correction. A stock list assembled today may exclude companies that subsequently delisted. A price series downloaded today may express old prices on the basis of a split years later. Every row has a date, yet the table may never have reconstructed the information state that existed then.

Point-in-time data, as used here, means data organized according to information obtainable at a particular historical moment. Adding a date field does not satisfy that requirement. We need to explain which fact, in which version, through which channel and under which conditions, entered a particular decision.

Passing a timestamp check is not the same as possessing a dataset whose historical point-in-time integrity has been established.

1. A historical row can contain future knowledge

Suppose a company's fiscal year ends on 31 March. Its financial statements are disclosed on 15 May at 15:00 Japan time, reporting revenue of ¥100 billion.

A table obtained today might contain:

Fiscal period end     31 March
Revenue               ¥100 billion

If a research program joins that record to daily prices using the fiscal period end, an early-April strategy may already use the revenue figure. It has not read a future price, but it still knows the future.

Two kinds of time have been conflated. Period end answers which economic period the number describes. Disclosure time answers when outsiders first had an opportunity to learn it. A financial fact belonging to March does not automatically become tradable information in March.

The J-Quants Pro Financial Data documentation lists CurrentPeriodEndDate separately from DisclosedDate and DisclosedTime. The distinction matters: the period described and the moment of disclosure are different things.

Disclosure is not the whole story of system availability, either. A public announcement does not mean the researcher's data channel has delivered the record, or that a local program has finished downloading, parsing and checking it. J-Quants Pro provides separate data update timing documentation. Research design should account for the particular product and retrieval method in use.

At least four times need to be distinguished:

The research objective affects how these times should be used. Studying market reactions to public information requires a reasonable reconstruction of public visibility and processing latency. Replaying a program that actually ran makes local receipt records especially important. Today's download time cannot substitute for a disclosure time two years ago. An old announcement timestamp cannot establish that one's own system received the information then.

We can adopt a conservative delay assumption, but it should be described as an assumption. Adding an extra day's lag does not automatically restore missing historical versions or universe membership.

2. available_at solves a specific problem

A useful rule in a research system is:

available_at ≤ decision_at

Its meaning is straightforward: a record may enter a decision's information set only if its declared availability time is no later than the decision time.

In the example above, if available_at is 15 May at 15:00, a decision at 14:00 that day must not see the record. The system can enforce the boundary explicitly, without relying on every factor author to handle disclosure delays manually.

The rule deserves implementation and testing. But it establishes a conditional claim: given the time metadata supplied to the system, this input is not later than the decision.

It does not independently establish that the metadata is correct. A corrected record may retain an original disclosure timestamp and pass the comparison. The same may happen if delivery latency is ignored. Filling a missing availability timestamp with fiscal period end merely converts an unknown into a field that looks complete.

Even equality requires interpretation. If disclosure and decision are both marked 15:00, can the latter use the former? That depends on timestamp precision, event ordering and processing delay. A timestamp precise only to the minute cannot establish causal order within that minute. The gate needs explicit event and latency rules.

An audit should therefore ask several distinct questions:

These questions cannot substitute for one another. A clean timestamp does not supply revision history. A complete version archive does not reconstruct security eligibility.

Saying that point-in-time data is not a Boolean does not mean its requirements cannot be tested. Each specific check can pass or fail. The problem is using one check's Boolean result as evidence for the entire historical information set.

3. Revisions can preserve the date while rewriting the past

Return to the hypothetical company. Its original disclosure on 15 May reports revenue of ¥100 billion. On 10 June, a correction changes the figure to ¥92 billion.

The latest account of that fiscal year may appropriately use ¥92 billion. A reconstruction of a decision on 20 May should still use the ¥100 billion figure visible then.

If a database retains only the latest value, today's response may look like this:

Fiscal period end     31 March
Revenue               ¥92 billion

Attaching the original 15 May disclosure timestamp does not solve the problem. It allows a later value to borrow an old timestamp and travel back to a date before it existed.

Historical reconstruction needs records that preserve the different versions:

With those records, 20 May and 11 June can produce different answers while each remains faithful to its own information date.

A vintage is the data version obtainable at a particular moment. It need not be the version ultimately judged correct. Historical investors encountered errors, incomplete information and facts corrected only later. Reconstructing their decision conditions requires preserving those imperfections.

Versions also matter for derived features. Suppose cumulative figures are subtracted to calculate a single-quarter metric. If one cumulative figure is later restated, a quarterly value calculated today may differ from the one calculable then. The result needs a timestamp, but the versions of its underlying inputs also need lineage.

This does not mean every individual researcher must first build a complex database. Keeping original files, disclosure identifiers, retrieval times and transformation scripts is a useful starting point. When historical versions cannot be obtained, that limitation should remain attached to the conclusion.

Repeating a calculation can establish procedural consistency. It cannot establish the historical identity of the input. Two runs using the same later-revised dataset may produce identical results and identically send later knowledge into the past.

4. The universe can leak future information too

Future information may be hidden in a list of security identifiers rather than a financial figure.

Suppose today's index constituents are retrieved and their last ten years of prices downloaded for a backtest. Every price comes from an actual trading record. Signals read only data preceding each decision. The timing appears sound.

Yet the list itself is known today.

During those ten years, companies may have delisted, failed, been acquired or left the index. Today's constituent list will generally not fully represent those historical members. Carrying it backward decides in advance which securities are allowed to remain in the sample.

This is commonly called survivorship bias. Its scope extends beyond companies that remain alive: index changes, listing dates, security-class changes, identifier changes and information used by eligibility rules also matter.

Historical prices do not make a historical universe.

A historical universe needs reconstruction at each decision. Listing dates, index entries and exits, renamings and code changes are part of security identity. Liquidity filters must likewise use trading information available then. Selecting the most active stocks using average turnover over the entire backtest period introduces hindsight into sample selection.

Announcement and effective membership must also be separated. An index change may be announced before it takes effect. The rules should specify whether the strategy studies expectations after announcement or trades only members whose inclusion is already effective.

If only today's list is available, we can still study how this particular set of companies performed historically. That is a meaningful question. It does not answer what investing in historical index constituents would have produced. A changed input scope changes the research question.

After repairing membership, delisted securities' returns, last tradable prices and exit treatment cannot simply be discarded. Restoring their names while deleting the hardest price periods still falls short of reconstructing historical investment conditions.

5. Corporate actions make historical price a model choice

Historical prices need interpretation too.

Suppose a stock undergoes a two-for-one split. To keep its price history comparable, a data service may divide pre-split prices by two. Today's adjusted history then differs from the nominal quotes displayed at the time.

The transformation is not inherently wrong. Raw prices alone can show a split-day jump that is not an economic loss. For some return and momentum calculations, consistent adjustments help avoid mistaking a change of units for a market move.

But saying that adjusted prices were used does not fully describe the method.

First, specify which events the adjustment covers. A split-adjusted series and a total-return series incorporating cash dividends answer different questions. Whether dividends are reinvested and when they reach the cash account also affect return calculations.

Second, specify where the price enters the process. A standardized signal price need not be a simulated execution price. The backtest ledger must keep execution prices, holdings, cash flows and corporate actions consistent. Multiplying adjusted prices by share counts that have not been adjusted accordingly can create a mismatch.

Finally, check whether future adjustments change decisions. For some ratio-based indicators, multiplying every price within a historical window by the same factor may leave the result unchanged. That invariance may not hold for nominal price thresholds, per-share metrics or comparisons across fields. An unaffected return calculation does not establish that every feature is unaffected.

JPX's Corporate Action Data Service lists cash dividends, splits, mergers, tender offers and security-master changes as separate event categories. Its search distinguishes announcement, record and effective dates. Corporate actions should be treated as events with timing and meaning.

The choices to document include raw or adjusted signal prices, split-related quantity changes, dividend treatment, factor availability and whether per-share data and prices use the same unit basis.

We should therefore avoid a blanket claim that today's adjusted history is unusable for backtesting. Equally, a vendor's adjusted close is not a historical fact requiring no explanation. The questions are whether the representation introduces later information into decisions and whether it is compatible with execution and accounting conventions.

6. What KabuForge's checks actually support

These distinctions are boundaries I want research infrastructure to preserve.

The earlier KabuForge framework article introduced the relationships among data, factors, strategies, target portfolios, risk and order planning. Here the question is narrower: how far should a system's checks allow us to extend our claims?

The case in this article refers to KabuForge v0.1.1, with the cited source fixed to its release commit. Its checks should not automatically be generalized to later versions. The research methodology documentation explicitly separates timestamp gating from complete certification. Research inputs declare a decision time and snapshot identity. check_point_in_time and check_lookahead inspect visibility through available_at. The documentation also states that this does not fully certify provenance, vendor timing, revisions, survivorship, corporate actions or universe construction.

The gate identifies and constrains rows that should not enter the current information set. It does not turn supplied metadata into a trustworthy historical archive.

Input identity and run records address another question. Preserving file hashes, snapshot identifiers and manifests helps establish exactly what an experiment read and whether files changed afterward. A hash identifies the stored object; it does not establish whether its contents were historically knowable. A file frozen today may still contain later-revised data even if it is never changed again.

Source-distribution manifests must also be distinguished from research-input manifests. Fixing the software version does not fix the data version. Fixing the data version does not validate its historical availability. Both help review, but they carry different evidentiary responsibilities.

The release's historical-price research path retains these declarations:

readiness: RESEARCH-ONLY
pit_guarantee: false

They apply to that research path, rather than describing every dataset and every run. Real historical data can support teaching, workflow verification and exploration. The word real alone does not establish historical information integrity. The price-research implementation also declares assumptions including the current downloaded vintage and unverified historical visibility, and rejects split-adjusted periods. Those limitations should accompany the results.

What matters to me is making completed checks and unsupported claims visible together. If a path passes a timestamp gate, we can report that check. If historical vintages remain unverified, the result cannot be expanded into a claim of complete PIT assurance.

This also matters for AI-assisted research. AI can quickly generate code for reading data, joining fields and calculating factors. Successful execution cannot supply missing version history. Preserving input identity, timing assumptions and unverified boundaries gives the next modification an explicit reference point.

The KabuForge project page describes current project status. The question here is not whether a strategy works or whether simulated results translate into live returns. It is how a research system can accurately state the conclusions its evidence supports.

7. PIT should be a chain of claims

When describing a backtest, saying that historical data was used is only the starting point.

A stronger claim is that inputs later than the decision were prevented from entering the calculation. A further claim explains how availability was reconstructed from disclosure and the specified channel. Historical versions, security eligibility, corporate actions and transformations then require their own account.

These claims can be arranged as an expanding chain of evidence:

I used historical data.
        ↓
I constrained inputs obviously later than the decision.
        ↓
I reconstructed historical availability through a specified channel.
        ↓
I preserved or reconstructed the versions visible then.
        ↓
I reconstructed the eligible securities at each point in time.
        ↓
I treated prices, quantities and corporate actions consistently.
        ↓
I can trace each decision to its inputs and transformation evidence.

This is not a universal certification scale or an engineering sequence every study must follow. The responsibilities are not entirely linear. One study may preserve complete financial vintages but lack historical index membership. Another may have reliable security history but only approximate delivery-latency assumptions.

The chain helps readers see the components of a claim instead of covering their differences with a label.

A more reviewable statement than saying a backtest has no look-ahead might be:

This experiment constrains financial inputs using disclosure times and an explicit latency assumption, with a fixed data snapshot and transformation rules. Financial revisions have not been fully reconstructed. The security sample uses current membership, so the results describe this set's historical performance and do not estimate historical index investment outcomes.

That conclusion is less polished, but it tells readers what the result supports and which evidence should be added next.

Even after information-set problems are addressed, research can still select the best-looking strategy by chance from many attempts. Bailey and López de Prado discuss this selection bias in their article on backtest overfitting. Historical knowability and statistical validity both need examination. The former cannot substitute for the latter.

For individual researchers, the distinction also determines where time is best spent. When vintage archives are missing, refining timestamp checks may not reduce the main source of error. When a universe incorporates hindsight, adding factors does not bring the sample closer to historical reality. Locating the uncertainty first helps assess whether the next engineering task is worthwhile.

In Why ArcaViso: Reason as Our Guide amid Uncertainty, I discussed observation, judgment and the discipline of finding our own errors. In a backtest, that discipline means avoiding a proof burden that an implementation check cannot carry.

The purpose of research infrastructure is not to turn uncertainty into a green check mark. It is to make clear which uncertainties have actually been reduced and which remain.