We've parsed over 140,000 XBRL filings from EDINET — every annual securities report for 11,000+ Japanese entities going back a decade. The parser runs in production, processes new filings every 30 minutes, and feeds an API that quant funds, AI agents, and fintech products rely on for accurate data.
Along the way, we catalogued the failure modes that make Japanese financial data extraction genuinely difficult. Not parse errors or missing files — structural ambiguities that produce wrong numbers silently. Numbers that pass type checks, look reasonable in isolation, and corrupt your model without warning.
These aren't edge cases that affect one company. They're classes of problems that recur across filings, across standards, across decades of data.
1. Attribution Ambiguity
Every consolidated financial statement contains multiple versions of the same metric at different scopes. The group total includes all entities. The parent-attributed figure excludes non-controlling interests.
Both values appear in the XBRL. Both are valid. Only one is correct for a given analytical purpose. EPS, ROE, net margin, and screening filters should all use the parent-attributed figure — it's what the company reports to investors, what analysts reference, and what the stock price reflects.
The gap between the two varies. For a company with minimal subsidiaries, it might be less than 1%. For a conglomerate with large joint ventures, it can exceed 20%. In extreme cases, the group total is positive while the parent-attributed figure is negative — same filing, same year, opposite sign.
This isn't just an income statement problem. The same ambiguity exists in equity figures, and it cascades. If both the numerator and denominator of ROE are pulled from different scopes, every derived metric is internally inconsistent.
2. Structural False Positives
XBRL allows companies to define their own elements beyond the standard taxonomy. These company-specific elements follow naming conventions that sometimes collide with standard financial concepts.
A real estate line item might share a prefix with total equity. An investment category might pattern-match against a balance sheet aggregate. The standard element is absent or lower-priority, so a naïve parser selects the company-specific element instead.
The resulting error isn't a small rounding difference. It can be orders of magnitude off. And it only affects companies that happen to have these naming collisions — your parser works for thousands of companies and silently produces garbage for the handful that don't fit the pattern.
This category of failure is particularly dangerous because it's intermittent. You can't catch it with aggregate validation. You need assertion-level checks against known values for specific filers.
3. Temporal Discontinuities
Japan has three accounting standards in active use: JP-GAAP, IFRS, and US-GAAP. Companies transition between them. When they do, the XBRL filing from the transition year contains elements from both standards.
Both sets of elements map to the same normalized fields. Both carry valid values. But one set reflects the current reporting period under the new standard, while the other carries comparative or legacy data from the old standard.
A parser that doesn't track which standard the company currently reports under might extract a value from the wrong standard — and the wrong fiscal period. The resulting error can be tens of trillions of yen for Japan's largest companies. Revenue from four years ago instead of this year, extracted from a filing that parses without errors.
Hundreds of companies have transitioned standards over the past decade. Every transition leaves ghost elements in subsequent filings.
4. Entity Duplication
Large corporate groups sometimes have multiple entities filing separately on EDINET. The parent company files its consolidated report. A subsidiary — before a reorganization, or while still independently listed — files its own.
Both filings contain financial data. Both are valid XBRL. But they represent different entities with different revenue bases. If your system doesn't deduplicate by entity, you get phantom entries: a subsidiary's partial data mixed in with the parent's consolidated data for the same fiscal year.
The phantom rows often have incomplete data — balance sheet items but no income statement, or a single metric from a specific filing section. Any automated analysis encounters these as rows with most fields null, breaking time-series calculations.
5. Sign Incoherence
Financial statements have fields that must be directionally consistent. If net income is negative, earnings per share must be negative. If operating income is positive, operating margin should be positive.
But these fields can be extracted from different sections of the same filing, and different sections can use different scoping rules. A summary section might use parent attribution while the financial statements section uses the group total. When one scope is positive and the other is negative, derived fields contradict each other.
This breaks any analysis that assumes internal consistency. A DCF model using net income for cash flow but EPS for valuation will produce contradictory results. A screening filter for "profitable companies" will include or exclude the same company depending on which field it checks.
The fix isn't just selecting the right element. It's ensuring that every derived metric traces back to the same source figure. No mixing of scopes across fields.
6. Context Resolution
Every XBRL value carries a context identifier that specifies the period, the entity, and the consolidation scope. A single filing contains the same element in multiple contexts — consolidated current year, non-consolidated current year, prior year, by segment, by geography.
The context naming is not standardized. Different filers use different patterns. Some patterns are ambiguous — a context name might match a "current year" regex while actually referring to a different period.
Then there are standalone filers — smaller companies that don't produce consolidated accounts. Their parent-only numbers are the only numbers. A parser that always prefers consolidated context returns null for every field on these companies, discarding perfectly valid data.
The split matters at scale. Roughly 4,100 entities on EDINET are listed companies with consolidated data. Another 7,000+ are standalone filers. A binary "always pick consolidated" rule zeroes out the majority of the dataset.
7. Historical Schema Drift
The FSA updates the XBRL taxonomy annually. Element names change. Elements get deprecated and replaced. New elements appear.
A company that has filed for fifteen years has used multiple versions of the taxonomy. The element that maps to a given financial concept today might not be the same element that mapped to it a decade ago. Or the old element persists in the current taxonomy for backward compatibility but is no longer populated.
The result is time-series data with invisible discontinuities. A metric changes not because the company's business changed, but because the underlying XBRL element changed. The individual values are plausible — the break only becomes visible when you compare across years and notice an impossible shift in a derived ratio.
Why These Failures Persist
These seven classes of ambiguity share a common property: the parser runs, produces output, and returns no errors. The values pass type validation. Totals still foot. Statements still balance. The corruption only becomes visible when you cross-reference against primary sources, compare time-series for discontinuities, or calculate derived metrics that should be internally consistent.
Looks correct does not mean is correct.
Handling these failure modes requires longitudinal analysis across filings, assertion-based validation against known values, and continuous regression testing against historical data. It's not a parsing problem — it's a financial data quality problem that happens to be encoded in XML.
We run 665 automated correctness checks against our parser, covering all three accounting standards, both XBRL formats, consolidation scoping, standard transitions, naming collisions, and cross-field consistency. Every production issue becomes a regression test before it becomes a fix. The suite grows monotonically. We haven't deleted a test yet.
The result is an API where the numbers match audited filings. For all 11,000+ entities, across a decade of data.
If you're building with Japanese financial data, you shouldn't have to learn these failure modes the hard way.