Less Technical Explanation

How this is made

Turning a decade and a half of SEC filings back into financial statements — what breaks along the way, and what the result is honestly good for.

Want the SQL? Read the technical explanation.

The short version

Every public company in the US files its financial statements with the SEC in a machine-readable format called XBRL. You’d think that means you can just download a company’s income statement.

You can’t.

What the SEC actually publishes is the filing taken apart into pieces and stored in seven separate files. One file has all the numbers, in no particular order. Another knows which line goes where, but has no numbers. A third knows the arithmetic — what adds up into what — but doesn’t know the order or the labels. A fourth knows the statement titles. None of them, on its own, knows what a balance sheet looks like.

This project puts the pieces back together.

The pieces

Think of a financial statement as needing four things, each living in a different file.

🔢 The numbers

A giant bag of values. "4,545,000,000." It knows the concept and the date, and nothing else. No order, no label, no idea which statement it belongs to.

📄 The running order

Every line the filing prints, in order, with the label the company chose — "Cost of revenue", "Total current assets". This is the skeleton, but every bone is hollow. No values.

🧮 The arithmetic

A list of parent–child relationships: these lines add up into that line. This is where indentation comes from. But it has no order and no labels — just concept names pointing at each other.

🏷️ The titles

Which report is the balance sheet, which is a note, which is the cover page.

Reassembly is mostly a matter of joining these correctly, and the joins are rarely as clean as you’d hope.

The trick: structure and values are separate problems

Here’s the thing that shapes everything else.

A 10-K only shows you two or three years. So if you want fifteen years of revenue, you can’t get it from one filing — you need fifteen filings. But if you build your table from fifteen different filings, which one decides what the rows are? Companies rename lines, add lines, drop lines, and reorganise their statements constantly.

The answer this project uses: the newest 10-K defines the shape, every filing supplies the numbers.

The most recent annual report decides which rows exist, what they’re called, and how they nest. Then the model goes hunting through every 10-K and 10-Q the company has ever filed to fill those rows in, as far back as the data goes.

It’s a bit like taking this year’s org chart and then going back through fifteen years of payroll records to fill in what each role was paid — you get a clean, consistent table, but the shape is always today’s shape.

That choice has consequences. We’ll get to them.

Where it gets interesting

The indentation problem

Financial statements are indented, and the indentation carries meaning. "Cash and cash equivalents" is nested under "Current assets:", which is nested under "ASSETS:". Collapse the detail and you should be left with the subtotals.

The SEC data has no indentation column. None. It has to be reconstructed from the arithmetic file — if a line feeds into a subtotal, it sits one level below it. Walking that parent–child structure recursively gives you a tree, and the tree gives you the indentation.

Except the header rows ("ASSETS:", "Operating expenses:") aren’t part of any arithmetic — they’re just headings, they don’t add up to anything. So they have no depth at all. Those get inferred by looking at what comes underneath them: a run of headers sitting above a real line steps up one level per row.

The two files that don’t know about each other

The file with the running order numbers its reports 1, 2, 3… The file with the arithmetic numbers its networks 1, 2, 3… These numbers have nothing to do with each other. There is no key linking them.

So the model matches them by overlap: which arithmetic network shares the most concepts with this report? That mostly works, but naively it fails in an instructive way — the balance sheet and the equity statement share so many concepts that the balance sheet’s arithmetic gets attached to the equity statement.

The fix is to require a good match, not merely the best one. If nothing scores well enough, that statement simply renders flat, with no indentation. That’s the honest answer: many companies’ equity statements genuinely have no arithmetic network of their own.

The year that vanished

Here’s my favourite one.

NVIDIA’s revenue is tagged with a concept called Revenues. Simple enough — except that in fiscal 2019 NVIDIA switched to a different concept for revenue (an accounting-standard change), used it for three years, and then switched back in fiscal 2022.

Each 10-K only carries three years of history. The last filing to use the old name reached back to FY2018. The first filing to restore it reached back to FY2020. FY2019 fell straight through the gap. Build the row from today’s 10-K and FY2019 is blank — not because the number is missing, but because it’s filed under a name this row no longer uses.

Nothing published anywhere would tell you those two concepts are the same thing — the old one was never deprecated, the company just changed its mind twice. So the model works it out from the company’s own numbers: if two concepts reported identical values in every period where both appear, and never disagreed, and have the same sign convention and the same time basis, they’re the same line.

Those filled-in values are marked in amber wherever they appear, and every substitution is listed with its evidence. Nothing is silently swapped.

The concept that changes underfoot

That NVIDIA story turned out to be common. Across the companies processed so far, there are hundreds of concept changes — lines that quietly changed what they were tagged with, some of them reverting years later. That history is logged as a by-product: every change, what it was before, what it became, which filing introduced it, and whether the company had used that concept before.

Nothing is standardised

This is the most important thing to understand about the output, so it gets its own section.

Every number is exactly as the company reported it. Nothing is normalised, mapped, restated, reclassified or made comparable across companies. There is no "standard" chart of accounts here. What you see is what the filer filed.

That means:

  • If two companies call the same thing by different names, they stay different. One company’s "Revenue" is another’s "Net sales" is another’s "Total revenues".
  • If one company reports "Cost of revenue" and another reports "Cost of goods sold" and a third splits it into three lines, you get exactly that.
  • Line ordering follows each company’s own statement, not a common template.
  • Subtotals are the company’s own subtotals.
  • Even the fiscal calendar is the company’s own — a fiscal year is labelled for the calendar year it ends in.

The one place values are substituted is the concept-alias case described above, and only within a single company, only where that company’s own numbers prove the two concepts are the same line, and always visibly marked.

If you want cross-company comparability, that’s a different — and much harder — project, and it necessarily involves judgement calls about what counts as equivalent. This project deliberately stops short of that so that everything in it can be traced back to a filing.

Limitations

An honest list. Most of these are consequences of design choices rather than bugs, and each one has a plausible remedy if you’re willing to pay for it. Click any to expand.

1 The row structure is today’s structure

The newest 10-K defines the rows. A line the company reported for years and has since dropped doesn’t appear at all — not as a blank row, not anywhere. Conversely, a line introduced recently shows a short history.

Remedy: build the row set from the union of all filings rather than one, and carry a "last reported in" marker per row. This is substantially harder because line order and labels conflict across filings — you’d need a merge strategy for rows that exist in some years and not others.

2 Concept substitution is inference, not fact

The alias rule requires at least two periods of exact agreement, zero disagreements, matching sign convention and time basis, and that the alternate concept isn’t itself a line on the statement. That’s strong evidence, but it’s still inference. Two genuinely different concepts that happen to coincide for one company can slip through — an issuance line and an employee-stock-plan line that always matched, say.

Remedy: a reviewed exception list, so a human can veto a specific pair. Raising the agreement threshold also helps, at the cost of missing real renames that had short overlap windows.

3 A column can mix filings, so a balance sheet may not foot

Each concept independently takes its most recently filed value. If a later filing restated one line but didn’t carry another, one column ends up mixing vintages, and the internal articulation breaks. Observed in Berkshire Hathaway (two years) and Walmart (one year), with discrepancies around 0.05%.

Remedy: treat a statement as a coherent artifact — for each period, pick the single most authoritative filing and take the whole column from it, falling back per-concept only where that filing is silent.

4 US domestic filers only

The model anchors on a 10-K. Foreign private issuers file 20-F (or 40-F for Canadian issuers) and are skipped entirely — ASML, HSBC, Alibaba and ARM among them. Many also report under IFRS rather than US GAAP.

Remedy: extend the form filter and handle IFRS taxonomies. Non-trivial: 20-F statement structures differ, and IFRS concept names are a separate vocabulary.

5 No standalone fourth quarter

Companies don’t tag a Q4 duration — the fourth quarter only exists inside the annual figure. Flow items (revenue, expenses, cash flows) show Q1, Q2, Q3 and FY. Balance-sheet lines are unaffected, since every quarter end has its own snapshot.

Remedy: derive Q4 as FY minus the first three quarters. Correct for additive items, wrong for per-share figures, averages and ratios — so it needs a per-concept rule about what may be subtracted.

6 Year-to-date figures are dropped

A 10-Q reports both the quarter and the cumulative year to date. Keeping both would put two different numbers under one column heading, so only the single quarter is kept.

Remedy: add the duration to the column identity and show YTD as its own set of columns. Roughly doubles the width of the quarterly output.

7 Restatements overwrite the original

Where a figure was later restated, the newest version wins. You cannot see what was originally reported — which is precisely what a restatement study would want.

Remedy: keep all vintages with the filing that reported each, and let the consumer choose "as originally filed" or "as most recently restated". The data to do this is already collected; only the collapse discards it.

8 Statements only — the notes are excluded

Only reports the SEC classifies as financial statements are included. The notes, where segment detail, tax reconciliations, debt schedules and lease maturities live, are not.

Remedy: include the note reports too. They’re available and structured the same way; the cost is a much larger and messier output, since note tables are irregular.

9 Segment breakdowns depend on the statement declaring an axis

A product or geography split attaches to a line only if that statement declares the matching dimension. This prevents a concept from dragging every breakdown it has anywhere into every statement — but a filer who reports a breakdown without declaring the axis on that statement will have it omitted.

Remedy: fall back to attaching breakdowns by concept where no axis is declared, flagged as lower confidence.

10 Header indentation is inferred

Header rows carry no arithmetic, so their nesting level is guessed from the rows beneath them. It’s usually right and occasionally puts a heading one level off.

Remedy: nothing available in this dataset — the indentation genuinely isn’t published. Parsing the rendered filing itself would settle it.

11 Some statements come out flat

If no arithmetic network matches a report well enough, the whole statement renders without indentation. Equity statements are the usual casualty.

Remedy: fall back to a presentation-order heuristic, or hand-build hierarchies for the common statement types.

12 Line identity in the change log is the label

The concept-change log tracks a line by its label, since the concept is the thing that moved. A company that reworded a label and retagged in the same filing appears as one line ending and another starting, rather than as a change.

Remedy: cross-reference the value-agreement evidence, which catches renames the label-based match misses. Both signals exist; they’re just not currently merged.

13 Corporate history isn’t tracked

Everything keys on CIK. Mergers, redomiciliations and re-listings create new CIKs, so history simply starts there — Broadcom begins in 2015 because the current entity was created by the 2018 restructuring, and its predecessor sits under a different CIK.

Remedy: a CIK-succession map, hand-curated or derived from EDGAR’s former-name records.

14 Odd dates create odd columns

Companies occasionally report a figure at a non-reporting date — shares authorised on the day of a stock split, for instance. These are real disclosures and get their own column, labelled with the month to distinguish them from quarters. They interrupt the rhythm of an otherwise regular time series.

Remedy: filter them out, at the cost of dropping genuine data.

15 Fiscal-year naming may not match the company’s own

A fiscal year is labelled for the calendar year it ends in. That matches how Apple, Microsoft and NVIDIA name their own fiscal years — but some retailers with January and February year ends call it the previous year. The label is internally consistent; it may not match a given company’s marketing.

Remedy: a per-company override, or simply show the period end date alongside.

16 Mixed units in one column

Dollars, share counts and per-share amounts sit in the same column, distinguished only by row. Summing a column is meaningless.

Remedy: the unit is carried per row and could drive separate sections or sheets.

17 Data currency and completeness

Everything comes from the SEC’s quarterly Financial Statement Data Sets, which lag filings by roughly a couple of months, and from whichever archive files are present on disk. A missing quarterly file leaves a hole with no warning.

Remedy: validate archive continuity on load and warn on gaps.

18 Filer-defined concepts are company-specific

Companies can invent their own concepts when the standard taxonomy doesn’t fit. These work fine within a company and are meaningless across companies. Roughly a fifth of the concept changes observed involve one.

Remedy: nothing to fix, really — but they’re flagged in the change log so you can see where a company left the standard vocabulary.

What it’s good for

Given all that, what this is good for: seeing a single company’s own statements, as that company reported them, over a much longer span than any one filing shows — with the arithmetic structure intact, every substitution visible, and every number traceable back to the filing it came from.

What it is not: a comparable cross-company database. Building one of those means making thousands of judgement calls about equivalence, and every one of them is a place where the data stops being what the company actually said.

Go have a look

The workbooks are free. If you find something that looks wrong, that’s the most useful email you can send: olsent@gonzaga.edu. I’d love to hear from you!