How to handle large JSON files

Where the cost actually is

A 50 MB JSON file is not 50 MB once parsed. Every object, array and string becomes a separate heap allocation with its own overhead, and the in-memory result is routinely five to ten times the file size. A browser tab that has a gigabyte or two available runs out well before you expect.

Rendering is the second and larger cost. Drawing a tree of a million nodes means a million DOM elements, and no amount of memory makes that responsive.

Rough limits in a browser tab

• Up to ~5 MB — everything works, including full tree and table rendering.
• 5–25 MB — parsing and conversion stay fast; tree rendering needs paging, and expand-all should be bounded.
• 25–100 MB — parse it, but do not render it whole. Work with a slice, or query it.
• Above 100 MB — a browser is the wrong tool. Stream it.

Techniques that work

• Do the parsing in a Web Worker so the page keeps responding and the job stays cancellable.
• Render only what is on screen — page or virtualise long arrays instead of expanding them.
• Query rather than browse: extract the part you need with a path expression, then work with that.
• Convert to NDJSON — one JSON document per line — and process line by line, which removes the memory problem entirely.

# top-level shape, without parsing everything
head -c 2000 big.json

# extract one array element at a time
jq -c '.items[]' big.json > items.ndjson

# then work a line at a time
wc -l items.ndjson

Big integers are still the quiet failure

Size and precision are independent problems, but large documents are the ones most likely to contain large ids. Any JavaScript-based parse rounds integers above 2^53-1 without telling you, and at scale that means duplicate keys where the source had distinct ones.

Use a lossless mode that keeps the original digits, or handle those fields as strings before they reach a JSON parser at all.

When the browser is the wrong tool

If the file is a database export, load it into the database. If it is an event log, it is almost certainly better as NDJSON. If you need repeated queries against it, put it somewhere that indexes. Browser tools are excellent up to the tens of megabytes and are not a substitute for a data pipeline above that.