DuckDB for real-time dashboards: lessons from World Data Visualizer

World Data Visualizer is a real-time market intelligence dashboard I built between January and March 2026. It covers eleven economic sectors, tracks US Congress trading transactions, ingests RSS news with a small NLP pipeline, plots AIS oil-tanker positions on a world map, and computes a "Cortisol Gauge" market-stress indicator. The backend is Node.js. The database is DuckDB. There is no Postgres anywhere. Repo: github.com/aelmufti/world-data-visualizer.

The headline number is that aggregate queries over a few million rows complete in under 100 ms with no tuning. I want to explain why that is true, and where DuckDB is the wrong choice so you do not over-fit to my use case.

What DuckDB is, briefly

DuckDB is an in-process OLAP database. It ships as a single library you link into your application — the same architectural shape as SQLite. There is no server. The "database" is either a file on disk or pure memory. Queries are vectorized and column-oriented, which is the opposite of a row-store like Postgres.

For OLTP workloads — short transactions, many concurrent writers — Postgres remains the right answer. For OLAP workloads — long aggregate scans, few writers, many readers — DuckDB is faster by an order of magnitude on the same hardware, with one tenth the operational complexity.

Why this project is OLAP-shaped

The dashboard has three traffic patterns and they are all reads:

• Periodic ingestion. Every few minutes a worker pulls fresh data from upstream APIs (stocks, RSS feeds, Congress disclosures) and appends rows.
• Aggregate queries. The UI asks for the "sector-wide moving average over the past hour", which scans the relevant slice and aggregates.
• Real-time fan-out. When new data lands, the server recomputes the affected aggregates once and pushes the result to every connected client over WebSocket.

Writes are batched and infrequent. Reads are heavy, repetitive, and aggregate-shaped. There is no concurrent transactional contention to speak of. This is exactly DuckDB's strong suit.

What I gave up by skipping Postgres

Honesty matters more than evangelism. The tradeoffs were real:

• No concurrent writers. DuckDB allows a single writer process. For my topology that is fine — the ingestion worker is the only writer — but on a system with multiple ingestion paths I would need to coordinate or pick differently.
• No row-level locking or transactional updates. OLAP databases assume you mostly append. If your access pattern is "load row, mutate row, save row", DuckDB is going to be frustrating.
• Smaller ecosystem. Fewer extensions, fewer dashboards-as-a-service know how to connect, fewer Stack Overflow questions answered. The DuckDB community is great but it is not the size of Postgres.

WebSocket over polling

The original prototype polled every five seconds from the browser. With ~50 connected clients during a market session that was ~10 requests per second to recompute the same aggregates. Switching to a single recompute-on-write + WebSocket fan-out reduced backend load by roughly an order of magnitude and eliminated the staleness window between source updates and UI updates.

The implementation is conventional: ws on the server, a thin client wrapper on the React side, and a small retry/backoff state machine for reconnects. Nothing exotic.

Where I would draw the line

I do not think DuckDB is universally better than Postgres. I think it is dramatically better for a specific shape of workload and people consistently overestimate how often their workload falls outside that shape. Three concrete heuristics:

1. If you would happily run your dashboard off a nightly Parquet dump, DuckDB will do that in real time with the same query surface and almost no ops overhead.
2. If your hot path is "for this user, atomically update their cart and decrement inventory", reach for Postgres without thinking.
3. If you are running both shapes in one app, run both databases. Use DuckDB for the analytics surface and Postgres for the transactional surface. They cohabit fine.

The unsexy conclusion

The fastest database in production is usually the one whose access patterns match the workload, not the one with the most impressive benchmarks. For a read-heavy dashboard with batch ingestion DuckDB is, in my experience, the highest leverage-to-complexity choice on the market. World Data Visualizer is open source — clone it, point it at your own upstream feeds, and you will see the same numbers.