Passa al contenuto principale

Get Dataset Freshness

GET 
/v1/datasets/:dataset_id/freshness

HOW OLD ARE THESE NUMBERS — the newest value in the data's own time column.

The one call that MEASURES a reader's first question: establishing the newest row means reading the rows, and the core / viz lambdas ship no duckdb or pyiceberg on purpose.

It is not the only call that can ANSWER it, and the difference matters to whoever is deciding whether to spend this query. This route writes what it read onto the dataset (DatasetConfig.freshness_probe), and so does the ingestion funnel on every transition to READY. So GET /v1/viz/{id} and GET /v1/vizs answer MEASURED about a dataset that carries one of those stored readings, UNKNOWN about one that does not, and which it is depends on what has already happened to the dataset rather than on anything in the request. Those cheap answers report asOfSource: probed_column_max beside asOfMeasuredAt, the moment the reading was taken — read that gap before re-spending this query, and read it rather than assuming a MEASURED verdict was taken for you. This route always takes a new reading and reports column_max.

Runs one SELECT MAX(<each date column>) over the dataset — the same shape the generated dashboard already runs on mount to size its date picker — and returns asOf (the age of the DATA) beside lastLoadedAt (when rows landed, a PIPELINE fact) and refresh (whether anything is scheduled to make them newer, including the answer "nothing, and nothing can be").

lastLoadedAt is THE TABLE'S OWN WRITE TIMESTAMP or null — read lastLoadedSource to tell which. It is never filled in from the dataset row's updatedAt: that is when the ROW was written, any edit moves it, and it moves forward, so under a key meaning "when rows last landed" it read flattering rather than merely weak. This route is the only one that can establish the fact, and it declines when the table has no snapshot to read.

status is one of:

  • MEASURED — asOf is a real value read from the rows.
  • NO_TIME_COLUMN — the data has no date column, so it has no as-of to have.
  • UNKNOWN — not established (the dataset is not queryable, or the probe failed; meaning says which). Never read UNKNOWN as current.

Costs one query against the dataset. It is deliberately NOT folded into GET /v1/datasets/{id}, which is a metadata read and must stay cheap.

A SIDE EFFECT, stated because it changes what later calls answer: the reading is written onto the dataset for the lambdas that cannot take one, which is how a published dashboard's footer can state the age of its own numbers. It is a cache write and not a row write — updatedAt does not move, because measuring something is not changing it.

Request​

Responses​

Successful Response