# Keep your data fresh A dashboard is only worth its URL if the numbers on it are current. That is the part an agent session cannot leave behind: your agent builds the dashboard and goes away, and the data underneath has to keep arriving on a schedule, for months, without anyone in the loop. Freshness is therefore platform state, not dashboard code. You set it up once against a dataset; it keeps running whether or not anyone is holding a session open. ## The three ways data moves forward[​](#the-three-ways-data-moves-forward "Direct link to The three ways data moves forward") | | What it is | Who triggers it | | --------------------- | ------------------------------------------- | ---------------------------------------------------------------------- | | **On-demand refresh** | Re-pull one dataset now | You, your agent, or your CI — `POST /v1/datasets/{dataset_id}/refresh` | | **Incremental load** | Pull only what changed since the last run | The same call, with `mode: "incremental"` | | **Scheduled refresh** | A cron expression the platform runs for you | Nobody — the platform's cron poller | ## Refresh a dataset now[​](#refresh-a-dataset-now "Direct link to Refresh a dataset now") ``` result = client.datasets.refresh(dataset_id) print(result.completed, result.effective_mode, result.message) ``` ``` const result = await client.datasets.refresh(datasetId); console.log(result.completed, result.effectiveMode, result.message); ``` Two outcomes, and the difference matters: * **200 OK** — the work actually finished in the request. `completed` is `true` and `dataset` is the final row. This is what a public-dataset re-download does. * **202 Accepted** — the refresh was *started*. The row is locked and marked `PROCESSING`, and a connector runs seconds later. `completed` is `false`. Poll the dataset, or listen on the websocket, until `processingStatus` leaves `PROCESSING`. A `202` is not a failure and it is not a success — it is a promise. Treat "the call returned" and "the data is current" as two different facts, and check the second one before you tell someone the dashboard is up to date. The call that checks it is [`get_data_freshness`](#how-old-are-the-numbers) — everything in this guide up to that section is about making data move, not about how old it is. ## Incremental loads, and the run that isn't incremental yet[​](#incremental-loads-and-the-run-that-isnt-incremental-yet "Direct link to Incremental loads, and the run that isn't incremental yet") Pass a mode to pull only new rows: ``` from vibedasher.models import RefreshRequest result = client.datasets.refresh(dataset_id, RefreshRequest(mode="incremental")) print(result.requested_mode, "->", result.effective_mode) ``` `requestedMode` echoes what you asked for. **`effectiveMode` is what actually ran, and they are not always the same.** An incremental request against a dataset with no checkpoint yet reports `effectiveMode: "bootstrap"`: it runs as a full overwrite *and* records the watermark, so the **next** run is the first genuinely incremental one. Branch on `effectiveMode`, never on what you requested. (This is the same rule as `client.query`, which reports the execution lane it actually used rather than the one you asked for — a pattern worth internalising: the platform tells you what happened, and it is your job to read it.) To force the next run to fetch everything again, clear the watermark: ``` client.datasets.reset_checkpoint(dataset_id) ``` ## Scheduled refresh[​](#scheduled-refresh "Direct link to Scheduled refresh") A schedule is a row you create once. It carries a cron expression and the dbt project it refreshes: ``` from vibedasher.models import RefreshCreate schedule = client.refreshes.create_refresh( RefreshCreate(name="nightly", project_id=project_id, cron="0 3 * * *") ) print(schedule.refresh.next_run, schedule.schedule_registration.registered) ``` The response carries a `scheduleRegistration` block. **Check `registered`.** It is `true` only when the schedule is genuinely live; when it is `false`, `reason` and `message` say why, and `nextRun` is null. That field exists because of a real outage, and the story is the best argument for the current design. The original implementation registered each schedule as its own cloud rule, wrapped the call in a broad `except`, and answered `201 Created` with a confident `nextRun`. The API role had no permission to create those rules, so **nothing was ever registered** — and every customer was told their data would refresh. It was found on 2026‑09‑05: zero rules existed, and the one schedule in the database had a `nextRun` five months in the past and an execution count of zero. The fix was not better error handling. It was removing the thing that could fail: **the row *is* the schedule.** A cron poller runs every five minutes, claims each due row with a compare-and-set, and executes it. There is no external registration to succeed or fail, so the only thing that can go wrong at create time is a cron expression that cannot be evaluated — and that is now rejected up front, not at fire time. Two consequences worth planning around: * A schedule fires **within about five minutes** of its due time, not at the second. Cron here means "no earlier than", not "exactly at". * `scheduleRegistration.backend` reads `db_cron`. It is kept in the response because clients branch on it. ## Reading what happened[​](#reading-what-happened "Direct link to Reading what happened") Every schedule row carries its own history — `lastRun`, `lastStatus`, `lastError`, `executionCount`, `nextRun`: ``` for r in client.refreshes.iter_all(): print(r.name, r.last_status, r.last_run, r.execution_count, r.last_error) ``` `executionCount` is the field to watch. A schedule that has existed for a week with `executionCount: 0` is not slow — it is not running, and that is exactly the failure the design above was built to make visible instead of silent. ## How old are the numbers?[​](#how-old-are-the-numbers "Direct link to How old are the numbers?") Everything above answers *"is the pipeline working"*. It does not answer *"is what this dashboard shows current"*, and the two come apart in both directions: * A schedule can run perfectly and load nothing new. `executionCount` climbs, `lastStatus` is `success`, and the newest row is still from March. * **A dataset can have no schedule at all**, and then `executionCount` has no row to read. A static upload is the common case here: its rows change when somebody uploads new ones and at no other time, so a healthy schedule list can sit beside a dataset nine days stale and say nothing about it. Absence of a schedule is not a schedule with a bad count. One call answers the reader's actual question, measured from the rows rather than from the platform's own bookkeeping: ``` f = client.datasets.get_freshness(dataset_id).freshness print(f.as_of, f.age_days, f.status) ``` ``` const { freshness } = await client.datasets.getFreshness(datasetId); console.log(freshness.asOf, freshness.ageDays, freshness.status); ``` Over MCP it is `get_data_freshness`, which takes **exactly one** of `viz_id` or `dataset_id`. Given a `viz_id` it resolves every dataset the dashboard reads and reports the **stalest** — a dashboard is only as current as its oldest input. The REST route is `GET /v1/datasets/{dataset_id}/freshness`. ### The three fields, and why they are three[​](#the-three-fields-and-why-they-are-three "Direct link to The three fields, and why they are three") | Field | What it is | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `asOf` | The newest value in the data's **own time column**. This is the answer, and it cannot flatter you: it is read from the rows. `ageDays` is how old that is; negative means forward-dated data. | | `lastLoadedAt` | When rows last **landed**. A pipeline fact, not an age — a file loaded this morning can hold nothing newer than March. Never quote it as an as-of. | | `refresh` | Whether anything is scheduled to make the numbers newer, **including the answer "nothing, and nothing can be"** for a flat file. This is the field that covers the no-schedule case the section above describes. | `status` says how much to trust `asOf`: `MEASURED` (it is real), `NO_TIME_COLUMN` (the data carries no date column, so it has no as-of to have), or `UNKNOWN` — which means the fact was not established and is **never** a pass. `lastLoadedSource` names which clock answered `lastLoadedAt`. `iceberg_snapshot` is the physical write time of the table's current snapshot — the only true landing time, and one a rename cannot move. **`not_established` means `lastLoadedAt` is `null` on purpose**: that read path did not consult the snapshot, so rather than answer with the dataset row's `updatedAt` — a row write clock that any edit moves forward, and which once reported rows landing nine days after they did — the field declines. A field that declines is strictly better than one that answers with the wrong clock. The route above is the call that reads the real one. `get_viz` and `list_vizzes` carry a free `dataFreshness` that is definite about the refresh posture and `UNKNOWN` about the as-of; this call is what resolves that `UNKNOWN`. ### What is *not* the age of the data[​](#what-is-not-the-age-of-the-data "Direct link to what-is-not-the-age-of-the-data") None of `publishStatus`, `lastBuild.createdAt`, `processingStatus` or `updatedAt` answers this question. They are, in order: whether a bundle exists, when the **code** was compiled, whether the file parsed once, and when the **row** last changed. Each of them reads *current* on a dashboard whose numbers are weeks old. If you are about to report recency from any of them, call the freshness route instead. It costs one `SELECT MAX()` per dataset — the same probe the rendered dashboard already runs when it mounts. ## Where each gesture lives[​](#where-each-gesture-lives "Direct link to Where each gesture lives") * **Schedules are the MCP surface.** `create_refresh`, `list_refreshes`, `get_refresh` and `delete_refresh` cover the whole lifecycle of a recurring refresh from your agent. * **"Refresh it now" is the SDK and the REST API.** A one-off run is a single call there — see the **[API Reference](/reference/vibedasher-sdk-api.md)**. * **dbt projects** carry their own run and refresh routes (`/v1/dbt/projects/{project_id}/run`, `/v1/dbt/projects/{project_id}/refreshes`).