Keep your data fresh
A dashboard is only worth its URL if the numbers on it are current. That is the part an agent session cannot leave behind: your agent builds the dashboard and goes away, and the data underneath has to keep arriving on a schedule, for months, without anyone in the loop.
Freshness is therefore platform state, not dashboard code. You set it up once against a dataset; it keeps running whether or not anyone is holding a session open.
The three ways data moves forward
| What it is | Who triggers it | |
|---|---|---|
| On-demand refresh | Re-pull one dataset now | You, your agent, or your CI — POST /v1/datasets/{dataset_id}/refresh |
| Incremental load | Pull only what changed since the last run | The same call, with mode: "incremental" |
| Scheduled refresh | A cron expression the platform runs for you | Nobody — the platform's cron poller |
Refresh a dataset now
result = client.datasets.refresh(dataset_id)
print(result.completed, result.effective_mode, result.message)
const result = await client.datasets.refresh(datasetId);
console.log(result.completed, result.effectiveMode, result.message);
Two outcomes, and the difference matters:
- 200 OK — the work actually finished in the request.
completedistrueanddatasetis the final row. This is what a public-dataset re-download does. - 202 Accepted — the refresh was started. The row is locked and marked
PROCESSING, and a connector runs seconds later.completedisfalse. Poll the dataset, or listen on the websocket, untilprocessingStatusleavesPROCESSING.
A 202 is not a failure and it is not a success — it is a promise. Treat "the call
returned" and "the data is current" as two different facts, and check the second one before
you tell someone the dashboard is up to date. The call that checks it is
get_data_freshness — everything in this guide up to that
section is about making data move, not about how old it is.
Incremental loads, and the run that isn't incremental yet
Pass a mode to pull only new rows:
from vibedasher.models import RefreshRequest
result = client.datasets.refresh(dataset_id, RefreshRequest(mode="incremental"))
print(result.requested_mode, "->", result.effective_mode)
requestedMode echoes what you asked for. effectiveMode is what actually ran, and they
are not always the same. An incremental request against a dataset with no checkpoint yet
reports effectiveMode: "bootstrap": it runs as a full overwrite and records the
watermark, so the next run is the first genuinely incremental one.
Branch on effectiveMode, never on what you requested. (This is the same rule as
client.query, which reports the execution lane it actually used rather than the one you
asked for — a pattern worth internalising: the platform tells you what happened, and it is
your job to read it.)
To force the next run to fetch everything again, clear the watermark:
client.datasets.reset_checkpoint(dataset_id)
Scheduled refresh
A schedule is a row you create once. It carries a cron expression and the dbt project it refreshes:
from vibedasher.models import RefreshCreate
schedule = client.refreshes.create_refresh(
RefreshCreate(name="nightly", project_id=project_id, cron="0 3 * * *")
)
print(schedule.refresh.next_run, schedule.schedule_registration.registered)
The response carries a scheduleRegistration block. Check registered. It is true
only when the schedule is genuinely live; when it is false, reason and message say
why, and nextRun is null.
That field exists because of a real outage, and the story is the best argument for the
current design. The original implementation registered each schedule as its own cloud
rule, wrapped the call in a broad except, and answered 201 Created with a confident
nextRun. The API role had no permission to create those rules, so nothing was ever
registered — and every customer was told their data would refresh. It was found on
2026‑09‑05: zero rules existed, and the one schedule in the database had a nextRun five
months in the past and an execution count of zero.
The fix was not better error handling. It was removing the thing that could fail: the row is the schedule. A cron poller runs every five minutes, claims each due row with a compare-and-set, and executes it. There is no external registration to succeed or fail, so the only thing that can go wrong at create time is a cron expression that cannot be evaluated — and that is now rejected up front, not at fire time.
Two consequences worth planning around:
- A schedule fires within about five minutes of its due time, not at the second. Cron here means "no earlier than", not "exactly at".
scheduleRegistration.backendreadsdb_cron. It is kept in the response because clients branch on it.
Reading what happened
Every schedule row carries its own history — lastRun, lastStatus, lastError,
executionCount, nextRun:
for r in client.refreshes.iter_all():
print(r.name, r.last_status, r.last_run, r.execution_count, r.last_error)
executionCount is the field to watch. A schedule that has existed for a week with
executionCount: 0 is not slow — it is not running, and that is exactly the failure the
design above was built to make visible instead of silent.
How old are the numbers?
Everything above answers "is the pipeline working". It does not answer "is what this dashboard shows current", and the two come apart in both directions:
- A schedule can run perfectly and load nothing new.
executionCountclimbs,lastStatusissuccess, and the newest row is still from March. - A dataset can have no schedule at all, and then
executionCounthas no row to read. A static upload is the common case here: its rows change when somebody uploads new ones and at no other time, so a healthy schedule list can sit beside a dataset nine days stale and say nothing about it. Absence of a schedule is not a schedule with a bad count.
One call answers the reader's actual question, measured from the rows rather than from the platform's own bookkeeping:
f = client.datasets.get_freshness(dataset_id).freshness
print(f.as_of, f.age_days, f.status)
const { freshness } = await client.datasets.getFreshness(datasetId);
console.log(freshness.asOf, freshness.ageDays, freshness.status);
Over MCP it is get_data_freshness, which takes exactly one of viz_id or
dataset_id. Given a viz_id it resolves every dataset the dashboard reads and reports the
stalest — a dashboard is only as current as its oldest input. The REST route is
GET /v1/datasets/{dataset_id}/freshness.
The three fields, and why they are three
| Field | What it is |
|---|---|
asOf | The newest value in the data's own time column. This is the answer, and it cannot flatter you: it is read from the rows. ageDays is how old that is; negative means forward-dated data. |
lastLoadedAt | When rows last landed. A pipeline fact, not an age — a file loaded this morning can hold nothing newer than March. Never quote it as an as-of. |
refresh | Whether anything is scheduled to make the numbers newer, including the answer "nothing, and nothing can be" for a flat file. This is the field that covers the no-schedule case the section above describes. |
status says how much to trust asOf: MEASURED (it is real), NO_TIME_COLUMN (the data
carries no date column, so it has no as-of to have), or UNKNOWN — which means the fact was
not established and is never a pass.
lastLoadedSource names which clock answered lastLoadedAt. iceberg_snapshot is the
physical write time of the table's current snapshot — the only true landing time, and one a
rename cannot move. not_established means lastLoadedAt is null on purpose: that
read path did not consult the snapshot, so rather than answer with the dataset row's
updatedAt — a row write clock that any edit moves forward, and which once reported rows
landing nine days after they did — the field declines. A field that declines is strictly
better than one that answers with the wrong clock. The route above is the call that reads
the real one. get_viz and list_vizzes carry a free dataFreshness that is definite about
the refresh posture and UNKNOWN about the as-of; this call is what resolves that UNKNOWN.
What is not the age of the data
None of publishStatus, lastBuild.createdAt, processingStatus or updatedAt answers
this question. They are, in order: whether a bundle exists, when the code was compiled,
whether the file parsed once, and when the row last changed. Each of them reads current
on a dashboard whose numbers are weeks old. If you are about to report recency from any of
them, call the freshness route instead.
It costs one SELECT MAX(<date column>) per dataset — the same probe the rendered dashboard
already runs when it mounts.
Where each gesture lives
- Schedules are the MCP surface.
create_refresh,list_refreshes,get_refreshanddelete_refreshcover the whole lifecycle of a recurring refresh from your agent. - "Refresh it now" is the SDK and the REST API. A one-off run is a single call there — see the API Reference.
- dbt projects carry their own run and refresh routes
(
/v1/dbt/projects/{project_id}/run,/v1/dbt/projects/{project_id}/refreshes).