routerflow.artthe marketour probecomparemethodologyprobestaxonomyprober methodapi/v12026-11-10

API v2: the leaderboards as static JSON

/api/v2/ is built from the public leaderboard tables (docs/DATA.md, schema 1.2) by api_v2.py after every nightly export_tables.py. Static files, no server: the same files are served by Vercel at https://routerflow.art/api/v2/ and by the box next to the tables. /api/v1/ is untouched: v1 is the site's working dataset (routers, prices, probes, one snapshot a day); v2 is the boards, every public leaderboard we read, with the same column names as the tables.

Paths

PathWhat
index.jsonWhat exists: counts, path templates, every file with its size
boards.jsonCatalog of every public board: source, title, url, entity type, category, period kind, metrics with unit and direction, rank metric, depth, what it measures, license and citation, first and last period, rows, last collection
boards/<board_id>/latest.jsonWhat the board shows now, ranked: every entry at its newest period (board_current), newest period first. At most 500 rows (truncated, rows_total say when the cap hit; the whole board is in the Parquet and CSV tables)
boards/<board_id>/series.jsonThe top 25 entities of the board over time: one series per entity, split (provider, effort, agent, variant, hardware) and metric, as [date, value] points, oldest first
models.json, models/<model_id>.jsonEvery model on at least one public board; one model on every board it is on, with rank, value, unit, period and movement
providers.json, providers/<id>.jsonThe same for providers, plus serving: the price, speed and uptime boards where the provider serves the ranked models
labs.json, labs/<id>.json, apps.json, apps/<id>.json, gateways.json, gateways/<id>.jsonThe same for labs, apps (products, never people) and gateways
sources.jsonEvery source, public or not: operator, license and its URL, redistribution (yes, attribution, unclear, no), the citation line it asks for, cadence, what it keeps upstream. A private source is listed but has no board ids or paths
overview.json, tables.jsonThe data release on one page (every table and board, private boards by id with public: false and no counts) and the data dictionary of the public tables, both as export_tables.py writes them
schema/<kind>.schema.jsonJSON Schema (draft 2020-12) of every file kind; column names, types and descriptions come from tables_schema.py
openapi.yamlThe static paths as an OpenAPI 3.1 document, responses pointing at the schemas

Board ids keep their dots as directory names (boards/vercel.leaderboard.models.day/latest.json). Model ids hold a slash and map to a nested path: models/z-ai/glm-5.3-flash.json. Every path is [A-Za-z0-9._/-] only.

The envelope

Every JSON file is

{"schema_version": "1.2.0", "generated": "2026-09-22T02:30Z", "license": "CC BY 4.0",
 "attribution": "routerflow.art; per-source citations in sources.json", "data": ...,
 "citation": "Source: OpenRouter (openrouter.ai/rankings), as of 2026-09-21T23:39:31.558Z. Licensed under CC BY 4.0."}

generated is the stamp of the tables the file was built from (tables.json), so a rebuild over unchanged tables gives byte-identical files. citation is present on a board's files, whose rows all come from one source; an entity file holds citations by source id instead. data is an array on the list files and an object on the rest. Keys are sorted; a row column whose value is null is left out of latest.json rows and of entity placements (the schema says which columns are always present).

Examples

curl -s https://routerflow.art/api/v2/boards.json | jq '.data[] | select(.source_id == "vercel") | .id'
curl -s https://routerflow.art/api/v2/boards/vercel.leaderboard.models.day/latest.json | jq '.data.rows[:5] | .[] | [.rank, .entity_raw, .metric, .value, .unit]'
curl -s https://routerflow.art/api/v2/boards/lmarena.text.overall/series.json | jq '.data.series[0] | {entity_raw, metric, points: .points[-3:]}'
curl -s https://routerflow.art/api/v2/models/z-ai/glm-5.3-flash.json | jq '.data.boards[] | [.board_id, .metric, .rank, .movement]'
curl -s https://routerflow.art/api/v2/providers/deepinfra.json | jq '.data.serving[:3]'
curl -s https://routerflow.art/api/v2/sources.json | jq '.data[] | select(.public) | {id, license, citation}'
curl -s https://routerflow.art/api/v2/schema/board_latest.schema.json | jq '."$defs".entry.properties.value'

Ranks and movement

Ranks are the boards' own: rank is upstream's position where upstream prints or implies one, else assigned by value within board, period and metric with ties sharing the better rank; residual rows (Other) and unranked values have none. An entity file shows each board at the entity's newest period there, the best-ranked of its rows (efforts, agents, providers, variants; entries counts them), the same choice as the model_everywhere view. previous_period_end is the board's period before that one for the metric, previous_rank the entity's best rank there, and movement = previous_rank - rank: positive means it climbed. Both are null on boards without periods (period = none), at a board's first period, or where the entity was not on the previous period.

series.json chooses its entities from what the board shows now: those at the newest period first, then by rank on the rank metric (the board's own ordering, else its first metric), then by value in the metric's direction; residual rows are never chosen. Points are one per date: period_end on dated boards, snapshot (the day a value was first seen) on boards without periods.

Snapshots

The tables keep one row per key for each run of equal values: snapshot (= first_seen) is the day a value was first collected and last_seen the newest day it was collected unchanged, so a source that republishes the same history nightly costs one row per value. Where upstream revises a past period (Vercel does), the API shows the value as last revised, the newest snapshot per key.

v1: the probe and the studies

Three files carry what we measured ourselves. They are rebuilt from the prober's records about every three hours, are not enveloped like v2, and have no history: each is the whole series so far.

PathWhat it holds
/api/v1/probe/latest.jsonOur probe tab: cache pairs, prices, routing, quality per endpoint; the taxonomy of every probe; studies with one headline per study.
/api/v1/probe/time-of-day.jsonThe time-of-day study. Per arm: all, by_hour_pt (keys 0-23, Pacific), by_day, by_item, blocks (sf_workday, small_hours_et), night_minus_day with its 95% interval, served_by_hour_pt for the routed arm, and the measured profile. Each block has n, passed, pass with a Wilson interval, reasoning and completion tokens p50, time to first token p50, tokens per second p50, truncated, usd. Plus design and taxonomy (this probe and every other on the six dimensions).
/api/v1/probe/task-cost.jsonThe task-cost study. Per arm: all and by_day (tasks, passed, passed_first, usd, usd_per_passed, usd_p50, tokens_p50, wall_s_p50), every task in tasks (time, dollars, attempts, tokens by kind, wall-clock, providers that served it, the first failing case), first_failures, the quote and the model's default effort. Plus design (with the spec as sent) and taxonomy.

No response text and no keys are published. Methodology: Studies.

License and citation

This aggregation is CC BY 4.0. Every row keeps its upstream license: sources.json gives each source's license, its URL and the citation line it asks for, and boards.json gives the license that applies to a board where it differs from its source. Rows are published only from sources whose redistribution is yes or attribution; sources marked unclear or no are listed in sources.json and overview.json as existing and carry no data. OpenRouter's Data API asks for its line verbatim with the as_of stamp, which every file that holds its rows carries filled in; MLPerf's line keeps its <round>, <benchmark> and <scenario> placeholders, to be filled from the row's extra. When you publish a number from these files, publish the citation line of its source with it.

Building

python3 api_v2.py [--tables data/tables] [--out site/api/v2] reads routerflow.duckdb, board_entry_latest.csv.gz, overview.json and tables.json from the public tables directory, never the private one, and writes the tree atomically (about 2,000 files, 35 MB, a few seconds, under 300 MB of memory). Files of an earlier build that the new one does not write are removed. tests/test_api_v2.py builds the API over the fixture tables and validates every file against its schema, checks that no private board or source reaches a data file, that model files agree with model_everywhere, that movement is what it says, and that a second build is byte-identical; the same checks run over site/api/v2/ when a full build is present.