routerflow.artthe marketour probecomparemethodologyprobestaxonomyprober methodapi/v12026-11-10

Methodology

Every number on routerflow.art traces to a source, a window, a formula, and a confidence tier. This page is the single definition. If the site and this page disagree, this page wins and the site has a bug.

Confidence tiers. Measured: we sent the request and recorded the result (the prober, live since 2026-09-19; the "Our probe" tab). Reported: a router or provider published it about its own traffic. Derived: computed by us from reported figures with a stated formula. Estimate: extrapolated; labeled as such.

Windows. Each source has its own. Nothing is summed across routers because each router is a different customer sample.

1. Sources

idEndpointFields usedWindow / retentionAuthLicense
openrouter rankingshttps://openrouter.ai/rankings (react-query state embedded in self.__next_f.push chunks)models/week: model_permaslug, variant, total_prompt_tokens, total_completion_tokens, count; apps day/week/month: app.title, app.categories, total_tokens, total_requeststrailing week ending the last complete daynoneCC BY 4.0 (OpenRouter rankings license)
openrouter providershttps://openrouter.ai/providers (embedded)name, slug, modelCount, dailyTokens, monthlyTokens per providerrolling day / 30 daysnonepage data
openrouter modelshttps://openrouter.ai/api/v1/modelsid, canonical_slug, name, hugging_face_id, pricingcurrentnonepublic API
openrouter endpointshttps://openrouter.ai/<model> page objects and /api/frontend/v1/stats/endpoint?permaslug=&variant=provider_name, provider_slug, quantization, pricing{prompt, completion, input_cache_read, discount}, is_deranked, is_disabled, variantcurrentnoneundocumented; paths credit pacell/openrouter-stats
openrouter effective pricing/api/frontend/v1/stats/effective-pricing?permaslug=&variant=&shape=v7&range=1mproviderSummaries[]{endpointId, providerName, providerSlug, effectiveInputPrice, effectiveOutputPrice, cacheHitRate, totalTokens}, weightedInputPrice, weightedOutputPrice, weightedCacheHitRate1 monthnoneundocumented
openrouter daily perf/stats/cache-hit-rate-comparison, /stats/throughput-comparison?percentile=p50, /stats/latency-comparison?percentile=p50, /stats/uptime-recentper endpoint id per day~8 days retained upstreamnoneundocumented
openrouter activity/stats/model-activityper day: prompt, completion, cached, reasoning tokens, requests, tool calls31 daysnoneundocumented
openrouter top apps/stats/top-apps-for-modeltop 5 apps: title, categories, total_tokens, total_requestsundocumented window (longer than a week)noneundocumented
openrouter provider charts/stats/provider-token-chart?provider=<slug>per day: tokens per model (top models + Others)90 daysnoneundocumented
openrouter TEE pageshttps://openrouter.ai/provider/{phala,venice,chutes,near-ai}model slugs servedcurrentnonepage
vercelhttps://vercel.com/api/ai/leaderboard-export?dataset={models,labs,apps,providers}&modality=all&format=jsondate, group, name, metric{tokens,requests,spend,...}, share_percentdaily since 2026-07-19noneCC BY 4.0
requestyhttps://www.requesty.ai/data/<slug>/data.json (index at /data/llms.txt)schema.org Dataset columns, rows, caveatsper note (April 2026 vintage)noneCC BY 4.0
kilohttps://kilo.ai/leaderboard/race (embedded rows)weekStart, provider, isOpenWeights, tokens63 weeksnonepage data
opencodehttps://opencode.ai/data (SolidStart payload)leaderboard {model, provider, author, tokens}, author {share}, geo {country, share}site's window, units undocumented; shares onlynonepage data
chuteshttps://api.chutes.ai/invocations/usage, /chutes/?limit=200&page=nchute_id, date, usd_amount, invocation_count; chute names~11 daysnonepublic API
trustedrouterhttps://trustedrouter.com/leaderboard (rows with data-* attrs and <dl> details), https://api.trustedrouter.com/v1/models (trustedrouter.endpoints[]), https://trustedrouter.com/providersprovider: rank, p50/p95 TTFT, completion, availability, within deadline, capacity accepted, throughput, samples, failure owner; route: privacy_tier 0-3, zdr, confidential, e2ee, HQ country; provider labels: verified confidential, ZDRleaderboard rolling sample ≤10,000; catalog currentnonepublic pages
aihordehttps://aihorde.net/api/v2/stats/text/{totals,models}requests/tokens per minute/hour/day/month/total; per-model tokenslivenonepublic API
tokscalehttps://tokscale.ai/api/leaderboardstats{totalTokens,totalCost,uniqueUsers}, top userslifetimenonepublic API
ollamahttps://ollama.com/library?sort=popularmodel, pullscumulativenonepage
huggingfacehttps://huggingface.co/api/models?pipeline_tag=text-generation&sort=downloadsid, downloadstrailing 30 daysnonepublic API
darkbloomhttps://api.darkbloom.dev/v1/providers/attestationproviders: status, trust_level, chip, memory_gb, modelscurrentnonepublic API
darkbloom x@gajesh "Darkbloom: Day N" posts via Apify (control machine only)tokens/day, cumulative, ARR, machines, uptimeper postApify tokenself-reported
registrygithub.com/amiller/awesome-private-inference data/quality.json + latest snapshotper provider endpoint: status, layers proven/required, missingdailynoneMIT/CC-BY
reportsNBER w34255 (pdf → text), Anthropic Economic Index March 2026fixed figures quoted in build_dataset.pyas publishednonecited

2. Definitions

Lab. The model author, canonicalized in LAB (build_dataset.py): z-ai/zai/zhipu → Z.ai, moonshotai → Moonshot, qwen → Alibaba, etc. Labs outside the eight palette slots fold into "Other labs" on charts only; the data keeps them.

Open weights. OpenRouter: the catalog entry has a hugging_face_id (reported). Kilo: the source's isOpenWeights (reported). Vercel and OpenCode: by lab membership in OPEN_LABS (derived).

Serving provider tier. tee if the provider slug is one of phala, venice, chutes, near-ai, tinfoil, redpill (attested confidential compute offered), else plain. Venice also runs non-TEE endpoints, so tee is an upper bound for Venice. TrustedRouter's verified_confidential label is a stricter, independently checked flag and is shown separately.

Attested share (OpenRouter). Two variants, both derived. Tile: sum(dailyTokens of tee providers) / sum(dailyTokens of all providers) from the providers page. Series: same ratio from provider-token-chart per day, which covers each provider's top models plus "Others" and sums to about a quarter of the providers-page total; used for trend, not level. Both are lower bounds on private inference overall because OpenRouter excludes private endpoints and zero-retention traffic from public stats.

HHI. 10000 × Σ share_i² over all 88 serving providers by daily tokens (derived). Series version from provider charts. DOJ thresholds: under 1,500 unconcentrated, 1,500 to 2,500 moderate.

Effective price (per endpoint). OpenRouter-reported effectiveInputPrice / effectiveOutputPrice: what customers paid per million tokens after cache reads and discounts over the last month (reported; cached-token counts underlying it are asserted by the provider). Weighted model figures are OpenRouter's token-weighted averages.

List price. The endpoint's posted pricing.prompt / pricing.completion in $/token, shown as $/M (reported). discount is the provider-declared cut versus list.

Cache hit (1m / today). Reported by OpenRouter per endpoint: monthly cacheHitRate from effective pricing; daily from cache-hit-rate-comparison. Provider-asserted underneath.

Undercut. min(list prompt price among paid standard endpoints) / (lab's own endpoint list prompt price) − 1 (derived). The lab's endpoint is matched by provider slug prefix (LAB_PROVIDER_SLUGS).

Cheapest quote ≠ cheapest bill. Count of boards where the provider with the lowest list input price is not the provider with the lowest effective input price (derived).

Model × provider matrix. Latest-day tokens per model per provider from all 88 provider charts, base slug with the date suffix stripped; Others rows dropped (reported).

App → model links. Each model's top-5 apps (reported, longer window) used as a mix, scaled to the model's latest-day tokens; remainder "Other apps". A mix, not a daily joint distribution.

Chutes TEE share. usd on chutes whose name ends in -TEE / all metered usd, per day (derived from reported). unlisted = chute ids absent from the public catalog.

TrustedRouter availability / TTFT. As published: availability excludes failures owned by the router or the customer; TTFT p50 across all of a provider's models in the rolling sample; providers need 30 samples with 3 measured first tokens to rank. Joined to OpenRouter providers by slug map (z-ai→zai, gmicloud→gmi, cloudflare→cloudflare-workers-ai, moonshotai→kimi, xai→grok, nvidia→nvidia-nim).

Privacy tier (TrustedRouter). Per route: 0 Standard, 1 No-store, 2 Zero retention, 3 Confidential + E2EE (reported by TrustedRouter).

The three tabs

The site is split by who produced the number, so the two are never confused.

The market tab: one section per layer

The market tab follows the supply chain: an overview of the flow, then Apps, Routers, Models, Providers and Hardware, then Public and private, which cuts across them. Every section has the same parts: a question, up to three key numbers of which one states coverage, one share figure, a side-by-side table of the layer's members, and a source line giving source, unit, window and date. The reasoning and the sites and guidance it draws on are in the design brief.

Routers in the flow diagram. The diagram reads apps, routers, models, providers, attested or not. OpenRouter is the only router that publishes token counts, so its branch is data. Vercel AI Gateway publishes each model's share of tokens and no total, so its branch is an estimate: an assumed total times the published shares. It is drawn hatched, without the moving particles, ends in "providers not published", and is marked est. wherever it appears; the mark opens a card with the assumption, its range and its sources. Requesty and TrustedRouter publish nothing that can be scaled and are compared in the Routers section only. Kilo Code and OpenCode also publish figures, but they are coding agents reporting their own usage across whatever router or key their users chose. They are apps, their figures overlap with OpenRouter's count of the same apps, and they are shown in the Apps section and never added to anything. An earlier version drew Kilo as a second router; that was wrong and was removed on 2026-09-19.

Three routers, collected fully. OpenRouter (token counts for apps, models and providers), Vercel AI Gateway (daily shares by model and lab for tokens, requests and spend, and two top-10 rankings of apps and providers, by tokens and by spend) and TrustedRouter (its own measurements of providers). Vercel follows the window control: a window is the mean of its daily shares, a model missing from a day's list counting as zero that day, because Vercel publishes no totals to weight days by. Its rankings appear as "rank on Vercel" columns in the Apps and Providers tables, matched by name; [low] means not in its top 10. Apps that only Vercel lists are added to the Apps table. Requesty's topic notes are no longer shown: they were five months old and carry no totals.

Vercel's provider layer, and the same endpoint quoted twice. Vercel's public API lists, for every language model, the providers that serve it with their listed prices, uptime, latency and throughput. It is collected nightly without a key. Where OpenRouter's top 20 models are served by the same provider on both routers, the Routers section sets the two listed input prices side by side. OpenRouter's side is the provider's plain endpoint where it has one, then a quantization or region variant, and a flex, batch or priority tier only if nothing else exists, because a service tier is a different product. These are quotes, not bills, and a difference can come from a promotion, a different quantization or a different contract behind the same provider name.

The router registry. Every router and gateway we know of is listed in a public repository, routerflow-registry, one file each. It is a registry and nothing more: who the router is, its API, and where its public data lives. No figures, statements or estimates; those live here with their sources. The "Add your router" button opens a pull request there. A router listed by name only shows [x] until somebody checks it. What is built and what is planned for scanning entries is in the registry plan.

Assumptions. A number that is not data lives in one file, data/curated/assumptions.json, with the value used, a low and a high, how the range was made, a caveat, the sources with verbatim quotes and whether we read each at the original, and the date it was last checked. The value used is the halfway point of the range. The dataset build attaches each assumption where it is used and publishes them all at api/v1/assumptions.json; the assumptions page is generated from the same file. Each one is re-checked every 30 days: the build prints a warning when one is overdue, the card says so, and the hourly publish job sends one notice. An estimate supports an order of magnitude and nothing finer.

Missing values. A cell is never blank and never a false zero. [x] means the source does not publish it. [z] means it does not apply, for example a provider that is not on OpenRouter has no OpenRouter share. [low] means the model is below that router's published top list.

No totals across routers. OpenRouter publishes token counts, Vercel publishes shares, Requesty publishes share of spend, TrustedRouter publishes no volume. The Routers table says what each publishes and sums nothing.

Windows. One control sets 1, 7 or 30 complete UTC days. It drives the flow diagram, the overview counts, Models, Providers and Public and private. Every windowed volume is tokens per day, the mean over the window, so windows and routers sit on one scale. Shares, the top-3 share and the Herfindahl index are pooled over the window: tokens are summed first and shares taken after, not averaged day by day. Sections that cannot follow the control say so in their header: Apps is fixed at the one week OpenRouter ranks, Routers keeps each router's own window, Hardware is today's catalog. Prices stay 30-day figures in every window, and the 7 and 30 day flow keeps the top 60 models and pools the rest.

Complete days only. The collector's current UTC day is partial, so every daily series and every key number stops at the last complete day.

Derived columns in the Providers table.

Hardware. No source publishes which chip served a request. The section shows declared quantization per endpoint, endpoints that run in a TEE, and Darkbloom's chip list. Three providers (Cerebras, Groq, SambaNova) state on their own sites that they serve on silicon they design; the table links each statement and nobody here has verified it.

Pins and links. Up to four rows can be pinned to the top of a table. The open row, the pins and the section are kept in the URL, so a view can be shared.

One system on every tab

Our probe, Compare, and Taxonomy and method use the same parts as the market tab: a rail of numbered sections, a Method toggle per section, key numbers, one figure, and side-by-side tables that sort, filter, pin up to four rows and open a row for detail. The tab, section, open row and pins are kept in the URL. In Our probe, the Cache section pools each provider over the models we probed (medians across its endpoints, with the number of pairs as n), and the Routing section has one row per model with counts, not rates, because n per cell is small. Compare has two views of the same join: every provider and model pair, and each provider pooled as an unweighted mean over its models. Taxonomy and method lists every probe on the six dimensions; opening a row shows what the probe sends.

The probe through three routers

OpenRouter is the reference: every probe job runs through it. The cache pair alone is also sent once a day through TrustedRouter and Vercel AI Gateway, the same bytes, pinned to a provider with that router's own mechanism, on the panel models each also serves, and on the providers OpenRouter also routes to first so the same provider can be read through two routers. Each router's records are summarized on their own and never pooled with OpenRouter's. TrustedRouter's catalog prices the model and not each endpoint, so its quotes are model-level. Where a router does not report the bill for a request, that request is left out of every cost ratio. A provider that recognizes our repeat through one router and not another is being reached differently (another region, another account, another pool); it is not evidence about the provider alone.

Measured quantities (Our probe)

Studies

Two questions get their own pages instead of a section in Our probe: time of day and task cost. Both are measured by us through OpenRouter with the same key, ledger and records as every other probe, but their records are kept out of the Our probe figures (a study's pinned Claude and GPT endpoints are not panel endpoints) and summarized on their own (studies.py in the prober; api/v1/probe/time-of-day.json and api/v1/probe/task-cost.json).

Where they sit in the taxonomy. Each is a probe on the same six dimensions as every other, shown on its page among all of them. The time-of-day puzzle is a One-shot, exact answer probe: about 1,200 tokens in, no reuse, one request, an answer a script checks exactly, and (unlike the other exact probes) most of the output is reasoning. The coding task is its own task type, Coding task, graded by running: about 500 tokens of spec, one request plus at most one repair turn that repeats the first, a medium-entropy answer (many correct programs, one behaviour) checked by running it, and output almost entirely reasoning plus code. Both are short-horizon in the taxonomy's sense; neither touches the cache question.

Compare: reported cache-hit rate against measured recognition

Reported is OpenRouter's 30-day cache-hit rate for that provider and model (cache_hit_1m): the share of all customers' prompt tokens the provider reported as cached, which depends on what customers send as much as on the provider. Measured is our recognition figure above. They are different quantities and need not be equal. A provider that reports a high rate overall and recognized none of our immediate repeats is worth a second look, not a conclusion: a repeat can land on another replica, caches can be keyed to a customer or written late, and most pairs are few.

Weighting: tokens or dollars

Every provider share on the site (attested share, provider shares, HHI, top-3) exists in two weightings, switchable on the charts.

The data as tables

Everything collected and measured is also published as tidy tables: one Parquet file per table and one DuckDB file, rebuilt every hour on the collection server and served at /data/tables/ with range requests on, so DuckDB and pandas can read them straight off the URL with no key. The probe table has one row per request we sent, with counts, bills, timings and hashes, and no prompt or response text. Tables, columns and example queries: the data as tables.

Five tables are built from the nightly raw files, not from the site dataset, so they carry everything collected and not only what the site draws.

TableSourceTier
vercel_leaderboard_dailyVercel leaderboard-export?dataset={models,labs}, five metrics, share_percent / 100, names as publishedreported, CC BY 4.0
openrouter_endpoint_statsOpenRouter /stats/endpoint and /stats/effective-pricing (one month), every catalog modelreported, undocumented, keyless
openrouter_endpoint_daily/stats/cache-hit-rate-comparison, /throughput-comparison, /latency-comparison (p50), /uptime-recent; percent values divided by 100reported, undocumented, keyless
openrouter_model_daily/stats/model-activityreported, undocumented, keyless
openrouter_model_app/stats/top-apps-for-model; window undocumentedreported, undocumented, keyless

Leaderboards: one long table

Every public model leaderboard we read lands in one long table, board_entry: one row per board, period, entity and metric, for each value it has had. Forty sources (usage boards from routers and apps, price catalogs, status pages, benchmark and arena boards) are read nightly by boards/ after the existing collectors; each source has a pure parser over its saved raw bytes, a fixture and a test, and declares its license. Rows are collected nightly and most sources republish their whole history each night, so board_entry keeps one row per key for each run of equal values: snapshot is the day a value was first collected, last_seen the newest day it was collected unchanged, and a key gets a new row only when its value, interval, rank, count or cost changes; board_latest is the newest row per key.

Names. Every row keeps the name exactly as published (entity_raw); entity_id and provider_id are filled at table-build time from curated alias files (data/curated/aliases/) and are null when no mapping is certain, so a corrected alias applies to all history. Canonical model ids are OpenRouter slugs where OpenRouter ever listed the model, else the lowercased Hugging Face repo id, else lab/name; effort, agent scaffold, free or batch tier, quantization and checkpoint dates go to variant, never into the id; *-latest pointers are never ids. Mappings are exact matches, written rules over a source's own crosswalk, or a person's judgement; nothing is matched on string similarity, and uncertain names wait in data/curated/proposed/REVIEW.md. Families (Ollama's llama3.1, one name spanning many sizes) resolve to a family id and never to one model.

Public and private. A board is published when its redistribution (its own, else its source's) is yes or attribution under a stated license; unclear and no boards are collected on the same schedule but built into a private directory that is never served, and source, board and board_coverage list every board with a public flag. The public tables carry each source's license and citation line; OpenRouter's Data API line is emitted verbatim with as_of.

Archived history. Where a board publishes no history (Kagi, MathArena, EQ-Bench), earlier states come from the Internet Archive: at most one capture a day, replayed through the same parser. Those rows are the site as the archive saw it that day; the snapshot is the capture time, not a night of ours; gaps are the days nobody archived the page, and a data file or table is only there for days the archive fetched it too.

API v2. /api/v2/ is static JSON built from the public tables only: the catalog (boards.json, sources.json), each board's newest period (boards/<id>/latest.json) and top-25 series, and one file per model, lab, provider, app and gateway listing every public board it appears on with rank, value and movement (previous_rank - rank at the board's previous period). Every file carries {schema_version, generated, license, attribution, data} and validates against schema/*.schema.json. /api/v1/ is untouched. Guide: the API.

The data tab. Everything the site holds, counted from the files when the tables are built: every table with its rows, bytes on disk, columns, first and last period and last sync; every board with its rows, period kind, metrics, license and redistribution; every source with its operator, license and the citation it asks for. The counts come from api/v2/overview.json, the columns from api/v2/tables.json, the board and source records from api/v2/boards.json and api/v2/sources.json; the page writes nothing of its own. Private rows are counted and marked, and are in no served file, view or API response.

3. Known caveats

4. Reproduce

python3 collectors/collect.py && python3 build_dataset.py

data/raw/_collect_status.json records which sources failed on a run; the build continues with what it has. site/api/v1/sources.json lists sources with their how string for the live snapshot.