Catalog methodology

1) Data collection sources

Amply combines three source classes into a single comparable routing surface:

• Synthetic probes (controlled, repeatable workloads on fixed schedules).
• Real route telemetry (production route decisions and execution outcomes).
• Provider-attested catalog metadata (validated, reviewed, and versioned before use).

2) Amply-measured signals

Scheduled jobs call this deployment's own POST /api/v1/route with standard workloads. Each run records the recommended provider and handler compute_ms in amply_provider_metrics_history as synthetic_probe. Successful customer routes also append rows as route_telemetry.

Seven-day rollups surface as measured_* columns on amply_route_providers, with measured_aggregated_at for freshness.

3) Provider-validated signals

Attested metrics (provenance + verification time) remain the primary trust path when fresh. Providers whose catalog row is linked for self-service review may submit a cost_model JSON from the dashboard; it is stored as cost_model_pending_review until staff merge it—pending submissions are not used in automated scoring.

Every accepted provider submission is reviewed for schema validity, unit consistency, and outlier risk before publication.

4) Refresh cadence

• Route telemetry updates continuously as requests complete.
• Synthetic probes run on scheduled cycles to maintain baseline comparability.
• Aggregates are recomputed on rolling windows; freshness timestamps are published with results.
• Status diagnostics expose current freshness and any lagging subsystems.

5) Blended scoring policy

About 70% of the operational signal comes from Amply synthetic probes plus production telemetry on the same code path for every provider. About 30% comes from provider-submitted data that has been validated and merged into the catalog. Live freshness hints appear under GET /api/v1/status → diagnostics.catalog_metrics.

Ranking is constrained to measurable factors (cost, latency, reliability, route fit, and confidence source strength). Provider sponsorship does not alter rank.

6) Verification and anti-gaming controls

• Outlier checks and sanity bounds are applied before metrics become routing inputs.
• New provider claims remain non-authoritative until review and merge complete.
• Confidence/freshness metadata is returned to the caller so agents can risk-adjust decisions.
• Low-signal windows and stale measurements degrade confidence instead of being treated as equivalent evidence.

7) What agents can verify directly

Agents can inspect: /api/v1/status for freshness diagnostics, /api/v1/providers for measured/attested metadata, and POST /api/v1/route response fields such as raw_metrics, compute_ms, and ranking confidence hints.