API reference
OpenAI-compatible, routed to independent nodes.
Swap your base URL and keep your code. Requests go through the TezGrid gateway to NVIDIA, Apple Silicon, AMD and CPU nodes run by operators. Usage settles per token in integer micro-units, where $1 is 1,000,000 µ.
https://api.tezgrid.com/v1
Every path below is relative to this base, except /health and /ready. If you run your own gateway for development, it listens on http://127.0.0.1:3000.
curl https://api.tezgrid.com/v1/chat/completions \ -H "Authorization: Bearer $TEZGRID_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model": "Qwen2.5-1.5B-Instruct", "messages": [{"role": "user", "content": "Hello"}]}'
Auth & billing
Chat completions need an API key from the console. Public reads like models, pricing and nodes don't.
- Authorization
Bearer <api_key>. Missing or invalid key returns401.- Charge
- 1 µ per prompt token plus 2 µ per completion token, i.e. $1 and $2 per million tokens.
- Split
- 99.8% to the operator that served the request, 0.2% platform fee.
- Empty wallet
- Returns
402. Nothing is sent to a node. - Idempotency-Key
- Retries with the same key are never billed or executed twice.
Routing headers
All optional. Plain OpenAI clients work without them.
Request
- X-TezGrid-Min-Trust
- Refuse nodes below a tier:
standard,verifiedorconfidential.confidentialis declared by the operator and not yet verified by hardware attestation. - X-TezGrid-Developer-Email
- Attribute usage to a developer account.
- X-External-Request-Id
- Your own correlation ID, echoed in usage records.
- X-TezGrid-Debug: latency
- Adds a
tezgrid_debugobject to non-streaming JSON responses.
Response
- X-TezGrid-Request-Id
- Gateway request ID.
- X-TezGrid-Provider-Id
- Node that served the request.
- X-TezGrid-Provider-Region
- Region of that node.
- X-TezGrid-Router-Latency-Ms
- Time spent choosing a node.
- X-TezGrid-Gateway-Overhead-Ms
- Total time added by the gateway.
- X-TezGrid-Estimated-Latency-Ms
- Router's latency estimate for the chosen node.
- X-TezGrid-Failover-Attempts
- How many nodes were tried before this one answered.
- X-TezGrid-TTFT-Ms
- Time to first token. Non-streaming only.
From the CLI, tezgrid ping-api --gateway URL runs the same probe. It needs TEZGRID_API_KEY for the chat check.
Operator setup
Inference runs on the operator's machine. The installer detects hardware and memory bandwidth, suggesting optimal models before you download any weights.
-
Install the CLI and llama-server (v0.3.0)
Creates an operator directory with
bin/andmodels/, and installs thetezgridCLI with acceleratedllama-serverfrom llama.cpp. Includes native Metal GPU acceleration on macOS (Apple Silicon), Vulkan on Windows and Linux x86_64, and CPU OpenMP on Linux arm64. Archives are downloaded fromget.tezgrid.com/releases/latest/(with GitHub releases as fallback) and verified againstSHA256SUMS. SetTEZGRID_VERSION=v0.3.0to pin a release. If you already run Ollama, TezGrid can use it instead. SetTEZGRID_LLAMA_SERVERto use your own custom llama.cpp binary.# Linux (x86_64, arm64) and macOS (Apple Silicon, Intel) curl -fsSL https://get.tezgrid.com/install.sh | bash # Windows (PowerShell) irm https://get.tezgrid.com/install-windows.ps1 | iex
-
Profile machine capacity & speed tiers
tezgrid doctorprofiles your CPU architecture, RAM channels, estimated memory bandwidth (e.g. DDR4 ~25–30 GB/s, DDR5 ~50–80 GB/s, Apple Unified Memory ~100–800 GB/s), GPU VRAM, disk space, and inference runtimes.tezgrid recommendevaluates the catalog against your hardware capacity and assigns speed badges:- 🟢 [BEST / FAST]: Ideal fit for available RAM and memory bandwidth. Delivers high token throughput (15–40+ tok/s) and sub-200ms TTFT.
- 🟡 [CAN RUN]: Fits in memory, usable interactive speeds (4–12 tok/s).
- 🟠 [DON'T USE / SLOW / EXCEEDS RAM]: Exceeds RAM or choked by memory bandwidth for practical interactive use.
# Run hardware diagnostics tezgrid doctor # Inspect recommended model speed tiers tezgrid recommend
-
Sign in or create operator account
Uses the same email and password as the console. If this is your first time, it registers your operator account. The session is securely saved in
session.jsonon this machine. Each machine is uniquely fingerprinted and bound to one operator account.tezgrid login you@example.com '<password>' # Or create an account: tezgrid signup you@example.com '<password>' -
Install an optimized model
Downloads verified GGUF model weights into your
models/directory. You can also pull models directly through Ollama using--from ollama.# Recommended fast models: tezgrid install-model Qwen2.5-1.5B-Instruct # or: tezgrid install-model Llama-3.2-1B-Instruct # or: tezgrid install-model Llama-3.2-3B-Instruct # or: tezgrid install-model DeepSeek-R1-Distill-Qwen-7B # or via Ollama: tezgrid install-model llama3.2 --from ollama
-
Go online
Starts accelerated
llama-serverwith automatic GPU offload (auto-ngl 99on Apple Silicon Metal or Vulkan), opens the outbound encrypted tunnel towss://api.tezgrid.com/v1/provider/tunnel(no open ports or firewall configuration required), and heartbeats every 15 seconds.tezgrid startis an alias.tezgrid up
-
Check status & stop
tezgrid statuschecks node connectivity, registered models, and settled token earnings.tezgrid downtakes your node offline cleanly.tezgrid status tezgrid down
-
CLI command reference
- tezgrid doctor
- Profiles CPU, memory bandwidth, GPU VRAM, free disk and available inference engines.
- tezgrid recommend
- Inspects catalog models and assigns 🟢 Best, 🟡 Can Run, and 🟠 Don't Use tiers.
- tezgrid install-model <id>
- Downloads verified GGUF weights or imports from Ollama (
--from ollama). - tezgrid models
- Lists all locally installed models and their file sizes.
- tezgrid remove-model <id>
- Deletes an installed model from local disk.
- tezgrid up
- Starts hardware-accelerated model server and opens the outbound WebSocket tunnel.
- tezgrid down
- Deregisters from gateway and terminates local engine processes.
- tezgrid status
- Displays node status, tunnel latency, models serving and token earnings.
- tezgrid ping-api
- Probes gateway connectivity, authentication and API latency.
Endpoints
Click an endpoint for its request and response shape.
Inference
POST /v1/chat/completionsChat completions, streaming or non-streaming auth
Request body
{
"model": "Qwen2.5-1.5B-Instruct",
"messages": [{"role": "user", "content": "Explain zero-copy streaming."}],
"stream": true,
"stream_options": { "include_usage": true },
"temperature": 0.7,
"max_tokens": 512
}
Streaming SSE response
data: {"id":"chatcmpl-a1","object":"chat.completion.chunk","created":1741285000,"model":"Qwen2.5-1.5B-Instruct","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}
data: {"id":"chatcmpl-a1","object":"chat.completion.chunk","created":1741285000,"model":"Qwen2.5-1.5B-Instruct","choices":[{"index":0,"delta":{"content":"Zero-copy"},"finish_reason":null}]}
data: {"id":"chatcmpl-a1","object":"chat.completion.chunk","created":1741285000,"model":"Qwen2.5-1.5B-Instruct","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":14,"completion_tokens":42,"total_tokens":56}}
data: [DONE]
Tokens are billed at 1 µ per prompt token and 2 µ per completion token ($1 and $2 per million tokens). 99.8% settles to the operator. Empty wallet returns 402, invalid key returns 401. Response headers provide live latency telemetry (X-TezGrid-TTFT-Ms, X-TezGrid-Router-Latency-Ms, X-TezGrid-Provider-Id).
GET /v1/modelsModels currently serving on online nodes public
Response
{
"object": "list",
"data": [
{ "id": "tezgrid-fast", "object": "model", "owned_by": "tezgrid" },
{ "id": "Qwen2.5-1.5B-Instruct", "object": "model", "owned_by": "tezgrid" },
{ "id": "Llama-3.2-3B-Instruct", "object": "model", "owned_by": "tezgrid" },
{ "id": "DeepSeek-R1-Distill-Qwen-7B", "object": "model", "owned_by": "tezgrid" }
]
}
Lists only models hosted by active online nodes in real time, plus the tezgrid-fast auto-routing alias. For the complete catalog of downloadable models, use GET /v1/models/catalog.
GET /v1/models/catalogComplete catalog of supported models & specs public
Response
[
{
"model_id": "Qwen2.5-1.5B-Instruct",
"format": "gguf",
"quantization": "Q4_K_M",
"parameter_count": "1.5B",
"file_size": 980000000,
"context_length": 8192,
"access": "Open",
"capabilities": { "chat": true, "streaming": true }
},
{
"model_id": "Llama-3.2-3B-Instruct",
"format": "gguf",
"quantization": "Q4_K_M",
"parameter_count": "3B",
"file_size": 2200000000,
"context_length": 8192,
"access": "Open",
"capabilities": { "chat": true, "streaming": true }
},
{
"model_id": "DeepSeek-R1-Distill-Qwen-7B",
"format": "gguf",
"quantization": "Q4_K_M",
"parameter_count": "7B",
"file_size": 4500000000,
"context_length": 8192,
"access": "Open",
"capabilities": { "chat": true, "streaming": true }
}
]
All catalog weights are pre-quantized (typically Q4_K_M) with verified hashes for safe, fast local downloading via tezgrid install-model.
GET /v1/pricingModeled marketplace pricing for reference models public
Economics from MarketplacePricingEngine for DeepSeek R1 Distill, Llama 8B and 70B, Qwen 72B and DeepSeek V3. These are modeled estimates, not a live per-request quote.
Accounts
POST /v1/auth/signupCreate a developer or provider account public
Body
{
"name": "Ada",
"email": "ada@example.com",
"password": "••••••••",
"role": "developer"
}
POST /v1/auth/login takes email, password and role.
GET /v1/developer/wallet/:developer_idWallet balance in micro-units auth
The developer ID is usually dev_<normalized_email>. The response includes wallet_balance_micro.
Related
GET /v1/developer/usage returns stored usage rows from MySQL. GET /v1/developer/tokens/hourly returns an hourly series for the console chart.
Network & providers
GET /v1/nodesRegistered and online provider nodes public
Returns { nodes: [...], mysql_enabled } with status, hardware, settled earnings and the telemetry fields the console uses.
GET /v1/provider/tunnelPersistent WebSocket tunnel for inference dispatch auth
WebSocket upgrade endpoint (wss://api.tezgrid.com/v1/provider/tunnel). Requires headers X-Provider-Id and X-Provider-Token.
Provider nodes dial out to this tunnel upon running tezgrid up. Requests are dispatched through this encrypted stream, and generated token deltas are pushed back with zero-buffering. Operators never need open ports, static IPs, or firewall port forwarding.
POST /v1/provider/registerRegister a node and its inference endpoint auth
Body
{
"provider_id": "<uuid>",
"node_name": "studio-mac",
"hardware": "Apple M4 Max",
"device_type": "Apple Silicon",
"inference_endpoint": "http://203.0.113.10:8080",
"models": ["Qwen2.5-1.5B-Instruct"],
"status": "ONLINE"
}
tezgrid up calls this for you. Most operators never call it directly.
GET /v1/network/metricsFleet-wide traffic and latency public
Includes a latency block with avg_router_ms, avg_ttft_ms, avg_decode_tps and success_rate_percent. Pass ?window=30m|24h|7d|30d for the traffic series that powers the network page.
Provider-scoped versions: GET /v1/provider/metrics and GET /v1/provider/dashboard.
GET /v1/diagnostics/latencyRouter and TTFT diagnostics, no inference needed public
Response
{
"gateway_reachable": true,
"online_providers": 12,
"avg_router_latency_ms": 0.9,
"avg_network_ttft_ms": 180,
"avg_network_decode_tps": 42,
"network_success_rate_percent": 99.2
}
Health
GET /healthGateway liveness public
Reports mysql_enabled, pilot readiness and any warnings. GET /ready is the readiness probe. Neither route is under /v1.