算法艺术创作
algorithmic-art
通过代码和可调参数创作生成艺术与交互式视觉效果。
video-generation-gateways【设计与创意】Select, integrate, and operate multi-model video-generation gateways — hosted inference aggregators such as fal.ai, Replicate, and WaveSpeed that expose many third-party video (and video-adjacent audio) models behind one account, one API surface, and one bill. Use when deciding whether to route video generation through a gateway versus a direct model…
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
A video-generation gateway is a hosted inference aggregator that resells many third-party video models (Veo, Kling, Seedance, Hailuo/MiniMax, Luma, Wan, LTX-Video, Pika, and others) behind a single account, a single API convention, and a single bill. You send a prompt and parameters to a gateway-owned endpoint; the gateway runs the underlying model on its own infrastructure and returns a hosted result URL. This skill covers fal.ai, Replicate, and WaveSpeed as the representative gateways, and the production decisions common to the category.
This skill is about routing and operating, not about art direction. Prompt craft, shot design, and motion direction for a specific model belong to model- or craft-specific skills. Here the questions are: which gateway, going through a gateway at all, how the async job contract works, how to discover a model's schema, how to pin a version, how to cap spend, how to fail over, and whose license governs the output.
Use a gateway when (production heuristics):
Go direct (or reconsider) when:
Out of scope for this skill: direct model-provider APIs (Google Vertex/Gemini Veo, Kling's own API, Runway, etc.), local/self-hosted inference, training or fine-tuning, and image-only gateway use.
Volatile facts below verified 2026-07-10; re-verify model IDs, prices, and limits before shipping — the catalogs and rate cards move frequently.
| Dimension | fal.ai | Replicate | WaveSpeed |
|---|---|---|---|
| Positioning | Generative-media-first (image/video/audio/3D), latency-focused | General model marketplace (community + "official" curated models), broadest catalog | Speed/latency-optimized inference for media |
| Catalog size (self-reported) | 1,000+ models | 100+ official models plus a large community catalog | 700–1,000+ models |
| Submit endpoint | https://queue.fal.run/{model-id} | https://api.replicate.com/v1/predictions (community) or .../v1/models/{owner}/{name}/predictions (official) | https://api.wavespeed.ai/api/v3/{model-id} |
| Async job states | IN_QUEUE → IN_PROGRESS → COMPLETED | starting → processing → succeeded/failed/canceled | created → processing → completed/failed |
| Sync option | Queue is the primary path; short models also have a run endpoint | Prefer: wait header (default 60s) for sync | Poll or webhook; result endpoint |
| Billing basis | Per-output (per-second or per-video for video); GPU-per-second fallback | Per-output for official models; per-second GPU time for community models | Per-model per-run, scales with resolution/duration/count |
| Balance model | Prepaid credits | Prepaid credit (hard cap at $0 for new accounts) | Prepaid credits (non-expiring) |
| Charged on failure? | No — pay only for successful output | Community models bill for GPU time even on model errors; official models bill per output | Pay per use; check per-model behavior |
| Schema discovery | Model-endpoints API with expand=openapi-3.0 | openapi_schema on the model version object | Per-model playground / API examples |
Sources: fal queue and pricing docs; Replicate HTTP API and pricing docs; WaveSpeed predictions and pricing docs (see Sources).
Reading the table: the three overlap heavily on what you can call — the same headline models (Veo, Kling, Seedance) appear on all three — so differentiation is mostly in the operational contract: billing granularity, whether failures cost money, how you discover a schema, and how versions are pinned. Treat the catalog as necessary but not sufficient; the operational contract is where integrations succeed or fail.
Video generation is slow (seconds to minutes), so every gateway is fundamentally an asynchronous job queue. Do not build on a synchronous request that blocks until the video is done — you will hit timeouts and lose work. The universal shape:
POST https://queue.fal.run/{model-id} with header
Authorization: Key $FAL_KEY. Returns request_id, response_url,
status_url, cancel_url, queue_position.GET https://queue.fal.run/{model-id}/requests/{request_id}/status
(append ?logs=1 for runner logs). States: IN_QUEUE, IN_PROGRESS,
COMPLETED.GET https://queue.fal.run/{model-id}/requests/{request_id}.PUT .../requests/{request_id}/cancel — removes queued jobs
immediately; in-progress jobs get a best-effort cancel signal.POST https://api.replicate.com/v1/predictions
with a version field. Submit (official):
POST https://api.replicate.com/v1/models/{owner}/{name}/predictions
(no version needed).Prefer: wait (or Prefer: wait=5) to hold the HTTP
connection up to 60s and return output inline for fast models — not suitable
for multi-minute video.Cancel-After header (5s–24h) auto-cancels the prediction
itself (distinct from Prefer: wait, which only bounds the HTTP wait).urls.get until succeeded or failed;
output is null until done. Web debug view at urls.web.POST https://api.wavespeed.ai/api/v3/{model-id} → returns a
task/prediction ID.GET https://api.wavespeed.ai/api/v3/predictions/{id}/result.
States: created, processing, completed, failed. outputs holds
result URLs; timings.inference gives duration in ms.Prefer webhooks for video: a minutes-long poll loop wastes requests and risks rate limits, and holding a connection open is fragile. Fall back to polling only when you cannot expose a public HTTPS endpoint (local dev, locked-down networks). If you poll, back off — image tasks ~2s, video tasks ~5s, then widen the interval (WaveSpeed's documented guidance, and a sound default everywhere). Never poll a single job multiple times per second.
Design every webhook handler to be idempotent: gateways retry deliveries and may deliver more than once for the same job ID.
Webhook payloads arrive from the public internet — verify them.
fal.ai signs webhooks with ED25519. To verify (verified 2026-07-10):
X-Fal-Webhook-Request-Id, X-Fal-Webhook-User-Id,
X-Fal-Webhook-Timestamp, X-Fal-Webhook-Signature (hex).https://rest.fal.ai/.well-known/jwks.json and cache it
up to 24h.webhook_ip_ranges from
https://api.fal.ai/v1/meta.fal delivery/retry (verified 2026-07-10): initial delivery timeout 15s;
on failure it retries 10 times over 2 hours. The success payload carries
status: "OK", results, and a seed; a request error carries
status: "ERROR"; a serialization failure sets payload: null with a
payload_error. The request_id is stable across retries; gateway_request_id
reflects the latest attempt.
Replicate lets you filter which events fire via webhook_events_filter
(e.g. ["completed"]); start and completed are always sent. Replicate
supports webhook signature verification via a signing secret — verify before
trusting a payload. (Verify current mechanism in Replicate's webhook docs
before shipping; verified conceptually 2026-07-10.)
WaveSpeed delivers a POST with id, model, input, outputs (on
completed), urls.get, status, and error (on failure). Confirm current
signing details in WaveSpeed's webhook docs before relying on them.
Each model exposes a different input surface (resolution enums, duration caps, image/video reference slots, audio toggles, seed, negative prompt). Never hardcode assumptions — discover the schema so your integration validates inputs and adapts when a model updates.
expand=openapi-3.0 and returns
the full OpenAPI 3.0 schema for an endpoint in the openapi field. Query
by endpoint_id (e.g. fal-ai/veo3) to get JSON Schema for every input
parameter. Use list/search modes to enumerate the catalog. (Verified
2026-07-10.)/v1/models/{owner}/{name}/versions/{version_id} and read
openapi_schema.components.schemas for the Input and Output schemas.
(Verified 2026-07-10.)Heuristic: cache the discovered schema per (model-id, version) and
re-fetch on a version change or on the first 422/validation error. Building a
small schema-validation layer in front of the gateway catches
parameter-shape breakage before you spend money on a failed job.
Reproducibility is the reason to care about versions: the same prompt on a silently-updated model can produce different motion, framing, or quality.
{owner}/{model}:{64-char-version-hash}. Pinning the hash guarantees
identical behavior over time; omitting it (or using an official model
addressed as {owner}/{model} with no hash) means Replicate keeps you on the
latest maintained version. Trade-off: pin the hash for reproducibility
and to avoid surprise behavior changes; use the unpinned official-model form
when you want automatic upgrades and less maintenance. Replicate documents
pinning as a reproducibility best practice. (Verified 2026-07-10.)fal-ai/veo3, or versioned IDs like fal-ai/kling-video/v2/...). Version
granularity is encoded in the ID the vendor publishes; watch changelogs for
new IDs and deprecations of old ones.Deprecation is a real operational risk on gateways, not a hypothetical. Underlying models are retired or superseded (a new Kling/Veo/Seedance version lands and the old endpoint is sunset). Mitigations (heuristics):
Video is the most expensive generative modality per call, and gateway spend can run away fast under retries or a bad loop. Controls differ sharply:
Production heuristics for cost safety:
Retries. Transient failures (429 rate limit, 5xx, capacity/cold-start
timeouts) are expected. Retry with exponential backoff and jitter, an
idempotency key you control, and a cap (both attempts and total spend).
Because a submitted-but-slow job is not the same as a failed job, distinguish
"submit failed" (safe to retry) from "job running slowly" (poll/await, do not
resubmit — resubmitting double-charges). Honor Retry-After when present.
Cross-gateway failover. Because the same headline models appear on multiple gateways, a gateway is a substitutable route, which is the strongest operational argument for the aggregator pattern. To fail over:
generateVideo(prompt, tier, constraints)), not a gateway-specific call, so the router picks a
provider+model.Two moderation layers stack: the underlying model's own safety behavior and the gateway's added moderation. They are not identical across gateways, so the same prompt on the same model can be accepted on one gateway and blocked on another.
enable_safety_checker toggle (default on); video-model moderation posture
is model-dependent. Users remain bound by fal's Acceptable Use Policy.
(Verified 2026-07-10.)Heuristics: (1) test moderation behavior per gateway, not once — a prompt cleared on your primary may be refused on your failover route, breaking availability assumptions. (2) Handle a moderation rejection distinctly from a transient error — it is deterministic, so retrying or failing over the same prompt often just fails or double-charges; instead surface it, adjust the prompt, or route to a model with a different policy. (3) Do not assume a gateway strips or adds a watermark, C2PA/provenance signal, or safety filter the same way the direct API does — verify.
Two separate questions: where does your data live, and who may use the output commercially.
Data retention / residency:
v3.fal.media/...). Retention is controllable via the
X-Fal-Object-Lifecycle-Preference header (expiration_duration_seconds, or
null for no expiry). Request/response JSON is stored 30 days by default.
Download outputs before they expire — expired files are unrecoverable.
(Verified 2026-07-10.)Because outputs are transient, always download and re-host anything you need; never link end users directly to a gateway CDN URL for durable content. For regulated data, confirm whether the gateway can offer a DPA or regional processing — most reseller flows send your inputs to third-party model infra.
Commercial-rights passthrough (the critical governance point): a gateway granting you rights to its service output does not override the underlying model's license. Rights flow through the model, not the gateway.
Heuristic: before using a gateway-generated video commercially, resolve the underlying model's license and the model provider's terms, not just the gateway's ToS. Premium models (Veo, Kling) frequently carry provider-specific commercial terms and per-clip restrictions; a gateway reselling them does not relax those terms. Record, per model you ship on, the license basis for commercial use so a later dispute or model swap is auditable.
| Symptom | Likely cause | Action |
|---|---|---|
| Submit returns fast but no video | Working as designed — it is async; you got a job ID, not a result | Poll status/result or await the webhook; do not treat the submit response as the output |
| Webhook never arrives | Endpoint not public HTTPS, handler slow (>15s on fal), or signature check rejecting valid payloads | Verify reachability, respond quickly then process async, confirm signature logic; fall back to polling |
| Duplicate results processed | Gateway retried delivery | Make the handler idempotent keyed on job/request ID |
422/validation error | Wrong parameter shape for this model/version | Re-fetch the model schema (fal openapi-3.0, Replicate openapi_schema); validate inputs before submit |
| Charged but no usable output | Community-model GPU-time billing on error (Replicate), or output expired before download | Prefer pay-per-output models for flaky work; download outputs immediately; check retention window |
| Model ID stopped resolving | Version deprecated/sunset | Update the config-driven model map to the new versioned ID or fail over to the tier fallback |
| Prompt blocked on one gateway, fine on another | Different stacked moderation layers | Treat as deterministic; adjust prompt or route to a different-policy model — do not blind-retry |
| Runaway spend | Unbounded retries or a bad loop against a prepaid balance | Cap retries/concurrency, keep balance small, instrument per-job cost |
| Output looks different after a "no-op" change | Unpinned version silently updated | Pin explicit version hashes/IDs; add a schema/behavior probe in CI |
| Result URL 404s later | Gateway CDN retention elapsed | Re-host durable content; never serve end users a raw gateway URL |
Intent: generate 500 five-second 720p vertical clips/day, cost-sensitive, some prompts near content-policy edges, must not exceed a fixed daily budget.
Reasoning: volume + cost sensitivity favors a per-second/per-run model on a gateway that (a) does not charge for failures and (b) supports a hard prepaid cap. Pick a mid-tier model (e.g. a Kling/Seedance turbo tier) rather than Veo. Put a second gateway behind the same capability tier for failover, since availability of a single model on a single gateway is not guaranteed at 500/day.
Plan: primary = fal queue.fal.run submit + ED25519-verified webhook,
prepaid balance sized to the daily budget as the circuit breaker; secondary =
same-tier model on WaveSpeed with its own funded balance. Router picks by
health + queue depth. Per-job cost is logged (job ID → model → params → charge).
Moderation rejections are surfaced to a review queue, not retried.
Expected result: clips delivered async, budget bounded by prepaid balances, a sunset or outage on one route degrades to the other. Failure modes: forgetting to fund the backup balance (failover silently fails); blind-retrying moderation rejections (wasted spend); serving raw CDN URLs that later 404.
Intent: a hero shot must render identically across months for a marketing pipeline; commercial use required.
Plan:
GET /v1/models/{owner}/{name}/versions and pick
the hash; store {owner}/{name}:{version_hash} in config.POST /v1/predictions with that pinned version, a webhook, and
webhook_events_filter: ["completed"]; set a Cancel-After deadline as a
spend guard.openapi_schema.components.schemas.Input before submitting.Why structured this way: pinning the hash locks reproducibility; the schema check prevents parameter drift from silently changing output; the retention-aware download prevents a lost result; the rights check prevents shipping a clip whose model license forbids commercial use. Failure modes: using the unpinned official-model form (auto-upgrades break reproducibility); assuming Replicate's ownership grant overrides the model's restrictive license.
Intent: integrate a video model you have not used; you do not know its duration cap, resolution enum, or whether it takes an image reference.
Plan: call the fal model-endpoints API with endpoint_id set to the model
(e.g. fal-ai/veo3) and expand=openapi-3.0; read the returned openapi
field's JSON Schema for the input object. Generate your request typing and a
validation layer from it; cache it keyed on the model ID; re-fetch on the first
422 or a changelog-announced version bump.
Why: this makes the integration self-describing and resilient to the model adding/renaming a parameter, instead of hardcoding a guessed body that breaks on the next model update. Failure mode: skipping discovery and hardcoding parameters copied from a blog post that lags the current schema.
All URLs verified 2026-07-10.
Model-specific prices (Veo 3.1 ~$0.20–0.60/s by resolution/audio; Kling 2.5 Turbo Pro ~$0.07/s) are from fal's pricing pages, verified 2026-07-10, and are volatile — always re-verify against the live rate card.