算法艺术创作
algorithmic-art
通过代码和可调参数创作生成艺术与交互式视觉效果。
flutter-genui【设计与创意】Flutter GenUI SDK — conversational AI-driven UI using A2UI protocol. Use when building GenUI renderers, widget catalogs, Conversation orchestration, DataModel binding, SurfaceController setup, or SSE-streamed agent-to-UI flows in Flutter. Covers catalog design, A2UI transport, state binding, custom widgets, and triage integration.
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
Tech Stack: Flutter 3.41.x / Dart 3.10.9,
genuipackage (experimental/alpha), A2UI Protocol v0.8+, Riverpod 3.x
GenUI is Flutter's official SDK for generative UI — it turns text-based LLM conversations into interactive native Flutter widgets at runtime. Instead of receiving walls of text from an AI agent, users interact with dynamically rendered graphical interfaces composed from a developer-defined widget catalog.
Key principle: The AI agent generates structured JSON (A2UI protocol); the Flutter client validates against an approved catalog and renders native widgets. Agents never generate Dart code — they describe intent via JSON, the client renders it safely.
Status: Highly experimental / alpha. API will change. Pin to a specific commit or version.
GENUI PAYLOADS ARE UNTRUSTED. VALIDATE COMPONENT TYPES AGAINST THE CATALOG BEFORE RENDERING. NEVER EXECUTE AGENT-PROVIDED CODE. NEVER RENDER UNREGISTERED WIDGET TYPES.
| Concept | Class/Type | Role |
|---|---|---|
| Catalog | Catalog (list of CatalogItem) | Defines which widgets the AI is allowed to use — schemas + builders |
| Conversation | Conversation | Manages history, sends user input, invokes the model, drives the UI generation loop |
| A2UI Transport | A2uiTransportAdapter, A2uiMessage | Translates streamed model output into UI commands (GenUI uses A2UI under the hood) |
| DataModel | DataModel | Central observable state store — data binding and update flow |
| SurfaceController | SurfaceController | Processes messages, manages surfaces, keeps generated UI in sync |
| CatalogItem | CatalogItem | Individual widget registration — JSON schema + Dart builder function |
| Package | Purpose |
|---|---|
genui | Core framework — Catalog, Conversation, SurfaceController, rendering |
genui_a2a | A2UI protocol connector for custom agent backends (ADK, REST, etc.) |
genai_primitives | Technology-agnostic AI application types |
json_schema_builder | Dart JSON Schema validation for widget definitions |
pubspec.yaml in the package that uses GenUI (e.g., packages/shared_ui/):
dependencies:
genui:
git:
url: https://github.com/flutter/genui.git
path: packages/genui
genui_a2a:
git:
url: https://github.com/flutter/genui.git
path: packages/genui_a2a
reference/genui-catalog-design.md — how to define Catalog, CatalogItem, JSON schemas, builder functionsreference/genui-conversation-orchestration.md — Conversation lifecycle, model adapters, streaming loopreference/genui-state-binding.md — DataModel, SurfaceController, reactive state, surface renderingreference/genui-a2ui-transport.md — A2uiTransportAdapter, A2uiMessage, SSE streaming, JSONL parsingreference/genui-functions.md — A2UIFunctionEvaluator, declarative functions (formatCurrency, required, email, and/or/not), integration patternreference/genui-custom-widgets.md — custom widget integration, Slider/AudioPlayer/Video, custom triage widgetsreference/genui-security.md — catalog allowlist enforcement, input sanitization, payload limitsdart-mcp-server or Context7 MCP before using any APICatalogItem objects with JSON schemas and builder functionsA2uiTransportAdapter to connect to your AI agent backend (ADK SSE endpoint)Conversation with the catalog, transport adapter, and model configurationSurfaceController to process incoming A2UI messages and render surfacesDataModel for reactive updatesmelos run test and flutter analyzeDetailed patterns are in reference/:
genui-catalog-design.md — Catalog, CatalogItem, JSON schema definition, builder patterns, widget registrationgenui-conversation-orchestration.md — Conversation lifecycle, model adapter setup, message history, streaming loopgenui-state-binding.md — DataModel observable state, SurfaceController, surface management, reactive renderinggenui-a2ui-transport.md — A2uiTransportAdapter, A2uiMessage types, SSE/JSONL streaming, backend integrationgenui-functions.md — A2UIFunctionEvaluator: validation (required, regex, email), formatting (formatCurrency, formatDate, pluralize), logical (and, or, not), navigation (openUrl) — full Dart implementationgenui-custom-widgets.md — Custom widget registration, extended A2UI catalog (Slider, AudioPlayer, Video), custom triage widgets (photo_upload, dropdown, free_text, rating, confirmation)genui-security.md — Catalog allowlist enforcement, untrusted payload handling, input sanitization, size limits../a2ui-angular/reference/a2ui-protocol.md for the full A2UI protocol spec (5 message types, JSONL format)../a2ui-angular/reference/a2ui-protocol-advanced.md for streaming, action model, Gemini quirksGenUI can power a triage flow — the conversational UI that replaces static request forms.
Flow:
POST /triage/start returns { job_id }GET /triage/{job_id}/stream SSE{ response, category, widgets[] } via A2UI protocolGenUIRenderer renders widgets as native Flutter widgetsFallback: If GenUI/ADK unavailable -> standard form fields shown; record needs_classification = true
File location in monorepo:
packages/shared_ui/
lib/
widgets/
gen_ui/
gen_ui_renderer.dart # SurfaceController + rendering logic
widget_catalog.dart # Catalog with all registered CatalogItems
widget_types.dart # Custom widget builders (PhotoUpload, etc.)
triage_conversation.dart # Conversation setup for triage flow
// FORBIDDEN: Rendering arbitrary widget types from agent
Widget buildFromAgent(Map<String, dynamic> spec) {
// Agent controls what gets rendered — XSS/injection vector
return widgetRegistry[spec['type']]!(spec);
}
// REQUIRED: Validate against Catalog before rendering
Widget buildFromAgent(Map<String, dynamic> spec) {
final item = catalog.findByType(spec['type']);
if (item == null) {
logger.warning('Unknown GenUI type rejected: ${spec['type']}');
return const SizedBox.shrink(); // Skip unknown types
}
return item.builder(spec);
}
// FORBIDDEN: Executing agent-provided code
eval(spec['script']); // Dart doesn't have eval, but don't try alternatives
// FORBIDDEN: Using dynamic widget creation from agent strings
Function.apply(spec['handler'], []); // Never
// REQUIRED: Declarative catalog-based rendering only
// FORBIDDEN: Accepting unbounded payloads
final widgets = parseWidgets(response); // No size check
// REQUIRED: Enforce limits
final widgets = parseWidgets(response);
if (widgets.length > kMaxWidgets) {
widgets.removeRange(kMaxWidgets, widgets.length);
logger.warning('GenUI: payload truncated — exceeded $kMaxWidgets widgets');
}
# Add GenUI dependency (from packages/shared_ui/)
# Note: GenUI is git-sourced, add to pubspec.yaml manually
# Bootstrap after adding dependency
melos bootstrap
# Run tests
melos run test
# Analyze
flutter analyze packages/shared_ui/
# Run specific GenUI tests
flutter test packages/shared_ui/test/widgets/gen_ui/
| Source | URL / Tool | Purpose |
|---|---|---|
| Flutter GenUI Docs | https://docs.flutter.dev/ai/genui | Official overview, concepts |
| GenUI Components | https://docs.flutter.dev/ai/genui/components | Catalog, CatalogItem, Conversation, DataModel, SurfaceController |
| GenUI Get Started | https://docs.flutter.dev/ai/genui/get-started | Setup, initialization, first app |
| GenUI Input Events | https://docs.flutter.dev/ai/genui/input-events | User interaction handling |
| GenUI GitHub | https://github.com/flutter/genui | Source code, examples, packages |
| A2UI Protocol | https://a2ui.org/ | Underlying protocol spec |
| A2UI GitHub | https://github.com/google/A2UI | Reference implementations |
| Dart MCP | dart-mcp-server MCP | Dart/Flutter API verification |
| Context7 | Context7 MCP | Library docs fallback |