---

name: web-interface description: "Operate and safely adapt the MedRAX Gradio chat interface around an initialized agent, including image and DICOM uploads, chat threads, tool-result images, and server configuration." disable-model-invocation: true metadata: disco-role: operating license: Apache 2.0

MedRAX web interface

Use this skill when a researcher needs to run or modify the browser UI around an already initialized MedRAX agent. This is an interface adapter, not a tool implementation or an agent-graph tutorial.

Operating contract

• Input: an initialized agent exposing workflow.stream(...), plus a tools_dict mapping the selected tool names to initialized tool instances.
• Output: a Gradio Blocks demo that accepts a user message and optional image, streams assistant/tool events, and returns (chat_history, image_path, text) for the bound outputs.
• Keep the original uploaded path for tools and keep a separately viewable path for the UI. Never silently replace a DICOM path with its converted PNG when a downstream tool needs the DICOM.
• The interface is a convenience for analysis, not a diagnostic device. Do not present model output as a clinical conclusion; retain a visible safety notice appropriate to the deployment.

For the detailed event and upload sequence, read workflows.md. For launch and resource settings, read configuration.md. For failure diagnosis, read troubleshooting.md.

Entrypoint and lifecycle

1. Load the medical system prompt and call the application initializer. Its configuration includes tools_to_use, model_dir, temp_dir, device, model name, temperature, top-p, and OpenAI-compatible keyword arguments.
2. Select only tools that are installed and affordable. The initializer's utility choices include ImageVisualizerTool and DicomProcessorTool; its model-backed choices include classification, segmentation, visual QA, report generation, grounding, LLaVA-Med, and generation. A selected tool must be present in both the agent tool list and tools_dict.
3. Call create_demo(agent, tools_dict). It constructs ChatInterface, a gr.Blocks layout, a messages-style Chatbot, text input, image display, upload buttons, and Clear Chat/New Thread controls.
4. Bind text submission as two stages: add_message first appends the user image and text and disables the textbox; process_message then streams agent events and updates chatbot, display image, and textbox. Re-enable the textbox after completion, including failure.
5. Launch with explicit, reviewed server settings. A safe local default is loopback and share=False; do not inherit the example's broad bind and public share settings without a deployment decision.

ChatInterface currently stores current_thread_id, original_file_path, and display_file_path on one Python object. That is adequate for a single-user demo only. For concurrent users, put these values in per-session gr.State (or a server-side session store), use a collision-resistant thread identifier, and never let one browser's upload or thread be reused by another browser.

Upload and path rules

• UploadButton is configured separately for ordinary images and DICOM. The current DICOM button accepts a generic file, so validate the lower-cased suffix before processing rather than trusting the widget.
• Copy uploads into a controlled temporary directory using a generated name. Preserve the original suffix; do not use the client filename as a path. Prefer a UUID plus suffix over a seconds-only timestamp to avoid collisions. Apply size, count, and suffix allowlists and clean old files deliberately.
• For an ordinary image, the copied path can serve both purposes after basic validation. For .dcm, call the selected DicomProcessorTool and read the returned display mapping's image_path for the UI.
• Keep original_file_path as the path sent in the textual image_path: message. Keep display_file_path as the PNG or other viewable conversion. If a multimodal payload is also built, encode the display path with its real MIME type; never label raw DICOM bytes as JPEG. This distinction is required even though the current implementation uses one fallback path for both.
• If DICOM is enabled, fail early with a clear configuration error when tools_dict["DicomProcessorTool"] is missing. Do not catch that as an unexplained chat failure.

Message and event handling

Build the current request in the shape expected by the agent:

• Optional path message: {"role": "user", "content": "image_path: <path>"}.
• Optional multimodal message: a user content list containing an image_url data payload for the viewable image.
• Optional text message: a user content list containing {"type": "text", "text": <message>}.

The current LangGraph call is conceptually:

agent.workflow.stream(
    {"messages": messages},
    {"configurable": {"thread_id": thread_id}},
)

The checkpointer and thread ID carry graph state; the visible Gradio history is not itself the complete graph conversation. New threads must receive a new ID. Clear Chat should also decide whether it is merely a visual reset or a new conversation; make that choice explicit instead of leaving stale graph state.

Handle the stream defensively:

• For a process event, take the latest message content, format it as text, and append an assistant ChatMessage.
• For an execute event, iterate tool messages, use the emitted tool name, and display textual results plus any validated image result. The current image branch recognizes the emitted name image_visualizer; do not assume it is the same spelling/case as ImageVisualizerTool in tools_dict.
• Never use eval on tool output. Prefer structured tool returns; for a legacy serialized value, use a constrained parser and validate the result shape and path before rendering it.
• Every normal and error path must yield the same number and order of outputs: (history, display_path, ""). Do not expose raw temporary paths in assistant text, and do not put an unredacted provider exception or credential in chat.

Reset, rendering, and assets

• Clear Chat should clear visible messages and the displayed image, remove or rotate the active session's upload references, and—if conversation reset is intended—rotate the thread ID too.
• New Thread should rotate only the conversation ID unless the product explicitly promises to remove the selected image. Document the behavior.
• Tool-generated images should be rendered only after checking that the path is an expected local result, not an arbitrary path returned by a model or tool.
• Resolve the avatar/logo from a packaged resource at runtime. If it is absent, use a no-avatar configuration instead of making startup depend on the process's current working directory.

Resource-constrained adaptation

For a CPU-only utility demonstration, initialize only ImageVisualizerTool and DicomProcessorTool with device="cpu" where the initializer requires a device. This avoids local vision-model weight loading, but the normal process_message still calls the configured chat model. A full chat demo therefore still needs an accessible OpenAI-compatible model endpoint, or a test/stub agent whose workflow.stream produces deterministic events. Do not claim a no-weight, no-model setup can answer medical questions.

Use configuration.md for a minimal selection matrix and troubleshooting.md when an optional model-backed tool prevents startup.

Scope boundaries

Route tool signatures, image processing, DICOM conversion implementation, and result schemas to chest-xray-analysis or image-data-utilities. Route LangGraph agent construction, prompts, checkpointers, and tool binding to agent-orchestration. Route benchmark execution and scoring to benchmark-evaluation. This skill only adapts their initialized outputs to a safe Gradio lifecycle.