界面已本地化;技能介绍与说明文档按所选语言由 AI 自动翻译,代码、命令、路径和专业术语保留原文,以原作者文档为准。
探索技能
设计与创意 · SKILL.md

visual_projection_training_step

visual_projection_training_step

【设计与创意】Execute a deterministic tiny projection update that mirrors LLaVA's visual-token to language-token alignment step.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准

技能说明

GitHub

name: visual_projection_training_step description: Execute a deterministic tiny projection update that mirrors LLaVA's visual-token to language-token alignment step.

Visual Projection Training Step

Use this skill when a recovery needs executable evidence that the trainable projection matrix mechanism ran. It is appropriate for reduced/proxy experiments when full CLIP/Vicuna training is blocked.

Inputs

  • Numeric visual feature vector.
  • Numeric target language embedding vector.
  • Initial scalar or matrix parameters.
  • Learning rate and number of steps.

Outputs

A training trace with loss_before, loss_after, params_before, params_after, and optimizer_state_changed.

Workflow

  1. Compute projected visual features with a linear map.
  2. Compute mean squared error against the target language vector.
  3. Apply deterministic gradient descent to trainable parameters.
  4. Report parameter and loss changes for recovery validation.

Validation

Run the standard-library tests or the Distiller skill-tree validator.

Limitations

This script is a tiny mathematical proxy. It does not load CLIP, Vicuna, LLaMA, or real image tensors.

我的收藏