界面已本地化;技能介绍与说明文档按所选语言由 AI 自动翻译,代码、命令、路径和专业术语保留原文,以原作者文档为准。
探索技能
开发与工程 · SKILL.md

tecoa_text_contrastive_objective

tecoa_text_contrastive_objective

【开发与工程】Compute TeCoA image-to-text contrastive logits, loss, margins, and accuracy for robustness recovery.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准

技能说明

GitHub

name: tecoa_text_contrastive_objective description: Compute TeCoA image-to-text contrastive logits, loss, margins, and accuracy for robustness recovery.

TeCoA Text Contrastive Objective

Use this skill when image and text embeddings must be scored according to the TeCoA objective. It implements the paper's cross-modal supervision contract: adversarial visual features should align with the correct text embeddings under normalized cosine logits.

Do not use this skill to generate adversarial perturbations or update model parameters. It is a deterministic scoring module consumed by attack and recovery harnesses.

Inputs

  • image_embeddings: numeric matrix with shape [batch, dim].
  • text_embeddings: numeric matrix with shape [classes, dim].
  • labels: integer target text index for each image row.
  • temperature: positive scalar, default 0.07.

Outputs

  • logits: cosine logits divided by temperature.
  • loss: mean image-to-text cross-entropy.
  • accuracy: top-1 image-to-text accuracy.
  • margins: correct logit minus best wrong logit for each row.
  • mean_margin: average margin.

Workflow

  1. Validate matrix dimensions, non-zero vectors, label range, and temperature.
  2. L2-normalize image and text embeddings row-wise.
  3. Compute image-to-text logits.
  4. Compute cross-entropy, top-1 accuracy, and margins.
  5. Return JSON-serializable metrics so recovery can compare clean, adversarial, and adapted states.

Validation

Run the tests in tests/test_contrastive_objective.py or the Distiller skill-tree validator with --run-tests.

Limitations

This skill uses Python standard-library math for portability. It is intended for deterministic reduced/proxy checks and numeric cross-checks, not high-throughput CLIP training.

我的收藏