---

name: mobicom-artifact-evaluation description: Use when packaging a MobiCom artifact for the evaluation committee — choosing among the three ACM badges as a calibration of what you can prove, building a hardware-optional reproduction path for evaluators without radios, writing the smoke run that establishes functionality, and making code, data, and traces available.

MobiCom Artifact Evaluation

MobiCom accepted papers may earn the three ACM badges — Artifacts Available, Artifacts Evaluated — Functional, and Results Reproduced — from a separate artifact-evaluation committee (AEC). The badges are independent: you may pursue one, some, or all. The engineering problem specific to MobiCom is that the evaluation depends on a radio testbed the AEC does not have, so the packaging must let an evaluator get as far as possible without one.

Choose badges as a claim of what you can prove

Badge	What it asserts	What the AEC needs
Artifacts Available	the artifact is publicly and permanently retrievable	a stable public archive (DOI-backed)
Artifacts Evaluated — Functional	the artifact runs and does what the paper describes	a working build + smoke run on reachable hardware
Results Reproduced	the paper's key results regenerate	a path from artifact to the headline numbers

Pick badges honestly. Available is almost always worth claiming and is the lowest-friction. Functional requires a path that runs without your specific radios. Results Reproduced is hardest for a wireless paper because the channel is not portable — claim it only where a trace-replay or downscaled path can regenerate the numbers (mobicom-reproducibility). The exact 2027 badge deadlines and process are 待核实 — read the current artifact call.

The hardware-optional path is the core deliverable

Most AEC members will not have your SDRs, tags, or testbed. Design the package so a reviewer without hardware still reaches Functional:

artifact/
  README            # claims, badge targets, hardware needed vs optional, time budget
  env/              # container or pinned deps; no radio required to build
  smoke/            # runs in ~minutes on a laptop, no radio; proves the pipeline
  replay/           # recorded captures + decode/analysis -> regenerates key figures
  hardware/         # optional: scripts + exact radio/firmware to reproduce on a testbed
  data/             # traces that can legally ship (or a loader + instructions)
  expected/         # expected outputs AND expected variance ranges

• The smoke run is the functionality proof: a short, laptop-only command that exercises the pipeline end-to-end on bundled data and prints a recognizable result.
• The replay path feeds recorded captures through the same processing that produced the paper's figures, so results regenerate without the radio.
• The hardware path is documented for the rare evaluator (or future reader) with a matching testbed, with exact radio model, firmware, and channel setup.

Make availability real

• Deposit in a permanent, DOI-backed archive, not a personal URL that rots.
• Confirm what can legally ship: firmware and proprietary radio stacks often cannot; user-identifiable captures must be stripped or aggregated; site data may be bound by an agreement (mobicom-reproducibility).
• If part of the artifact cannot be released, say so explicitly and ship the largest functional subset rather than a broken whole.

Document variance honestly

An over-the-air result differs run to run. Ship expected-variance files alongside expected outputs so a rerun that lands within a stated margin reads as a reproduction, not a failure. An AEC member who sees a number move and has no variance guidance will log it as a non-reproduction.

Pre-submission checklist

• Badge targets chosen to match what the package can actually prove.
• Smoke run works on a clean laptop with no radio, in a stated time budget.
• Replay path regenerates the key figures from bundled captures.
• Hardware path documents exact radio/firmware/channel for testbed reproduction.
• Available artifact is in a permanent DOI-backed archive.
• Legal/ethical shipping decision made for firmware, traces, and site data.
• Expected outputs and expected variance both provided.

Output format

[Badges] targeted vs justified by the package
[Smoke] laptop-only run present + time budget
[Replay] key figures regenerate without hardware? y/n
[Hardware path] radio/firmware/channel documented? y/n
[Availability] archive + DOI + legal-shipping decision
[Gaps] what blocks each targeted badge