{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "# 02 物理成功评估与视频复核\n\n        这一节解决一个关键问题：日志里的 success 是否真的代表机器人夹起杯子并放到盘子上。这里把旧的几何成功条件和 `physical_success` 分开统计，并用视频关键帧复核成功与失败。\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "from pathlib import Path\nimport json\nimport os\nimport shutil\nimport subprocess\nimport sys\n\n\ndef find_topic_root():\n    cwd = Path.cwd().resolve()\n    for candidate in [cwd, *cwd.parents]:\n        if (candidate / \"assets\" / \"metrics_snapshot.json\").exists():\n            return candidate\n    raise RuntimeError(\"请从 AMD ROCm 专题目录或 notebooks 子目录启动 Jupyter。\")\n\n\nTOPIC_ROOT = find_topic_root()\nASSET_DIR = TOPIC_ROOT / \"assets\"\nNOTEBOOK_DIR = TOPIC_ROOT / \"notebooks\"\nPROJECT_ROOT = Path(os.environ.get(\"PROJECT_ROOT\", \"/path/to/every-embodied/mujoco_pnp\"))\nDATA_ROOT = Path(os.environ.get(\"DATA_ROOT\", \"/path/to/datasets/every_embodied\"))\nOUTPUT_ROOT = Path(os.environ.get(\"OUTPUT_ROOT\", TOPIC_ROOT / \"outputs\"))\nMODEL_ROOT = Path(os.environ.get(\"MODEL_ROOT\", PROJECT_ROOT / \"ckpt\"))\n\nprint(\"TOPIC_ROOT =\", TOPIC_ROOT)\nprint(\"PROJECT_ROOT =\", PROJECT_ROOT)\nprint(\"DATA_ROOT =\", DATA_ROOT)\nprint(\"OUTPUT_ROOT =\", OUTPUT_ROOT)\nprint(\"MODEL_ROOT =\", MODEL_ROOT)\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "try:\n    from IPython.display import Image, Markdown, display\nexcept Exception:\n    class Markdown(str):\n        pass\n\n    def display(obj):\n        print(obj)\n\n    def Image(filename=None, width=None):\n        return f\"[image] {filename}\"\n\n\ndef show_asset(filename, width=960):\n    path = ASSET_DIR / filename\n    if path.exists():\n        display(Image(filename=str(path), width=width))\n    else:\n        print(f\"缺少素材：{path}\")\n\n\ndef md_table(headers, rows):\n    lines = [\"| \" + \" | \".join(headers) + \" |\", \"| \" + \" | \".join([\"---\"] * len(headers)) + \" |\"]\n    for row in rows:\n        lines.append(\"| \" + \" | \".join(str(x) for x in row) + \" |\")\n    display(Markdown(\"\\n\".join(lines)))\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Checkpoint 1：读取本专题的示例指标\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "snapshot_path = ASSET_DIR / \"metrics_snapshot.json\"\nsnapshot = json.loads(snapshot_path.read_text(encoding=\"utf-8\"))\nprint(json.dumps(snapshot[\"best_summary\"], ensure_ascii=False, indent=2))\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Checkpoint 2：理解 `physical_success`\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "def physical_success(row, min_lift=0.03, min_lift_steps=3, upright_threshold=0.7):\n    \"\"\"示例判定逻辑。实际项目中应以 eval 脚本记录的字段为准。\"\"\"\n    legacy = bool(row.get(\"legacy_success\", row.get(\"success\", False)))\n    max_lift = float(row.get(\"max_target_lift\", row.get(\"max_mug_lift\", 0.0)))\n    lifted_steps = int(row.get(\"lifted_steps\", 0))\n    upright = float(row.get(\"final_target_upright_cos\", row.get(\"final_mug_upright_cos\", 1.0)))\n    return legacy and max_lift >= min_lift and lifted_steps >= min_lift_steps and upright >= upright_threshold\n\n\nexamples = [\n    {\"name\": \"几何成功但倒杯\", \"success\": True, \"max_mug_lift\": 0.08, \"lifted_steps\": 80, \"final_mug_upright_cos\": 0.2},\n    {\"name\": \"推到盘边但没夹起\", \"success\": True, \"max_mug_lift\": 0.005, \"lifted_steps\": 0, \"final_mug_upright_cos\": 0.99},\n    {\"name\": \"真抓取并直立放置\", \"success\": True, \"max_mug_lift\": 0.09, \"lifted_steps\": 120, \"final_mug_upright_cos\": 0.96},\n]\nmd_table([\"样例\", \"physical_success\"], [(e[\"name\"], physical_success(e)) for e in examples])\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "旧 success 可以作为辅助指标，但不能替代视频和物理状态复核。特别是抓杯任务里，推、挤、倒杯都可能让终态几何看起来接近成功。\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Checkpoint 3：复核成功与失败关键帧\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "show_asset(\"smolvla_blue_failure_sequence.jpg\", width=1100)\nshow_asset(\"smolvla_blue_success_sequence.jpg\", width=1100)\nshow_asset(\"act_failure_sequence.jpg\", width=1100)\nshow_asset(\"act_success_sequence.jpg\", width=1100)\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "看视频时，沿时间轴观察四件事：是否接触目标杯、是否稳定夹起、是否搬运到盘子上、终态是否直立。\n"
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": "## Checkpoint 4：批量 JSONL 统计模板\n"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": "def summarize_jsonl(path):\n    path = Path(path)\n    if not path.exists():\n        print(f\"文件不存在：{path}\")\n        return None\n    rows = [json.loads(line) for line in path.read_text(encoding=\"utf-8\").splitlines() if line.strip()]\n    total = len(rows)\n    legacy = sum(bool(r.get(\"success\") or r.get(\"legacy_success\")) for r in rows)\n    physical = sum(bool(r.get(\"physical_success\")) for r in rows)\n    return {\"total\": total, \"legacy_success\": legacy, \"physical_success\": physical}\n\n\nexample_jsonl = OUTPUT_ROOT / \"eval_result.jsonl\"\nprint(\"把自己的 JSONL 放到这里后运行：\", example_jsonl)\nprint(summarize_jsonl(example_jsonl))\n"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}