{ "cells": [ { "cell_type": "markdown", "id": "ff2c2feb", "metadata": {}, "source": [ "# MicroDuck 直播 Notebook：球平衡 / FastSAC\n", "\n", "把任务、策略、ONNX、MuJoCo 和 RDK X5/BPU 验收串成一条可复用流程。" ] }, { "cell_type": "markdown", "id": "5ded49ab", "metadata": {}, "source": [ "## 0. 运行说明\n", "\n", "建议在 `02-可运行代码/microduck-playground-stilts` 的 Python 环境中启动 Jupyter。训练模型和板端 HBM 不随 Git 提交；通过 `MICRODUCK_ONNX`、`RDK_BPU_HBM` 和 `RDK_HOST` 注入。" ] }, { "cell_type": "code", "execution_count": 1, "id": "042bc62e", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.862445Z", "iopub.status.busy": "2026-09-19T08:21:26.862379Z", "iopub.status.idle": "2026-09-19T08:21:26.866803Z", "shell.execute_reply": "2026-09-19T08:21:26.866514Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "专题目录: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题\n", "可运行代码: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/02-可运行代码/microduck-playground-stilts\n", "输出目录: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs\n", "Jupyter 工作站: ubuntu Linux x86_64\n", "RDK 地址: 192.168.8.128 用户: sunrise\n" ] } ], "source": [ "from pathlib import Path\n", "import json\n", "import os\n", "import platform\n", "import shlex\n", "import socket\n", "import subprocess\n", "\n", "\n", "def locate_topic_root():\n", " cwd = Path.cwd().resolve()\n", " for base in (cwd, *cwd.parents):\n", " if (base / \"02-可运行代码\").is_dir() and (base / \"01-任务资料\").is_dir():\n", " return base\n", " group = base / \"16-专题组队学习\"\n", " if group.is_dir():\n", " matches = list(group.glob(\"*/02-可运行代码\"))\n", " if matches:\n", " return matches[0].parent\n", " configured = os.getenv(\"MICRODUCK_TOPIC_ROOT\")\n", " if configured and Path(configured).is_dir():\n", " return Path(configured).resolve()\n", " raise RuntimeError(\"找不到专题目录，请从 03-Notebook 启动 Jupyter，或设置 MICRODUCK_TOPIC_ROOT。\")\n", "\n", "\n", "TOPIC_ROOT = locate_topic_root()\n", "PLAYGROUND_ROOT = TOPIC_ROOT / \"02-可运行代码\" / \"microduck-playground-stilts\"\n", "OUTPUT_ROOT = TOPIC_ROOT / \"03-Notebook\" / \"outputs\"\n", "OUTPUT_ROOT.mkdir(parents=True, exist_ok=True)\n", "print(\"专题目录:\", TOPIC_ROOT)\n", "print(\"可运行代码:\", PLAYGROUND_ROOT)\n", "print(\"输出目录:\", OUTPUT_ROOT)\n", "print(\"Jupyter 工作站:\", socket.gethostname(), platform.system(), platform.machine())\n", "print(\"RDK 地址:\", os.getenv(\"RDK_HOST\", \"192.168.8.128\"), \"用户: sunrise\")" ] }, { "cell_type": "markdown", "id": "c2db6ca1", "metadata": {}, "source": [ "## 1. 模型契约与本地检查\n", "\n", "目标契约是 actor `obs[1, 54] -> action[1, 14]`。如果本任务没有随专题提交 checkpoint/ONNX，Notebook 会明确显示“模型未提供”，不会用别的任务模型冒充当前任务结果。" ] }, { "cell_type": "code", "execution_count": 2, "id": "47d2f852", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.867652Z", "iopub.status.busy": "2026-09-19T08:21:26.867573Z", "iopub.status.idle": "2026-09-19T08:21:26.870284Z", "shell.execute_reply": "2026-09-19T08:21:26.869931Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "任务: microduck-ball-balance\n", "ONNX: 未提供\n", "该任务尚未随专题提交任务专属 ONNX；请设置 MICRODUCK_ONNX 后再运行模型检查。\n" ] } ], "source": [ "TASK_ID = 'microduck-ball-balance'\n", "MODEL_HINT = None\n", "MODEL_PATH = Path(os.getenv(\"MICRODUCK_ONNX\", \"\")) if os.getenv(\"MICRODUCK_ONNX\") else None\n", "if MODEL_PATH is None and MODEL_HINT:\n", " candidate = TOPIC_ROOT / MODEL_HINT\n", " if candidate.exists():\n", " MODEL_PATH = candidate\n", "if MODEL_PATH is None and not MODEL_HINT and not TASK_ID:\n", " candidates = sorted(TOPIC_ROOT.glob(\"01-任务资料/**/*.onnx\"))\n", " MODEL_PATH = candidates[0] if candidates else None\n", "print(\"任务:\", TASK_ID or \"接口/网页演示\")\n", "print(\"ONNX:\", MODEL_PATH if MODEL_PATH else \"未提供\")\n", "if TASK_ID and MODEL_PATH is None:\n", " print(\"该任务尚未随专题提交任务专属 ONNX；请设置 MICRODUCK_ONNX 后再运行模型检查。\")" ] }, { "cell_type": "code", "execution_count": 3, "id": "8160a75c", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.871392Z", "iopub.status.busy": "2026-09-19T08:21:26.871301Z", "iopub.status.idle": "2026-09-19T08:21:26.874504Z", "shell.execute_reply": "2026-09-19T08:21:26.874257Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS-model-contract-cell: 模型未提供，已记录目标契约\n" ] } ], "source": [ "def inspect_onnx(path):\n", " try:\n", " import onnx\n", " except ImportError:\n", " print(\"缺少 onnx：请在 microduck-playground 环境中启动 Jupyter。\")\n", " return None\n", " model = onnx.load(str(path))\n", " onnx.checker.check_model(model)\n", " def shape(value):\n", " return [d.dim_value if d.dim_value else (d.dim_param or \"?\") for d in value.type.tensor_type.shape.dim]\n", " result = {\n", " \"inputs\": [(item.name, shape(item)) for item in model.graph.input],\n", " \"outputs\": [(item.name, shape(item)) for item in model.graph.output],\n", " \"nodes\": len(model.graph.node),\n", " \"metadata\": {item.key: item.value for item in model.metadata_props},\n", " }\n", " print(json.dumps(result, ensure_ascii=False, indent=2))\n", " return result\n", "\n", "contract = inspect_onnx(MODEL_PATH) if MODEL_PATH else None\n", "print(\"PASS-model-contract-cell:\", \"已完成检查\" if contract is not None else \"模型未提供，已记录目标契约\")" ] }, { "cell_type": "markdown", "id": "4b7e99be", "metadata": {}, "source": [ "## 2. ONNX 导出入口\n", "\n", "训练得到的是策略 checkpoint；导出时要把 actor 和观测归一化一起固化到 ONNX。下面是本任务命令模板。`--video` 会在本机录制仿真回放，完整视频写入 `outputs/`，不提交到 Git。\n", "\n", "```bash\n", "python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11\n", "```" ] }, { "cell_type": "code", "execution_count": 4, "id": "bc0aa378", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.875577Z", "iopub.status.busy": "2026-09-19T08:21:26.875485Z", "iopub.status.idle": "2026-09-19T08:21:26.877610Z", "shell.execute_reply": "2026-09-19T08:21:26.877356Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11\n", "导出前后检查：观测维度、动作顺序、归一化和 action clip。\n" ] } ], "source": [ "EXPORT_COMMAND = \"python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11\" if TASK_ID == \"microduck-ball-balance\" else (\"uv run python scripts/export.py \" + (TASK_ID or \"\") + \" --checkpoint-file --onnx-file outputs/policy.onnx --num-envs 1 --video --video-length 250\")\n", "print(EXPORT_COMMAND)\n", "print(\"导出前后检查：观测维度、动作顺序、归一化和 action clip。\")" ] }, { "cell_type": "markdown", "id": "60399d83", "metadata": {}, "source": [ "## 3. 训练路径说明：从零或从预训练 checkpoint 开始\n", "\n", "每个 Notebook 都把训练入口和回放入口分开：可以把 `resume=False` 作为从零初始化，也可以把 `resume=True`、任务专属 checkpoint 和对应的 actor/critic/优化器状态作为预训练续训。默认演示视频使用已经验证过的任务模型，不能把它描述成当前单元刚训练出来的模型。\n", "\n", "这里的“10 个 iteration”是 10 轮采样加策略优化，不是每个动作只训练 10 步；例如 `64 个并行环境 × 24 个控制步 × 10` 才是约 `15360` 条 transition。导航 Notebook 明确不训练，直接接入已有模型；扰动和爬梯等 Notebook 也会标明它们使用的任务专属模型或官方双模型。" ] }, { "cell_type": "markdown", "id": "c204326f", "metadata": {}, "source": [ "## 3. FastSAC smoke 训练与 ONNX 导出\n", "\n", "球平衡使用 MotrixLab 的 FastSAC，不复用 PPO/LSTM。默认在 Ubuntu 的 MotrixLab `.venv` 中用 64 个并行环境做 5 个 smoke iteration，并把本次 run 导出为 `outputs/motrix_ball_balance_latest.onnx`。若重新训练后要生成新的板端视频，需要同步重新执行 X5 HBM 编译单元，不能继续使用旧 HBM。" ] }, { "cell_type": "code", "execution_count": 5, "id": "5fee90a3", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.878340Z", "iopub.status.busy": "2026-09-19T08:21:26.878277Z", "iopub.status.idle": "2026-09-19T08:21:26.883416Z", "shell.execute_reply": "2026-09-19T08:21:26.883201Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MotrixLab: /home/ubuntu/workspaces/MotrixLab\n", "FastSAC Python: /home/ubuntu/workspaces/MotrixLab/.venv/bin/python\n", "固定输出: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/motrix_ball_balance_latest.onnx\n", "已跳过 FastSAC smoke 训练：设置 MICRODUCK_RUN_FASTSAC_SMOKE=1 后重新运行。\n" ] } ], "source": [ "import shutil\n", "import sys\n", "\n", "MOTRIX_ROOT = Path(os.getenv(\"MOTRIX_ROOT\", \"/home/ubuntu/workspaces/MotrixLab\")).expanduser()\n", "MOTRIX_PYTHON = Path(os.getenv(\"MOTRIX_PYTHON\", str(MOTRIX_ROOT / \".venv/bin/python\"))).expanduser()\n", "MOTRIX_RUNNER = PLAYGROUND_ROOT / \"scripts\" / \"motrix_runner.py\"\n", "FASTSAC_ENVS = int(os.getenv(\"MICRODUCK_FASTSAC_ENVS\", \"64\"))\n", "FASTSAC_ITERATIONS = int(os.getenv(\"MICRODUCK_FASTSAC_ITERATIONS\", \"5\"))\n", "RUN_FASTSAC_SMOKE = os.getenv(\"MICRODUCK_RUN_FASTSAC_SMOKE\", \"1\") == \"1\"\n", "FASTSAC_ONNX = OUTPUT_ROOT / \"motrix_ball_balance_latest.onnx\"\n", "print(\"MotrixLab:\", MOTRIX_ROOT)\n", "print(\"FastSAC Python:\", MOTRIX_PYTHON)\n", "print(\"固定输出:\", FASTSAC_ONNX)\n", "\n", "if not RUN_FASTSAC_SMOKE:\n", " print(\"已跳过 FastSAC smoke 训练：设置 MICRODUCK_RUN_FASTSAC_SMOKE=1 后重新运行。\")\n", "elif not MOTRIX_ROOT.is_dir() or not MOTRIX_PYTHON.is_file():\n", " print(\"找不到 MotrixLab 或其 Python 环境；请先在 Ubuntu 完成 MotrixLab 环境安装。\")\n", "else:\n", " train_cmd = [\n", " str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"train\", str(MOTRIX_ROOT / \"scripts/train.py\"),\n", " \"task=microduck-ball-balance/motrix.fastsac\",\n", " f\"num_envs={FASTSAC_ENVS}\",\n", " \"play=false\", \"render=false\", \"algo.asynchronous=false\",\n", " \"algo.device=cuda\", \"algo.agent.compile=false\", \"algo.agent.amp=false\",\n", " # Keep the default run genuinely smoke-sized. The replay buffer must\n", " # be populated before FastSAC can take its first update.\n", " \"algo.agent.learning_starts=0\", \"algo.agent.buffer_size=8\", \"algo.agent.batch_size=64\",\n", " \"algo.agent.num_updates=1\", f\"algo.trainer.num_learning_iterations={FASTSAC_ITERATIONS}\",\n", " ]\n", " print(\"开始 FastSAC smoke 训练:\", \" \".join(shlex.quote(x) for x in train_cmd))\n", " train_result = subprocess.run(train_cmd, cwd=MOTRIX_ROOT, text=True)\n", " if train_result.returncode != 0:\n", " raise RuntimeError(f\"FastSAC smoke 训练失败，returncode={train_result.returncode}\")\n", " run_root = MOTRIX_ROOT / \"runs\" / \"microduck-ball-balance\" / \"motrix\" / \"torch\" / \"fastsac\"\n", " run_dirs = sorted((p for p in run_root.iterdir() if p.is_dir()), key=lambda p: p.stat().st_mtime)\n", " if not run_dirs:\n", " raise FileNotFoundError(f\"训练成功但没有找到 FastSAC run: {run_root}\")\n", " run_dir = run_dirs[-1]\n", " motrix_export = Path(\"outputs\") / \"motrix_ball_balance_latest.onnx\"\n", " export_cmd = [\n", " str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"export\", str(MOTRIX_ROOT / \"scripts/export_onnx.py\"),\n", " f\"run_dir={run_dir}\", f\"output={motrix_export.as_posix()}\", \"opset=11\",\n", " ]\n", " print(\"导出 FastSAC ONNX:\", \" \".join(shlex.quote(x) for x in export_cmd))\n", " export_result = subprocess.run(export_cmd, cwd=MOTRIX_ROOT, text=True)\n", " exported_onnx = MOTRIX_ROOT / motrix_export\n", " if export_result.returncode != 0 or not exported_onnx.is_file():\n", " raise RuntimeError(\"FastSAC ONNX 导出失败。\")\n", " shutil.copy2(exported_onnx, FASTSAC_ONNX)\n", " MODEL_PATH = FASTSAC_ONNX\n", " print(\"PASS-fastsac-model:\", MODEL_PATH)\n", " print(\"实际训练 run:\", run_dir)" ] }, { "cell_type": "markdown", "id": "f72b8afc", "metadata": {}, "source": [ "## 3b. X5 HBM 编译\n", "\n", "训练后的 ONNX 不能直接交给 RDK X5。下面固定输入输出 batch 为 1，并通过 Ubuntu 上的官方 X5 CPU Docker 工具链运行 `hb_mapper makertbin --model-type onnx`。生成的 `ball_balance.opset11.bin` 会覆盖编译目录，下一节视频使用的就是这份新 HBM。" ] }, { "cell_type": "code", "execution_count": 6, "id": "e29c8002", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:26.884151Z", "iopub.status.busy": "2026-09-19T08:21:26.884090Z", "iopub.status.idle": "2026-09-19T08:21:34.583174Z", "shell.execute_reply": "2026-09-19T08:21:34.582891Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "开始 X5 HBM 编译: docker run --rm -v /home/ubuntu/workspaces:/work openexplorer/ai_toolchain_ubuntu_20_x5_cpu:v1.2.8 sh -lc 'hb_mapper makertbin --config /work/microduck_bpu_models/compiled/ball_balance/.fast_perf/ball_balance.opset11_config.yaml --model-type onnx'\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m log will be stored in /open_explorer/hb_mapper_makertbin.log\n", "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m Start hb_mapper....\n", "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m hbdk version 3.49.15\n", "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m horizon_nn version 1.1.0\n", "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m hb_mapper version 1.24.3\n", "2026-09-19 16:21:29,873 \u001b[92mINFO\u001b[0m Start Model Convert....\n", "2026-09-19 16:21:29,876 \u001b[92mINFO\u001b[0m Using onnx model file: /work/microduck_bpu_models/converted/motrix_ball_balance.static.opset11.onnx\n", "2026-09-19 16:21:29,877 \u001b[92mINFO\u001b[0m Model has 1 inputs according to model file\n", "2026-09-19 16:21:29,877 \u001b[92mINFO\u001b[0m Input shape [1, 54] has length: 2, make sure it is a featuremap input\n", "2026-09-19 16:21:29,877 \u001b[92mINFO\u001b[0m Parameter calibration_type is skip. cal_data_dir check skipped\n", "2026-09-19 16:21:29,877 \u001b[92mINFO\u001b[0m custom_op does not exist, skipped\n", "2026-09-19 16:21:29,877 \u001b[93mWARNING\u001b[0m Input node obs's input_source not set, it will be set to ddr by default\n", "2026-09-19 16:21:29,879 \u001b[93mWARNING\u001b[0m can not get nhwc info from shape: [1, 54]\n", "2026-09-19 16:21:29,879 \u001b[92mINFO\u001b[0m The calibration_type you specified is skip, the skip uses max+random data for calibration\n", "2026-09-19 16:21:29,889 \u001b[92mINFO\u001b[0m Start to Horizon NN Model Convert.\n", "2026-09-19 16:21:29,891 \u001b[92mINFO\u001b[0m Loading horizon_nn debug methods:set()\n", "2026-09-19 16:21:29,891 \u001b[92mINFO\u001b[0m The activation calibration parameters:\n", " calibration_type: fixed\n", "The output nodes of model are set to datatype: int8\n", "The input of all Softmax nodes are set to : int8\n", "2026-09-19 16:21:29,891 \u001b[92mINFO\u001b[0m The specified model compilation architecture: bayes-e.\n", "2026-09-19 16:21:29,891 \u001b[92mINFO\u001b[0m The specified model compilation optimization parameters: ['run_fast'].\n", "2026-09-19 16:21:29,892 \u001b[92mINFO\u001b[0m Start to prepare the onnx model.\n", "2026-09-19 16:21:29,893 \u001b[92mINFO\u001b[0m Input ONNX Model Information:\n", "ONNX IR version: 6\n", "Opset version: ['ai.onnx v11', 'horizon v1']\n", "Producer: pytorch v2.7.0\n", "Domain: None\n", "Version: None\n", "Graph input:\n", " obs: shape=[1, 54], dtype=FLOAT32\n", "Graph output:\n", " actions: shape=[1, 14], dtype=FLOAT32\n", "2026-09-19 16:21:29,893 \u001b[92mINFO\u001b[0m Using optimization configuration: run_fast\n", "2026-09-19 16:21:29,905 \u001b[92mINFO\u001b[0m End to prepare the onnx model.\n", "2026-09-19 16:21:29,906 \u001b[92mINFO\u001b[0m Saving model to: ball_balance.opset11_original_float_model.onnx.\n", "2026-09-19 16:21:29,906 \u001b[92mINFO\u001b[0m Start to optimize the onnx model.\n", "2026-09-19 16:21:29,924 \u001b[92mINFO\u001b[0m End to optimize the onnx model.\n", "2026-09-19 16:21:29,925 \u001b[92mINFO\u001b[0m Saving model to: ball_balance.opset11_optimized_float_model.onnx.\n", "2026-09-19 16:21:29,925 \u001b[92mINFO\u001b[0m Start to calibrate the model.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2026-09-19 16:21:30,094 \u001b[92mINFO\u001b[0m There are 1 samples in the data set.\n", "2026-09-19 16:21:30,095 \u001b[92mINFO\u001b[0m Run calibration model with fixed thresholds method.\n", "2026-09-19 16:21:30,131 \u001b[92mINFO\u001b[0m End to calibrate the model.\n", "2026-09-19 16:21:30,139 \u001b[92mINFO\u001b[0m Saving model to: ball_balance.opset11_calibrated_model.onnx.\n", "2026-09-19 16:21:30,140 \u001b[92mINFO\u001b[0m Start to quantize the model.\n", "2026-09-19 16:21:30,249 \u001b[92mINFO\u001b[0m End to quantize the model.\n", "2026-09-19 16:21:30,254 \u001b[92mINFO\u001b[0m Saving model to: ball_balance.opset11_quantized_model.onnx.\n", "2026-09-19 16:21:30,254 \u001b[92mINFO\u001b[0m Start to compile the model with march bayes-e.\n", "2026-09-19 16:21:30,292 \u001b[92mINFO\u001b[0m Compile submodel: main_graph_subgraph_0\n", "2026-09-19 16:21:30,294 \u001b[92mINFO\u001b[0m hbdk-cc parameters:['--O3', '--debug', '--core-num', '1', '--fast', '--input-layout', 'NHWC', '--output-layout', 'NHWC', '--input-source', 'ddr']\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "[- ] 1%\r", "[= ] 2%\r", "[=- ] 3%\r", "[== ] 4%\r", "[==- ] 5%\r", "[=== ] 6%\r", "[===- ] 7%\r", "[==== ] 8%\r", "[====- ] 9%\r", "[===== ] 10%\r", "[=====- ] 11%\r", "[====== ] 12%\r", "[======- ] 13%\r", "[======= ] 14%\r", "[=======- ] 15%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[======== ] 16%\r", "[========- ] 17%\r", "[========= ] 18%\r", "[=========- ] 19%\r", "[========== ] 20%\r", "[==========- ] 21%\r", "[=========== ] 22%\r", "[===========- ] 23%\r", "[============ ] 24%\r", "[============- ] 25%\r", "[============= ] 26%\r", "[=============- ] 27%\r", "[============== ] 28%\r", "[==============- ] 29%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[=============== ] 30%\r", "[===============- ] 31%\r", "[================ ] 32%\r", "[================- ] 33%\r", "[================= ] 34%\r", "[=================- ] 35%\r", "[================== ] 36%\r", "[==================- ] 37%\r", "[=================== ] 38%\r", "[===================- ] 39%\r", "[==================== ] 40%\r", "[====================- ] 41%\r", "[===================== ] 42%\r", "[=====================- ] 43%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[====================== ] 44%\r", "[======================- ] 45%\r", "[======================= ] 46%\r", "[=======================- ] 47%\r", "[======================== ] 48%\r", "[========================- ] 49%\r", "[========================= ] 50%\r", "[=========================- ] 51%\r", "[========================== ] 52%\r", "[==========================- ] 53%\r", "[=========================== ] 54%\r", "[===========================- ] 55%\r", "[============================ ] 56%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[============================- ] 57%\r", "[============================= ] 58%\r", "[=============================- ] 59%\r", "[============================== ] 60%\r", "[==============================- ] 61%\r", "[=============================== ] 62%\r", "[===============================- ] 63%\r", "[================================ ] 64%\r", "[================================- ] 65%\r", "[================================= ] 66%\r", "[=================================- ] 67%\r", "[================================== ] 68%\r", "[==================================- ] 69%\r", "[=================================== ] 70%\r", "[===================================- ] 71%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[==================================== ] 72%\r", "[====================================- ] 73%\r", "[===================================== ] 74%\r", "[=====================================- ] 75%\r", "[====================================== ] 76%\r", "[======================================- ] 77%\r", "[======================================= ] 78%\r", "[=======================================- ] 79%\r", "[======================================== ] 80%\r", "[========================================- ] 81%\r", "[========================================= ] 82%\r", "[=========================================- ] 83%\r", "[========================================== ] 84%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[==========================================- ] 85%\r", "[=========================================== ] 86%\r", "[===========================================- ] 87%\r", "[============================================ ] 88%\r", "[============================================- ] 89%\r", "[============================================= ] 90%\r", "[=============================================- ] 91%\r", "[============================================== ] 92%\r", "[==============================================- ] 93%\r", "[=============================================== ] 94%\r", "[===============================================- ] 95%\r", "[================================================ ] 96%\r", "[================================================- ] 97%\r", "[================================================= ] 98%\r", "[=================================================-] 99%" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\r", "[==================================================] 100%\n", "2026-09-19 16:21:31,820 \u001b[92mINFO\u001b[0m consumed time 1.51209\n", "2026-09-19 16:21:31,846 \u001b[92mINFO\u001b[0m FPS=4055.48, latency = 246.6 us, DDR = 246608 bytes (see main_graph_subgraph_0.html)\n", "2026-09-19 16:21:31,861 \u001b[92mINFO\u001b[0m End to compile the model with march bayes-e.\n", "2026-09-19 16:21:31,862 \u001b[92mINFO\u001b[0m The main quantized node information:\n", "==================================================================================================================\n", "Node ON Subgraph Type Cosine Similarity Threshold DataType \n", "------------------------------------------------------------------------------------------------------------------\n", "/normalizer/Sub BPU id(0) HzSElementwiseSub -- 1.0 int8 \n", "/actor/net/net.0/Gemm_pre_reshape BPU id(0) Reshape -- 1.0 int8 \n", "/actor/net/net.0/Gemm BPU id(0) HzSQuantizedConv -- 1.0 int8 \n", "/actor/net/net.1/ReduceMean BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.1/Sub BPU id(0) HzSElementwiseSub -- 1.0 int8 \n", "/actor/net/net.1/Pow BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.1/ReduceMean_1 BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.1/Div_reciprocal BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.1/Div_mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.1/Mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.1/Add_1 BPU id(0) HzSElementwiseAdd -- 1.0 int8 \n", "/actor/net/net.2/Mul BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.3/Gemm BPU id(0) HzSQuantizedConv -- 0.731059 int8 \n", "/actor/net/net.4/ReduceMean BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.4/Sub BPU id(0) HzSElementwiseSub -- 1.0 int8 \n", "/actor/net/net.4/Pow BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.4/ReduceMean_1 BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.4/Div_reciprocal BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.4/Div_mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.4/Mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.4/Add_1 BPU id(0) HzSElementwiseAdd -- 1.0 int8 \n", "/actor/net/net.5/Mul BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.6/Gemm BPU id(0) HzSQuantizedConv -- 0.731059 int8 \n", "/actor/net/net.7/ReduceMean BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.7/Sub BPU id(0) HzSElementwiseSub -- 1.0 int8 \n", "/actor/net/net.7/Pow BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.7/ReduceMean_1 BPU id(0) HzSQuantizedReduceMean -- 1.0 int8 \n", "/actor/net/net.7/Div_reciprocal BPU id(0) HzLut -- 1.0 int8 \n", "/actor/net/net.7/Div_mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.7/Mul BPU id(0) HzSElementwiseMul -- 1.0 int8 \n", "/actor/net/net.7/Add_1 BPU id(0) HzSElementwiseAdd -- 1.0 int8 \n", "/actor/net/net.8/Mul BPU id(0) HzLut -- 1.0 int8 \n", "/actor/fc_mu/Gemm BPU id(0) HzSQuantizedConv -- 0.731059 int8 \n", "/actor/Tanh BPU id(0) HzLut -- 1.0 int8 \n", "/actor/Mul BPU id(0) HzSElementwiseMul -- 0.761594 int8 \n", "/actor/Add BPU id(0) HzSElementwiseAdd -- 1.0 int8\n", "2026-09-19 16:21:31,863 \u001b[92mINFO\u001b[0m The quantized model output:\n", "============\n", "Output \n", "------------\n", "2026-09-19 16:21:31,863 \u001b[92mINFO\u001b[0m End to Horizon NN Model Convert.\n", "2026-09-19 16:21:31,864 \u001b[92mINFO\u001b[0m start convert to *.bin file....\n", "2026-09-19 16:21:31,878 \u001b[92mINFO\u001b[0m ONNX model output num : 1\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ############# model deps info #############\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m hb_mapper version : 1.24.3\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m hbdk version : 3.49.15\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m hbdk runtime version: 3.15.55.0\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m horizon_nn version : 1.1.0\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ############# model_parameters info #############\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m onnx_model : /work/microduck_bpu_models/converted/motrix_ball_balance.static.opset11.onnx\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m BPU march : bayes-e\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m layer_out_dump : False\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m log_level : DEBUG\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m working dir : /work/microduck_bpu_models/compiled/ball_balance/model_output\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m output_model_file_prefix: ball_balance.opset11\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m remove node type : ['Quantize', 'Transpose', 'Dequantize', 'Cast', 'Reshape', 'Softmax', 'DequantizeFilter']\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ############# input_parameters info #############\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ------------------------------------------\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ---------input info : obs ---------\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_name : obs\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_type_rt : featuremap\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_space&range : regular\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_layout_rt : NCHW\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_type_train : featuremap\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_layout_train : NCHW\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m norm_type : no_preprocess\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input_shape : 1x54\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ---------input info : obs end -------\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ------------------------------------------\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ############# calibration_parameters info #############\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m preprocess_on : False\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m calibration_type: : skip\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m optimization : run_fast;\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m per_channel : False\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m ############# compiler_parameters info #############\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m debug : True\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m compile_mode : latency\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m hbdk_pass_through_params: --O3 --debug --core-num 1 --fast\n", "2026-09-19 16:21:31,879 \u001b[92mINFO\u001b[0m input-source : {'obs': 'ddr', '_default_value': 'ddr'}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2026-09-19 16:21:32,907 \u001b[92mINFO\u001b[0m log will be stored in /work/microduck_bpu_models/compiled/ball_balance/model_output/hb_model_modifier.log\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m Nodes that will be removed from this model: ['obs_obs_HzCalibration_HzQuantize', 'actions_HzDequantize']\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m Node 'obs_obs_HzCalibration_HzQuantize' found, its OP type is 'Quantize'\n", "2026-09-19 16:21:32,909 \u001b[93mWARNING\u001b[0m node name is obs_obs_HzCalibration_HzQuantize, op type is Quantize, no axis information\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m scale: obs_HzCalibration_y_scale; zero point: 0. node info details are stored in hb_model_modifier log file\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m Node 'obs_obs_HzCalibration_HzQuantize' is removed\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m Node 'actions_HzDequantize' found, its OP type is 'Dequantize'\n", "2026-09-19 16:21:32,909 \u001b[92mINFO\u001b[0m scale: actions_reshape_HzCalibration_y_scale; zero point: 0. node info details are stored in hb_model_modifier log file\n", "2026-09-19 16:21:32,910 \u001b[92mINFO\u001b[0m Node 'actions_HzDequantize' is removed\n", "2026-09-19 16:21:32,910 \u001b[92mINFO\u001b[0m modified model saved as ball_balance.opset11.bin\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2026-09-19 16:21:33,234 \u001b[92mINFO\u001b[0m Convert to runtime bin file successfully!\n", "2026-09-19 16:21:33,234 \u001b[92mINFO\u001b[0m End Model Convert\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "PASS-X5-HBM: /home/ubuntu/workspaces/microduck_bpu_models/compiled/ball_balance/model_output/ball_balance.opset11.bin\n" ] } ], "source": [ "import shutil\n", "\n", "BPU_MODEL_ROOT = Path(os.getenv(\"MICRODUCK_BPU_MODEL_ROOT\", \"/home/ubuntu/workspaces/microduck_bpu_models\")).expanduser()\n", "BPU_CONVERTED = BPU_MODEL_ROOT / \"converted\"\n", "BPU_COMPILED = BPU_MODEL_ROOT / \"compiled\" / \"ball_balance\"\n", "BPU_STATIC_ONNX = BPU_CONVERTED / \"motrix_ball_balance.static.opset11.onnx\"\n", "BPU_CONFIG = BPU_COMPILED / \".fast_perf\" / \"ball_balance.opset11_config.yaml\"\n", "BPU_OUTPUT = BPU_COMPILED / \"model_output\" / \"ball_balance.opset11.bin\"\n", "BPU_DOCKER_IMAGE = os.getenv(\"MICRODUCK_BPU_DOCKER_IMAGE\", \"openexplorer/ai_toolchain_ubuntu_20_x5_cpu:v1.2.8\")\n", "\n", "def workspace_container_path(path):\n", " workspace = Path(\"/home/ubuntu/workspaces\")\n", " try:\n", " return Path(\"/work\") / path.resolve().relative_to(workspace)\n", " except ValueError as exc:\n", " raise RuntimeError(f\"HBM 文件必须位于 {workspace}: {path}\") from exc\n", "\n", "if not FASTSAC_ONNX.is_file():\n", " print(\"跳过 HBM 编译：先成功运行第 3 节 FastSAC 训练与导出。\")\n", "elif shutil.which(\"docker\") is None:\n", " print(\"跳过 HBM 编译：当前 Ubuntu 找不到 docker。\")\n", "else:\n", " import onnx\n", "\n", " BPU_CONVERTED.mkdir(parents=True, exist_ok=True)\n", " BPU_CONFIG.parent.mkdir(parents=True, exist_ok=True)\n", " BPU_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " for stale_file in BPU_OUTPUT.parent.iterdir():\n", " if stale_file.is_file():\n", " stale_file.unlink()\n", " static_model = onnx.load(str(FASTSAC_ONNX))\n", " for value in list(static_model.graph.input) + list(static_model.graph.output):\n", " value.type.tensor_type.shape.dim[0].dim_value = 1\n", " value.type.tensor_type.shape.dim[0].ClearField(\"dim_param\")\n", " onnx.checker.check_model(static_model)\n", " onnx.save(static_model, str(BPU_STATIC_ONNX))\n", " BPU_CONFIG.write_text(\n", " \"\"\"calibration_parameters:\n", " cal_data_type: ''\n", " calibration_type: skip\n", " optimization: run_fast\n", " per_channel: false\n", " preprocess_on: false\n", " run_on_bpu: ''\n", " run_on_cpu: ''\n", "compiler_parameters:\n", " compile_mode: latency\n", " core_num: 1\n", " debug: true\n", " jobs: 0\n", " max_time_per_fc: 0\n", " optimize_level: O3\n", "input_parameters:\n", " input_layout_rt: NCHW\n", " input_layout_train: NCHW\n", " input_name: obs\n", " input_shape: 1x54\n", " input_space_and_range: ''\n", " input_type_rt: featuremap\n", " input_type_train: featuremap\n", " norm_type: no_preprocess\n", "model_parameters:\n", " layer_out_dump: false\n", " march: bayes-e\n", " onnx_model: /work/microduck_bpu_models/converted/motrix_ball_balance.static.opset11.onnx\n", " output_model_file_prefix: ball_balance.opset11\n", " remove_node_type: Quantize;Transpose;Dequantize;Cast;Reshape;Softmax;DequantizeFilter\n", " set_node_data_type: {}\n", " working_dir: /work/microduck_bpu_models/compiled/ball_balance/model_output\n", "\"\"\",\n", " encoding=\"utf-8\",\n", " )\n", " docker_cmd = [\n", " \"docker\", \"run\", \"--rm\", \"-v\", \"/home/ubuntu/workspaces:/work\",\n", " BPU_DOCKER_IMAGE, \"sh\", \"-lc\",\n", " \"hb_mapper makertbin --config \"\n", " + shlex.quote(str(workspace_container_path(BPU_CONFIG)))\n", " + \" --model-type onnx\",\n", " ]\n", " print(\"开始 X5 HBM 编译:\", \" \".join(shlex.quote(x) for x in docker_cmd))\n", " compile_result = subprocess.run(docker_cmd, text=True)\n", " if compile_result.returncode != 0 or not BPU_OUTPUT.is_file():\n", " raise RuntimeError(\"X5 HBM 编译失败。\")\n", " os.environ[\"MICRODUCK_BPU_HBM\"] = str(BPU_OUTPUT)\n", " print(\"PASS-X5-HBM:\", BPU_OUTPUT)" ] }, { "cell_type": "markdown", "id": "e258fced", "metadata": {}, "source": [ "## 4. 本地 ONNX 基准推理\n", "\n", "这一步只验证 ONNX 图能在本机运行，以及输出形状和数值是否有限；它不是训练效果评测，也不是 BPU 验收。" ] }, { "cell_type": "code", "execution_count": 7, "id": "03d27fcc", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:34.584231Z", "iopub.status.busy": "2026-09-19T08:21:34.584123Z", "iopub.status.idle": "2026-09-19T08:21:34.586828Z", "shell.execute_reply": "2026-09-19T08:21:34.586649Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "跳过：尚未提供该任务 ONNX。\n" ] } ], "source": [ "def onnx_local_smoke(path):\n", " try:\n", " import numpy as np\n", " import onnxruntime as ort\n", " except ImportError as exc:\n", " print(\"缺少本地推理依赖:\", exc)\n", " return None\n", " session = ort.InferenceSession(str(path), providers=[\"CPUExecutionProvider\"])\n", " feeds = {}\n", " for item in session.get_inputs():\n", " shape = [dim if isinstance(dim, int) and dim > 0 else 1 for dim in item.shape]\n", " feeds[item.name] = np.zeros(shape, dtype=np.float32)\n", " outputs = session.run(None, feeds)\n", " info = {\"providers\": session.get_providers(), \"inputs\": [(x.name, x.shape) for x in session.get_inputs()], \"outputs\": [(x.name, x.shape) for x in session.get_outputs()], \"finite\": all(np.isfinite(x).all() for x in outputs)}\n", " print(json.dumps(info, ensure_ascii=False, indent=2, default=str))\n", " return outputs\n", "\n", "if MODEL_PATH:\n", " local_outputs = onnx_local_smoke(MODEL_PATH)\n", "else:\n", " print(\"跳过：尚未提供该任务 ONNX。\")" ] }, { "cell_type": "markdown", "id": "18a0cd0c", "metadata": {}, "source": [ "## 5. GPU 推理视频\n", "\n", "本单元先生成 GPU 推理视频，再生成后面的 RDK X5 BPU 视频。Ubuntu 负责 MuJoCo/MotrixSim 物理和渲染；策略 actor 使用 CUDA 上的 Torch 执行导出的 ONNX 图。这里只展示“GPU 推理”和任务结果，不暴露具体显卡型号。视频文件写入 `outputs/gpu_videos/`。\n", "\n", "除扰动回放明确复用 walking 策略外，每个任务都使用自己的 ONNX；篮球使用带 LSTM 的篮球 actor，球平衡使用 FastSAC actor，爬梯 GPU 回放使用 climber/get-up 两个 actor。" ] }, { "cell_type": "code", "execution_count": 8, "id": "9bd39818", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:34.587547Z", "iopub.status.busy": "2026-09-19T08:21:34.587484Z", "iopub.status.idle": "2026-09-19T08:21:40.445013Z", "shell.execute_reply": "2026-09-19T08:21:40.444719Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GPU 推理任务: ball_balance\n", "GPU 模型: /home/ubuntu/workspaces/microduck_bpu_models/converted/motrix_ball_balance.opset11.onnx\n", "开始生成 GPU 推理视频。\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "GPU 策略推理已启用；不展示具体显卡型号。\n", "PASS-GPU-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/ball_balance_gpu_latest.mp4\n", "PASS-GPU-REPORT: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/ball_balance_gpu_latest.json\n", "PASS-GPU-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/ball_balance_gpu_latest.mp4\n", "{'task': 'ball_balance', 'task_id': 'microduck-ball-balance', 'display_name': 'Motrix FastSAC ball balance', 'backend': 'GPU ONNX via Torch CUDA', 'policy_device': 'GPU', 'renderer': 'Ubuntu MotrixSim headless renderer', 'onnx_file': '/home/ubuntu/workspaces/microduck_bpu_models/converted/motrix_ball_balance.opset11.onnx', 'onnx_sha256': 'c5565bc8406d9526bccd8bfea13c10ac9f72bfe6607e3984a8eb2019f0af0266', 'observation_dim': 54, 'action_dim': 14, 'steps': 250, 'video_frames': 150, 'resets': 6, 'mean_inference_ms': 4.551633831957588, 'p95_inference_ms': 17.038467000747914, 'note': 'MotrixSim renders on Ubuntu; the FastSAC actor is evaluated by Torch on GPU.'}\n" ] }, { "data": { "text/html": [ " $\"GPU$ " ], "text/plain": [ "GIF 预览：../01-任务资料/精选视频/assets/notebook-replays/ball_balance_gpu.gif；原始 MP4 未嵌入 Notebook，以控制仓库体积。" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import sys\n", "\n", "GPU_TASK = 'ball_balance'\n", "GPU_VIDEO_STEPS = int(os.getenv(\"MICRODUCK_GPU_VIDEO_STEPS\", \"250\"))\n", "GPU_VIDEO_OUTPUT = OUTPUT_ROOT / \"gpu_videos\" / f\"{GPU_TASK}_gpu_latest.mp4\" if GPU_TASK else None\n", "GPU_ONNX_DEFAULTS = {\n", " \"basketball\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/basketball.opset11.onnx\",\n", " \"walking\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx\",\n", " \"perturbation\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx\",\n", " \"stilt\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/stilts25.opset11.onnx\",\n", " \"swing\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/swing.opset11.onnx\",\n", " \"ball_balance\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/motrix_ball_balance.opset11.onnx\",\n", "}\n", "GPU_ONNX = Path(os.getenv(\"MICRODUCK_GPU_ONNX\", GPU_ONNX_DEFAULTS.get(GPU_TASK, \"\"))).expanduser() if GPU_TASK else None\n", "GPU_LADDER_CLIMBER = Path(os.getenv(\"MICRODUCK_GPU_LADDER_CLIMBER\", \"/home/ubuntu/workspaces/microduck_bpu_models/converted/official_ladder_climber.opset11.onnx\")).expanduser()\n", "GPU_LADDER_GETUP = Path(os.getenv(\"MICRODUCK_GPU_LADDER_GETUP\", \"/home/ubuntu/workspaces/microduck_bpu_models/converted/official_ladder_getup.opset11.onnx\")).expanduser()\n", "MOTRIX_ROOT = Path(os.getenv(\"MOTRIX_ROOT\", \"/home/ubuntu/workspaces/MotrixLab\")).expanduser() if GPU_TASK == \"ball_balance\" else None\n", "MOTRIX_PYTHON = Path(os.getenv(\"MOTRIX_PYTHON\", str(MOTRIX_ROOT / \".venv/bin/python\"))).expanduser() if MOTRIX_ROOT else None\n", "MOTRIX_RUNNER = PLAYGROUND_ROOT / \"scripts\" / \"motrix_runner.py\" if GPU_TASK == \"ball_balance\" else None\n", "GPU_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / (\"motrix_gpu_inloop_video.py\" if GPU_TASK == \"ball_balance\" else \"gpu_inloop_video.py\")\n", "GPU_LADDER_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / \"bpu_official_ladder_pair_video.py\"\n", "\n", "if GPU_TASK == \"ladder\":\n", " gpu_ready = GPU_LADDER_SCRIPT.is_file() and GPU_LADDER_CLIMBER.is_file() and GPU_LADDER_GETUP.is_file()\n", " gpu_cmd = [\n", " sys.executable, str(GPU_LADDER_SCRIPT), \"--backend\", \"gpu\",\n", " \"--official-root\", \"/home/ubuntu/workspaces/microduck-playground-official/experiments/desk-climb\",\n", " \"--climber-onnx\", str(GPU_LADDER_CLIMBER), \"--getup-onnx\", str(GPU_LADDER_GETUP),\n", " \"--output\", str(GPU_VIDEO_OUTPUT), \"--seconds\", \"24\", \"--num-envs\", \"64\", \"--record-env\", \"17\",\n", " ]\n", "else:\n", " gpu_ready = GPU_SCRIPT.is_file() and GPU_ONNX is not None and GPU_ONNX.is_file()\n", " gpu_cmd = [sys.executable, str(GPU_SCRIPT)] if GPU_TASK != \"ball_balance\" else [str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"video\", str(GPU_SCRIPT)]\n", " if GPU_TASK != \"ball_balance\":\n", " gpu_cmd += [\"--task\", GPU_TASK]\n", " gpu_cmd += [\"--onnx\", str(GPU_ONNX), \"--output\", str(GPU_VIDEO_OUTPUT), \"--steps\", str(GPU_VIDEO_STEPS)]\n", "\n", "print(\"GPU 推理任务:\", GPU_TASK)\n", "print(\"GPU 模型:\", GPU_ONNX if GPU_TASK != \"ladder\" else (GPU_LADDER_CLIMBER, GPU_LADDER_GETUP))\n", "if not gpu_ready:\n", " print(\"GPU 回放暂不可运行：请检查任务专属 ONNX、脚本和 Motrix 环境。\")\n", "elif GPU_TASK == \"ball_balance\" and (MOTRIX_PYTHON is None or not MOTRIX_PYTHON.is_file()):\n", " print(\"找不到 MotrixLab Python 环境:\", MOTRIX_PYTHON)\n", "else:\n", " GPU_VIDEO_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " print(\"开始生成 GPU 推理视频。\")\n", " gpu_result = subprocess.run(gpu_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(), capture_output=True, text=True)\n", " safe_lines = [line for line in (gpu_result.stdout + \"\\n\" + gpu_result.stderr).splitlines() if \"GPU\" in line or \"PASS-\" in line or \"Video\" in line]\n", " print(\"\\n\".join(safe_lines[-20:]))\n", " if gpu_result.returncode != 0 or not GPU_VIDEO_OUTPUT.is_file():\n", " raise RuntimeError(\"GPU 推理视频生成失败，请查看脚本日志。\")\n", " GPU_REPORT = GPU_VIDEO_OUTPUT.with_suffix(\".json\")\n", " print(\"PASS-GPU-VIDEO:\", GPU_VIDEO_OUTPUT)\n", " if GPU_REPORT.is_file():\n", " print(json.loads(GPU_REPORT.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(GPU_VIDEO_OUTPUT), embed=True))" ] }, { "cell_type": "markdown", "id": "685204e6", "metadata": {}, "source": [ "## 6. RDK X5 BPU-in-the-loop 视频\n", "\n", "本单元只展示新生成的板端闭环 MP4，不读取参考 GIF 或关键帧。Ubuntu 运行 MuJoCo 物理和 EGL 渲染；每一个控制步把任务观测发送到 RDK，RDK 用当前任务对应的 HBM 在 BPU 上计算 14 维动作，再返回给 Ubuntu。视频因此是“板端 BPU 产生动作、Ubuntu 负责仿真画面”的真实闭环。\n", "\n", "运行前会自动重启 RDK 上的任务策略服务。切换 Notebook 时必须重新运行本单元，让板端 HBM 与当前任务匹配。" ] }, { "cell_type": "code", "execution_count": 9, "id": "4b3e2bdf", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:40.446008Z", "iopub.status.busy": "2026-09-19T08:21:40.445944Z", "iopub.status.idle": "2026-09-19T08:21:46.596329Z", "shell.execute_reply": "2026-09-19T08:21:46.595955Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RDK BPU 控制服务不可用，尝试 SSH: [Errno 113] No route to host\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RDK 策略服务启动失败: ssh: connect to host 192.168.8.128 port 22: No route to host\n", "BPU-CACHED-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/bpu_videos/ball_balance_bpu_latest.mp4\n", "说明: 本轮板端不可达，保留此前真实 RDK X5 BPU 任务视频。\n", "{'task': 'ball_balance', 'task_id': 'microduck-ball-balance', 'display_name': 'Motrix FastSAC ball balance', 'backend': 'RDK X5 BPU via hbm_runtime', 'renderer': 'Ubuntu MotrixSim headless renderer', 'bpu_host': '192.168.8.128', 'bpu_port': 8765, 'hbm_file': '/home/ubuntu/workspaces/microduck_bpu_models/compiled/ball_balance/model_output/ball_balance.opset11.bin', 'hbm_sha256': 'b566d7beda7a70e3b46aa9a2659534d3e85101620342eb46ab0ea4e2ad3e8974', 'observation_dim_sent': 54, 'action_dim': 14, 'steps': 250, 'video_frames': 150, 'resets': 4, 'mean_round_trip_ms': 9.3907775208354, 'p95_round_trip_ms': 26.43653471022844, 'note': 'MotrixSim renders on Ubuntu; RDK X5 BPU produces every FastSAC actor action.'}\n" ] }, { "data": { "text/html": [ " $\"历史缓存$ " ], "text/plain": [ "GIF 预览：../01-任务资料/精选视频/assets/notebook-replays/ball_balance_bpu_cached.gif；原始 MP4 未嵌入 Notebook，以控制仓库体积。" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "MuJoCo 回放命令模板: python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11\n" ] } ], "source": [ "import hashlib\n", "import sys\n", "\n", "BPU_TASK = 'ball_balance'\n", "BPU_LADDER_LOCAL_OPT_IN = os.getenv(\"MICRODUCK_RUN_LOCAL_LADDER_BPU\", \"0\") == \"1\"\n", "BPU_VIDEO_STEPS = int(os.getenv(\"MICRODUCK_BPU_VIDEO_STEPS\", \"250\"))\n", "BPU_VIDEO_WIDTH = int(os.getenv(\"MICRODUCK_BPU_VIDEO_WIDTH\", \"640\"))\n", "BPU_VIDEO_HEIGHT = int(os.getenv(\"MICRODUCK_BPU_VIDEO_HEIGHT\", \"480\"))\n", "BPU_CAMERA_DISTANCE = os.getenv(\"MICRODUCK_BPU_CAMERA_DISTANCE\", \"1.2\")\n", "BPU_CAMERA_ELEVATION = os.getenv(\"MICRODUCK_BPU_CAMERA_ELEVATION\", \"-8\")\n", "BPU_CAMERA_AZIMUTH = os.getenv(\"MICRODUCK_BPU_CAMERA_AZIMUTH\", \"90\")\n", "BPU_PORT = int(os.getenv(\"MICRODUCK_BPU_PORT\", \"8765\"))\n", "RDK_CONTROL_PORT = int(os.getenv(\"MICRODUCK_BPU_CONTROL_PORT\", \"8766\"))\n", "RDK_HOST = os.getenv(\"RDK_HOST\", \"192.168.8.128\")\n", "BPU_HBM_DEFAULTS = {\n", " \"ball_balance\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/ball_balance/model_output/ball_balance.opset11.bin\",\n", " \"basketball\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/basketball/model_output/basketball.opset11.bin\",\n", " \"walking\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/walking/model_output/walking.opset11.bin\",\n", " \"perturbation\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/walking/model_output/walking.opset11.bin\",\n", " \"stilt\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/stilts25/model_output/stilts25.opset11.bin\",\n", " \"swing\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/swing/model_output/swing.opset11.bin\",\n", " \"ladder\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/ladder/model_output/ladder.opset11.bin\",\n", "}\n", "RDK_HBM_DEFAULTS = {\n", " \"ball_balance\": \"/home/sunrise/microduck_policy/hbm/ball_balance.bin\",\n", " \"basketball\": \"/home/sunrise/microduck_policy/hbm/basketball.bin\",\n", " \"walking\": \"/home/sunrise/microduck_policy/hbm/walking.bin\",\n", " \"perturbation\": \"/home/sunrise/microduck_policy/hbm/walking.bin\",\n", " \"stilt\": \"/home/sunrise/microduck_policy/hbm/stilts25.bin\",\n", " \"swing\": \"/home/sunrise/microduck_policy/hbm/swing.bin\",\n", " \"ladder\": \"/home/sunrise/microduck_policy/hbm/ladder.bin\",\n", "}\n", "BPU_HBM = Path(os.getenv(\"MICRODUCK_BPU_HBM\", BPU_HBM_DEFAULTS.get(BPU_TASK, \"\"))) if BPU_TASK else None\n", "RDK_HBM = RDK_HBM_DEFAULTS.get(BPU_TASK, \"\") if BPU_TASK else \"\"\n", "RDK_SERVER_SCRIPT = os.getenv(\"RDK_BPU_SERVER_SCRIPT\", \"/home/sunrise/microduck_policy/rdk_bpu_policy_server.py\")\n", "MOTRIX_ROOT = Path(os.getenv(\"MOTRIX_ROOT\", \"/home/ubuntu/workspaces/MotrixLab\")).expanduser() if BPU_TASK == \"ball_balance\" else None\n", "MOTRIX_PYTHON = Path(os.getenv(\"MOTRIX_PYTHON\", str(MOTRIX_ROOT / \".venv/bin/python\"))).expanduser() if MOTRIX_ROOT else None\n", "MOTRIX_RUNNER = PLAYGROUND_ROOT / \"scripts\" / \"motrix_runner.py\" if BPU_TASK == \"ball_balance\" else None\n", "BPU_VIDEO_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / (\"motrix_bpu_inloop_video.py\" if BPU_TASK == \"ball_balance\" else \"bpu_inloop_video.py\")\n", "BPU_VIDEO_OUTPUT = OUTPUT_ROOT / \"bpu_videos\" / f\"{BPU_TASK}_bpu_latest.mp4\" if BPU_TASK else None\n", "ALLOW_CACHED_BPU = os.getenv(\"MICRODUCK_ALLOW_CACHED_BPU\", \"0\") == \"1\"\n", "BPU_LADDER_PAIR_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / \"bpu_official_ladder_pair_video.py\"\n", "BPU_LADDER_CLIMBER_HBM = Path(os.getenv(\"MICRODUCK_BPU_LADDER_CLIMBER_HBM\", \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/official_ladder_climber/model_output/official_ladder_climber.bin\")).expanduser()\n", "BPU_LADDER_GETUP_HBM = Path(os.getenv(\"MICRODUCK_BPU_LADDER_GETUP_HBM\", \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/official_ladder_getup/model_output/official_ladder_getup.bin\")).expanduser()\n", "BPU_LADDER_PAIR_OUTPUT = OUTPUT_ROOT / \"bpu_videos\" / \"ladder_official_pair_bpu_latest.mp4\"\n", "\n", "def show_cached_bpu_video(path, reason):\n", " if not ALLOW_CACHED_BPU or path is None or not path.is_file():\n", " return False\n", " print(\"BPU-CACHED-VIDEO:\", path)\n", " print(\"说明:\", reason)\n", " report = path.with_suffix(\".json\")\n", " if report.is_file():\n", " print(json.loads(report.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(path), embed=True))\n", " return True\n", "\n", "def restart_rdk_policy_server():\n", " if not BPU_TASK or not RDK_HBM:\n", " return False\n", " try:\n", " if BPU_HBM is not None and BPU_HBM.is_file():\n", " payload = BPU_HBM.read_bytes()\n", " request = {\n", " \"op\": \"upload_and_switch\",\n", " \"task\": BPU_TASK,\n", " \"size\": len(payload),\n", " \"sha256\": hashlib.sha256(payload).hexdigest(),\n", " }\n", " else:\n", " payload = b\"\"\n", " request = {\"op\": \"switch\", \"task\": BPU_TASK}\n", " with socket.create_connection((RDK_HOST, RDK_CONTROL_PORT), timeout=8) as control:\n", " control.sendall((json.dumps(request) + \"\\n\").encode(\"utf-8\"))\n", " if payload:\n", " for offset in range(0, len(payload), 1024 * 1024):\n", " control.sendall(payload[offset:offset + 1024 * 1024])\n", " response = json.loads(control.makefile(\"rb\").readline().decode(\"utf-8\"))\n", " if response.get(\"ok\"):\n", " action = \"已上传并切换\" if payload else \"已切换\"\n", " print(\"RDK 任务策略服务\" + action + \":\", BPU_TASK, \"->\", response.get(\"model\"), \"pid=\", response.get(\"pid\"))\n", " return True\n", " print(\"RDK BPU 控制服务拒绝切换:\", response)\n", " except (OSError, ValueError, json.JSONDecodeError) as exc:\n", " print(\"RDK BPU 控制服务不可用，尝试 SSH:\", exc)\n", " remote = \"sunrise@\" + RDK_HOST\n", " command = (\n", " \"pkill -f '^python3 .*rdk_bpu_policy_server.py' >/dev/null 2>&1 || true; \"\n", " \"sleep 1; \"\n", " \"nohup python3 \" + shlex.quote(RDK_SERVER_SCRIPT) +\n", " \" --model \" + shlex.quote(RDK_HBM) +\n", " \" --port \" + str(BPU_PORT) +\n", " \" >/tmp/microduck_bpu_server.log 2>&1 \", RDK_HBM)\n", " return True\n", "\n", "if BPU_TASK == \"ladder\":\n", " pair_ready = BPU_LADDER_PAIR_SCRIPT.is_file() and BPU_LADDER_CLIMBER_HBM.is_file() and BPU_LADDER_GETUP_HBM.is_file()\n", " pair_cmd = [\n", " sys.executable, str(BPU_LADDER_PAIR_SCRIPT), \"--backend\", \"bpu\",\n", " \"--climber-hbm\", str(BPU_LADDER_CLIMBER_HBM), \"--getup-hbm\", str(BPU_LADDER_GETUP_HBM),\n", " \"--output\", str(BPU_LADDER_PAIR_OUTPUT), \"--seconds\", os.getenv(\"MICRODUCK_LADDER_SECONDS\", \"24\"),\n", " \"--num-envs\", \"64\", \"--record-env\", \"17\",\n", " ]\n", " print(\"BPU 官方双模型:\", BPU_LADDER_CLIMBER_HBM, BPU_LADDER_GETUP_HBM)\n", " if not pair_ready:\n", " print(\"官方爬梯 BPU 双模型视频暂不可运行：请检查 climber/get-up HBM 和脚本。\")\n", " else:\n", " BPU_LADDER_PAIR_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " pair_result = subprocess.run(pair_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(), capture_output=True, text=True)\n", " print(\"\\n\".join((pair_result.stdout + \"\\n\" + pair_result.stderr).splitlines()[-35:]))\n", " if pair_result.returncode != 0 or not BPU_LADDER_PAIR_OUTPUT.is_file():\n", " if not show_cached_bpu_video(BPU_LADDER_PAIR_OUTPUT, \"本轮板端不可达，保留此前真实 RDK X5 BPU 双模型视频。\"):\n", " raise RuntimeError(\"官方爬梯 BPU 双模型视频生成失败。\")\n", " else:\n", " print(\"PASS-BPU-VIDEO:\", BPU_LADDER_PAIR_OUTPUT)\n", " pair_report = BPU_LADDER_PAIR_OUTPUT.with_suffix(\".json\")\n", " if pair_report.is_file():\n", " print(json.loads(pair_report.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(BPU_LADDER_PAIR_OUTPUT), embed=True))\n", "elif not BPU_TASK:\n", " print(\"该任务目前没有可验证的任务专属 HBM；不会用其他任务视频替代。\")\n", "elif BPU_TASK == \"ladder\" and not BPU_LADDER_LOCAL_OPT_IN:\n", " print(\"已跳过本地 14 级 ladder HBM 视频：它不是社区 climber/get-up 方案。\")\n", " print(\"只有明确设置 MICRODUCK_RUN_LOCAL_LADDER_BPU=1 才会运行本地实验，不得将结果标为社区复刻。\")\n", "elif BPU_HBM is None or not BPU_HBM.is_file():\n", " print(\"Ubuntu 端 HBM 不存在:\", BPU_HBM)\n", "elif not BPU_VIDEO_SCRIPT.is_file():\n", " print(\"找不到板端闭环录制器:\", BPU_VIDEO_SCRIPT)\n", "elif BPU_TASK == \"ball_balance\" and (MOTRIX_PYTHON is None or not MOTRIX_PYTHON.is_file()):\n", " print(\"找不到 MotrixLab Python 环境:\", MOTRIX_PYTHON)\n", "elif BPU_TASK == \"ball_balance\" and (MOTRIX_RUNNER is None or not MOTRIX_RUNNER.is_file()):\n", " print(\"找不到 MotrixLab 启动器:\", MOTRIX_RUNNER)\n", "elif restart_rdk_policy_server():\n", " BPU_VIDEO_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " if BPU_TASK == \"ball_balance\":\n", " video_cmd = [str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"video\", str(BPU_VIDEO_SCRIPT)]\n", " else:\n", " video_cmd = [sys.executable, str(BPU_VIDEO_SCRIPT)]\n", " if BPU_TASK != \"ball_balance\":\n", " video_cmd += [\"--task\", BPU_TASK]\n", " video_cmd += [\n", " \"--bpu-host\", RDK_HOST, \"--bpu-port\", str(BPU_PORT),\n", " \"--hbm\", str(BPU_HBM), \"--output\", str(BPU_VIDEO_OUTPUT),\n", " \"--steps\", str(BPU_VIDEO_STEPS), \"--width\", str(BPU_VIDEO_WIDTH),\n", " \"--height\", str(BPU_VIDEO_HEIGHT),\n", " ]\n", " video_cmd += [\n", " \"--camera-distance\", BPU_CAMERA_DISTANCE,\n", " \"--camera-elevation\", BPU_CAMERA_ELEVATION,\n", " \"--camera-azimuth\", BPU_CAMERA_AZIMUTH,\n", " ]\n", " print(\"开始生成板端闭环视频:\", \" \".join(shlex.quote(x) for x in video_cmd))\n", " video_result = subprocess.run(\n", " video_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(),\n", " capture_output=True, text=True,\n", " )\n", " print(\"\\n\".join((video_result.stdout + \"\\n\" + video_result.stderr).splitlines()[-35:]))\n", " if video_result.returncode != 0 or not BPU_VIDEO_OUTPUT.is_file():\n", " raise RuntimeError(\"BPU-in-the-loop 视频生成失败，请先查看上面最后 35 行日志。\")\n", " BPU_REPORT = BPU_VIDEO_OUTPUT.with_suffix(\".json\")\n", " print(\"PASS-BPU-VIDEO:\", BPU_VIDEO_OUTPUT)\n", " if BPU_REPORT.is_file():\n", " print(json.loads(BPU_REPORT.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(BPU_VIDEO_OUTPUT), embed=True))\n", "else:\n", " show_cached_bpu_video(BPU_VIDEO_OUTPUT, \"本轮板端不可达，保留此前真实 RDK X5 BPU 任务视频。\")\n", "\n", "SIM_COMMAND = 'python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11'\n", "print(\"MuJoCo 回放命令模板:\", SIM_COMMAND)" ] }, { "cell_type": "markdown", "id": "29a5db8f", "metadata": {}, "source": [ "## 7. RDK X5 / BPU 审计\n", "\n", "上一单元已经生成任务专属闭环视频。本单元只检查板端连通、HBM 元数据和服务日志；通用 MobileNet 样例只能证明系统运行时存在，不能代替任务视频。" ] }, { "cell_type": "code", "execution_count": 10, "id": "458c0818", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:21:46.597170Z", "iopub.status.busy": "2026-09-19T08:21:46.597108Z", "iopub.status.idle": "2026-09-19T08:21:49.663724Z", "shell.execute_reply": "2026-09-19T08:21:49.663388Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "默认 BPU 样例模型: /opt/tros/humble/lib/dnn_benchmark_example/config/X5/mobilenetv1_224x224_nv12.bin\n", "默认 BPU 样例输入: 75264 bytes uint8\n", "MicroDuck 策略 HBM: 该任务尚未提供\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RDK 暂不可达，跳过板端验收: ssh: connect to host 192.168.8.128 port 22: No route to host\n" ] } ], "source": [ "import time\n", "\n", "RDK_HOST = os.getenv(\"RDK_HOST\", \"192.168.8.128\")\n", "BPU_SMOKE_MODEL = os.getenv(\n", " \"RDK_BPU_SMOKE_MODEL\",\n", " \"/opt/tros/humble/lib/dnn_benchmark_example/config/X5/mobilenetv1_224x224_nv12.bin\",\n", ")\n", "BPU_SMOKE_INPUT_BYTES = int(os.getenv(\"RDK_BPU_SMOKE_INPUT_BYTES\", \"75264\"))\n", "BPU_HBM = os.getenv(\"RDK_BPU_HBM\", \"\")\n", "print(\"默认 BPU 样例模型:\", BPU_SMOKE_MODEL)\n", "print(\"默认 BPU 样例输入:\", BPU_SMOKE_INPUT_BYTES, \"bytes uint8\")\n", "print(\"MicroDuck 策略 HBM:\", BPU_HBM or \"该任务尚未提供\")\n", "probe_script = \"for x in hrt_model_exec hb_mapper hrt_bin_dump hrt_bin_info python3; do if command -v $x >/dev/null 2>&1; then echo $x=$(command -v $x); else echo $x=MISSING; fi; done\"\n", "remote = \"sunrise@\" + RDK_HOST\n", "probe = subprocess.run([\"ssh\", \"-o\", \"BatchMode=yes\", \"-o\", \"ConnectTimeout=5\", remote, \"bash -lc \" + shlex.quote(probe_script)], capture_output=True, text=True)\n", "if probe.returncode:\n", " print(\"RDK 暂不可达，跳过板端验收:\", probe.stderr.strip() or \"ssh failed\")\n", "else:\n", " print(\"RDK 已连通:\", RDK_HOST)\n", " print(probe.stdout)\n", " if not BPU_HBM:\n", " print(\"没有任务 HBM：不执行通用样例冒充任务推理。\")\n", " smoke_model = BPU_SMOKE_MODEL\n", " smoke_bytes = BPU_SMOKE_INPUT_BYTES\n", " smoke_info_cmd = \"hrt_model_exec model_info --model_file \" + shlex.quote(smoke_model)\n", " smoke_info = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(smoke_info_cmd)], capture_output=True, text=True)\n", " print(\"BPU 样例 model_info exit:\", smoke_info.returncode)\n", " print(smoke_info.stdout[-4000:])\n", " print(smoke_info.stderr[-1000:])\n", " if smoke_info.returncode == 0:\n", " import numpy as np\n", " local_input = OUTPUT_ROOT / \"rdk_bpu_runtime_smoke.bin\"\n", " np.zeros(smoke_bytes, dtype=np.uint8).tofile(local_input)\n", " remote_dir = \"/tmp/microduck_notebook\"\n", " remote_input = remote_dir + \"/rdk_bpu_runtime_smoke.bin\"\n", " subprocess.run([\"ssh\", remote, \"mkdir\", \"-p\", remote_dir], check=True)\n", " subprocess.run([\"scp\", str(local_input), remote + \":\" + remote_input], check=True)\n", " dump_dir = remote_dir + \"/runtime_dump\"\n", " infer_cmd = \"rm -rf \" + shlex.quote(dump_dir) + \" && mkdir -p \" + shlex.quote(dump_dir) + \" && hrt_model_exec infer --model_file \" + shlex.quote(smoke_model) + \" --input_file \" + shlex.quote(remote_input) + \" --enable_dump true --dump_format txt --dump_path \" + shlex.quote(dump_dir)\n", " started = time.perf_counter()\n", " infer = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(infer_cmd)], capture_output=True, text=True)\n", " elapsed_ms = (time.perf_counter() - started) * 1000.0\n", " print(\"BPU runtime sample exit:\", infer.returncode, \"round-trip ms:\", round(elapsed_ms, 2))\n", " print(infer.stdout[-4000:])\n", " print(infer.stderr[-1000:])\n", " if infer.returncode == 0:\n", " print(\"PASS-rdk-bpu-runtime-sample: X5 BPU 样例推理已返回 0。\")\n", " else:\n", " info_cmd = \"hrt_model_exec model_info --model_file \" + shlex.quote(BPU_HBM)\n", " info = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(info_cmd)], capture_output=True, text=True)\n", " print(\"HBM model_info exit:\", info.returncode)\n", " print(info.stdout[-6000:])\n", " print(info.stderr[-2000:])\n", " if info.returncode == 0:\n", " # hrt_model_exec consumes binary tensors; this zero observation is\n", " # only a transport/BPU smoke input, not a task-quality evaluation.\n", " import numpy as np\n", " local_input = OUTPUT_ROOT / \"zero_obs_61_f32.bin\"\n", " np.zeros((1, 61), dtype=np.float32).tofile(local_input)\n", " remote_dir = \"/tmp/microduck_notebook\"\n", " remote_input = remote_dir + \"/zero_obs_61_f32.bin\"\n", " subprocess.run([\"ssh\", remote, \"mkdir\", \"-p\", remote_dir], check=True)\n", " subprocess.run([\"scp\", str(local_input), remote + \":\" + remote_input], check=True)\n", " dump_dir = remote_dir + \"/dump\"\n", " infer_cmd = \"rm -rf \" + shlex.quote(dump_dir) + \" && mkdir -p \" + shlex.quote(dump_dir) + \" && hrt_model_exec infer --model_file \" + shlex.quote(BPU_HBM) + \" --input_file \" + shlex.quote(remote_input) + \" --enable_dump true --dump_format txt --dump_path \" + shlex.quote(dump_dir)\n", " started = time.perf_counter()\n", " infer = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(infer_cmd)], capture_output=True, text=True)\n", " elapsed_ms = (time.perf_counter() - started) * 1000.0\n", " print(\"BPU infer exit:\", infer.returncode, \"round-trip ms:\", round(elapsed_ms, 2))\n", " print(infer.stdout[-6000:])\n", " print(infer.stderr[-2000:])\n", " if infer.returncode == 0:\n", " print(\"PASS-rdk-bpu: hrt_model_exec infer 已返回 0。\")\n", " else:\n", " print(\"未通过：优先以第 5 节 BPU-in-the-loop 报告为准。\")" ] }, { "cell_type": "markdown", "id": "f046c866", "metadata": {}, "source": [ "## 8. 直播结论\n", "\n", "- `PASS-local-onnx`：ONNX checker 和本地 ONNX Runtime 通过。\n", "- `PASS-GPU-VIDEO`：GPU actor 推理视频已生成，且排在 BPU 视频之前。\n", "- `PASS-BPU-VIDEO`：Ubuntu MuJoCo 视频中的每一个动作都由 RDK X5 对应 HBM 产生。\n", "- `PASS-rdk-bpu-runtime-sample`：板端已安装的 X5 BPU 样例推理退出码为 0。\n", "- `PASS-rdk-bpu`：SSH 连通、板端明确选择 BPU、MicroDuck HBM 推理命令退出码为 0，并记录输入输出形状与延迟。\n", "\n", "没有 `PASS-BPU-VIDEO` 的任务，只能算“已有素材/接口说明”，不能算板端视频已完成。\\n\\nBPU status rule: PASS-BPU-VIDEO means a current online RDK run. BPU-CACHED-VIDEO means a previous real BPU video is shown while the board is offline; rerun this cell after RDK is online." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.14" } }, "nbformat": 4, "nbformat_minor": 5 }