{ "cells": [ { "cell_type": "markdown", "id": "7ffc632e", "metadata": {}, "source": [ "# MicroDuck 直播 Notebook：浏览器物理扰动 / MuJoCo Web\n", "\n", "把任务、策略、ONNX、MuJoCo 和 RDK X5/BPU 验收串成一条可复用流程。" ] }, { "cell_type": "markdown", "id": "e01ab554", "metadata": {}, "source": [ "## 0. 运行说明\n", "\n", "建议在 `02-可运行代码/microduck-playground-stilts` 的 Python 环境中启动 Jupyter。训练模型和板端 HBM 不随 Git 提交；通过 `MICRODUCK_ONNX`、`RDK_BPU_HBM` 和 `RDK_HOST` 注入。" ] }, { "cell_type": "code", "execution_count": 1, "id": "03021047", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:24.924723Z", "iopub.status.busy": "2026-09-19T08:19:24.924618Z", "iopub.status.idle": "2026-09-19T08:19:24.928140Z", "shell.execute_reply": "2026-09-19T08:19:24.927952Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "专题目录: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题\n", "可运行代码: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/02-可运行代码/microduck-playground-stilts\n", "输出目录: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs\n", "Jupyter 工作站: ubuntu Linux x86_64\n", "RDK 地址: 192.168.8.128 用户: sunrise\n" ] } ], "source": [ "from pathlib import Path\n", "import json\n", "import os\n", "import platform\n", "import shlex\n", "import socket\n", "import subprocess\n", "\n", "\n", "def locate_topic_root():\n", " cwd = Path.cwd().resolve()\n", " for base in (cwd, *cwd.parents):\n", " if (base / \"02-可运行代码\").is_dir() and (base / \"01-任务资料\").is_dir():\n", " return base\n", " group = base / \"16-专题组队学习\"\n", " if group.is_dir():\n", " matches = list(group.glob(\"*/02-可运行代码\"))\n", " if matches:\n", " return matches[0].parent\n", " configured = os.getenv(\"MICRODUCK_TOPIC_ROOT\")\n", " if configured and Path(configured).is_dir():\n", " return Path(configured).resolve()\n", " raise RuntimeError(\"找不到专题目录，请从 03-Notebook 启动 Jupyter，或设置 MICRODUCK_TOPIC_ROOT。\")\n", "\n", "\n", "TOPIC_ROOT = locate_topic_root()\n", "PLAYGROUND_ROOT = TOPIC_ROOT / \"02-可运行代码\" / \"microduck-playground-stilts\"\n", "OUTPUT_ROOT = TOPIC_ROOT / \"03-Notebook\" / \"outputs\"\n", "OUTPUT_ROOT.mkdir(parents=True, exist_ok=True)\n", "print(\"专题目录:\", TOPIC_ROOT)\n", "print(\"可运行代码:\", PLAYGROUND_ROOT)\n", "print(\"输出目录:\", OUTPUT_ROOT)\n", "print(\"Jupyter 工作站:\", socket.gethostname(), platform.system(), platform.machine())\n", "print(\"RDK 地址:\", os.getenv(\"RDK_HOST\", \"192.168.8.128\"), \"用户: sunrise\")" ] }, { "cell_type": "markdown", "id": "ad0cc1d7", "metadata": {}, "source": [ "## 1. 模型契约与本地检查\n", "\n", "目标契约是 actor `obs[1, 61] -> action[1, 14]`。如果本任务没有随专题提交 checkpoint/ONNX，Notebook 会明确显示“模型未提供”，不会用别的任务模型冒充当前任务结果。" ] }, { "cell_type": "code", "execution_count": 2, "id": "063d2189", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:24.928777Z", "iopub.status.busy": "2026-09-19T08:19:24.928696Z", "iopub.status.idle": "2026-09-19T08:19:24.931283Z", "shell.execute_reply": "2026-09-19T08:19:24.931081Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "任务: 接口/网页演示\n", "ONNX: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/01-任务资料/04-MicroDuck摆动旋转强化学习复现/assets/microduck_swing_alpha050.onnx\n" ] } ], "source": [ "TASK_ID = None\n", "MODEL_HINT = None\n", "MODEL_PATH = Path(os.getenv(\"MICRODUCK_ONNX\", \"\")) if os.getenv(\"MICRODUCK_ONNX\") else None\n", "if MODEL_PATH is None and MODEL_HINT:\n", " candidate = TOPIC_ROOT / MODEL_HINT\n", " if candidate.exists():\n", " MODEL_PATH = candidate\n", "if MODEL_PATH is None and not MODEL_HINT and not TASK_ID:\n", " candidates = sorted(TOPIC_ROOT.glob(\"01-任务资料/**/*.onnx\"))\n", " MODEL_PATH = candidates[0] if candidates else None\n", "print(\"任务:\", TASK_ID or \"接口/网页演示\")\n", "print(\"ONNX:\", MODEL_PATH if MODEL_PATH else \"未提供\")\n", "if TASK_ID and MODEL_PATH is None:\n", " print(\"该任务尚未随专题提交任务专属 ONNX；请设置 MICRODUCK_ONNX 后再运行模型检查。\")" ] }, { "cell_type": "code", "execution_count": 3, "id": "a8772a25", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:24.931769Z", "iopub.status.busy": "2026-09-19T08:19:24.931721Z", "iopub.status.idle": "2026-09-19T08:19:25.028429Z", "shell.execute_reply": "2026-09-19T08:19:25.028111Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"inputs\": [\n", " [\n", " \"obs\",\n", " [\n", " 1,\n", " 61\n", " ]\n", " ]\n", " ],\n", " \"outputs\": [\n", " [\n", " \"actions\",\n", " [\n", " 1,\n", " 14\n", " ]\n", " ]\n", " ],\n", " \"nodes\": 10,\n", " \"metadata\": {\n", " \"action_clip\": \"[-1, 1]\",\n", " \"action_clip_baked_into_onnx\": \"true\",\n", " \"run_path\": \"experiments\\\\swing\\\\checkpoints\\\\alpha050.pt\",\n", " \"joint_names\": \"left_hip_yaw,left_hip_roll,left_hip_pitch,left_knee,left_ankle,neck_pitch,head_pitch,head_yaw,head_roll,right_hip_yaw,right_hip_roll,right_hip_pitch,right_knee,right_ankle\",\n", " \"joint_stiffness\": \"1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000,1.000\",\n", " \"joint_damping\": \"0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000\",\n", " \"default_joint_pos\": \"0.000,0.000,-0.408,1.350,0.000,0.349,0.349,0.000,0.000,0.000,0.000,0.408,-1.350,0.000\",\n", " \"command_names\": \"twist\",\n", " \"observation_names\": \"base_ang_vel,projected_gravity,joint_pos,joint_vel,actions,command,head_command,body_command\",\n", " \"action_scale\": \"0.7\"\n", " }\n", "}\n", "PASS-model-contract-cell: 已完成检查\n" ] } ], "source": [ "def inspect_onnx(path):\n", " try:\n", " import onnx\n", " except ImportError:\n", " print(\"缺少 onnx：请在 microduck-playground 环境中启动 Jupyter。\")\n", " return None\n", " model = onnx.load(str(path))\n", " onnx.checker.check_model(model)\n", " def shape(value):\n", " return [d.dim_value if d.dim_value else (d.dim_param or \"?\") for d in value.type.tensor_type.shape.dim]\n", " result = {\n", " \"inputs\": [(item.name, shape(item)) for item in model.graph.input],\n", " \"outputs\": [(item.name, shape(item)) for item in model.graph.output],\n", " \"nodes\": len(model.graph.node),\n", " \"metadata\": {item.key: item.value for item in model.metadata_props},\n", " }\n", " print(json.dumps(result, ensure_ascii=False, indent=2))\n", " return result\n", "\n", "contract = inspect_onnx(MODEL_PATH) if MODEL_PATH else None\n", "print(\"PASS-model-contract-cell:\", \"已完成检查\" if contract is not None else \"模型未提供，已记录目标契约\")" ] }, { "cell_type": "markdown", "id": "d621e1a2", "metadata": {}, "source": [ "## 2. ONNX 导出入口\n", "\n", "训练得到的是策略 checkpoint；导出时要把 actor 和观测归一化一起固化到 ONNX。下面是本任务命令模板。`--video` 会在本机录制仿真回放，完整视频写入 `outputs/`，不提交到 Git。\n", "\n", "```bash\n", "uv run python scripts/export.py \\\n", " --checkpoint-file \\\n", " --onnx-file outputs/policy.onnx\\\n", " --num-envs 1 --video --video-length 250\n", "```" ] }, { "cell_type": "code", "execution_count": 4, "id": "29f91338", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:25.029350Z", "iopub.status.busy": "2026-09-19T08:19:25.029236Z", "iopub.status.idle": "2026-09-19T08:19:25.031038Z", "shell.execute_reply": "2026-09-19T08:19:25.030793Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "uv run python scripts/export.py --checkpoint-file --onnx-file outputs/policy.onnx --num-envs 1 --video --video-length 250\n", "导出前后检查：观测维度、动作顺序、归一化和 action clip。\n" ] } ], "source": [ "EXPORT_COMMAND = \"python scripts/export_onnx.py run_dir= output=outputs/motrix_ball_balance_latest.onnx opset=11\" if TASK_ID == \"microduck-ball-balance\" else (\"uv run python scripts/export.py \" + (TASK_ID or \"\") + \" --checkpoint-file --onnx-file outputs/policy.onnx --num-envs 1 --video --video-length 250\")\n", "print(EXPORT_COMMAND)\n", "print(\"导出前后检查：观测维度、动作顺序、归一化和 action clip。\")" ] }, { "cell_type": "markdown", "id": "4bf20492", "metadata": {}, "source": [ "## 3. 训练路径说明：从零或从预训练 checkpoint 开始\n", "\n", "每个 Notebook 都把训练入口和回放入口分开：可以把 `resume=False` 作为从零初始化，也可以把 `resume=True`、任务专属 checkpoint 和对应的 actor/critic/优化器状态作为预训练续训。默认演示视频使用已经验证过的任务模型，不能把它描述成当前单元刚训练出来的模型。\n", "\n", "这里的“10 个 iteration”是 10 轮采样加策略优化，不是每个动作只训练 10 步；例如 `64 个并行环境 × 24 个控制步 × 10` 才是约 `15360` 条 transition。导航 Notebook 明确不训练，直接接入已有模型；扰动和爬梯等 Notebook 也会标明它们使用的任务专属模型或官方双模型。" ] }, { "cell_type": "markdown", "id": "e2824e78", "metadata": {}, "source": [ "## 3. perturbation：回放任务，不训练\n", "\n", "本任务是接口/扰动回放，不包含独立策略训练。它只复用显式指定的已有策略进行物理回放；不会自动从其他任务 checkpoint 续训，也不会覆盖模型文件。" ] }, { "cell_type": "code", "execution_count": 5, "id": "725b60f6", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:25.031643Z", "iopub.status.busy": "2026-09-19T08:19:25.031592Z", "iopub.status.idle": "2026-09-19T08:19:25.032990Z", "shell.execute_reply": "2026-09-19T08:19:25.032756Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "回放任务：跳过训练。请在前面的策略路径单元显式指定已有 ONNX。\n" ] } ], "source": [ "print(\"回放任务：跳过训练。请在前面的策略路径单元显式指定已有 ONNX。\")" ] }, { "cell_type": "markdown", "id": "26c34c1c", "metadata": {}, "source": [ "## 4. 本地 ONNX 基准推理\n", "\n", "这一步只验证 ONNX 图能在本机运行，以及输出形状和数值是否有限；它不是训练效果评测，也不是 BPU 验收。" ] }, { "cell_type": "code", "execution_count": 6, "id": "f5d60fc6", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:25.033507Z", "iopub.status.busy": "2026-09-19T08:19:25.033460Z", "iopub.status.idle": "2026-09-19T08:19:25.071817Z", "shell.execute_reply": "2026-09-19T08:19:25.071473Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " \"providers\": [\n", " \"CPUExecutionProvider\"\n", " ],\n", " \"inputs\": [\n", " [\n", " \"obs\",\n", " [\n", " 1,\n", " 61\n", " ]\n", " ]\n", " ],\n", " \"outputs\": [\n", " [\n", " \"actions\",\n", " [\n", " 1,\n", " 14\n", " ]\n", " ]\n", " ],\n", " \"finite\": true\n", "}\n" ] } ], "source": [ "def onnx_local_smoke(path):\n", " try:\n", " import numpy as np\n", " import onnxruntime as ort\n", " except ImportError as exc:\n", " print(\"缺少本地推理依赖:\", exc)\n", " return None\n", " session = ort.InferenceSession(str(path), providers=[\"CPUExecutionProvider\"])\n", " feeds = {}\n", " for item in session.get_inputs():\n", " shape = [dim if isinstance(dim, int) and dim > 0 else 1 for dim in item.shape]\n", " feeds[item.name] = np.zeros(shape, dtype=np.float32)\n", " outputs = session.run(None, feeds)\n", " info = {\"providers\": session.get_providers(), \"inputs\": [(x.name, x.shape) for x in session.get_inputs()], \"outputs\": [(x.name, x.shape) for x in session.get_outputs()], \"finite\": all(np.isfinite(x).all() for x in outputs)}\n", " print(json.dumps(info, ensure_ascii=False, indent=2, default=str))\n", " return outputs\n", "\n", "if MODEL_PATH:\n", " local_outputs = onnx_local_smoke(MODEL_PATH)\n", "else:\n", " print(\"跳过：尚未提供该任务 ONNX。\")" ] }, { "cell_type": "markdown", "id": "3e5cde78", "metadata": {}, "source": [ "## 5. GPU 推理视频\n", "\n", "本单元先生成 GPU 推理视频，再生成后面的 RDK X5 BPU 视频。Ubuntu 负责 MuJoCo/MotrixSim 物理和渲染；策略 actor 使用 CUDA 上的 Torch 执行导出的 ONNX 图。这里只展示“GPU 推理”和任务结果，不暴露具体显卡型号。视频文件写入 `outputs/gpu_videos/`。\n", "\n", "除扰动回放明确复用 walking 策略外，每个任务都使用自己的 ONNX；篮球使用带 LSTM 的篮球 actor，球平衡使用 FastSAC actor，爬梯 GPU 回放使用 climber/get-up 两个 actor。" ] }, { "cell_type": "code", "execution_count": 7, "id": "a6be83a7", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:25.072776Z", "iopub.status.busy": "2026-09-19T08:19:25.072679Z", "iopub.status.idle": "2026-09-19T08:19:41.123753Z", "shell.execute_reply": "2026-09-19T08:19:41.123469Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "GPU 推理任务: perturbation\n", "GPU 模型: /home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx\n", "开始生成 GPU 推理视频。\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "GPU 策略推理已启用；不展示具体显卡型号。\n", "PASS-GPU-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/perturbation_gpu_latest.mp4\n", "PASS-GPU-REPORT: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/perturbation_gpu_latest.json\n", "PASS-GPU-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/gpu_videos/perturbation_gpu_latest.mp4\n", "{'task': 'perturbation', 'task_id': 'Mjlab-Velocity-Flat-MicroDuck', 'display_name': 'physical-perturbation', 'backend': 'GPU ONNX via Torch CUDA', 'policy_device': 'GPU', 'onnx_file': '/home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx', 'onnx_sha256': '2da8200426614e4a83ef736f80e1ecfc1ca5c11e5dd28efd3524111d03314174', 'observation_dim': 61, 'action_dim': 14, 'recurrent': False, 'steps': 250, 'terminated_or_truncated_resets': 1, 'reset_steps': [102], 'first_termination_step': 102, 'longest_uninterrupted_steps': 148, 'longest_uninterrupted_time_s': 2.96, 'video_nominal_time_s': 5.0, 'mean_inference_ms': 0.5789483119442593, 'p95_inference_ms': 2.360974099519808, 'note': 'Ubuntu runs the MuJoCo physics and renderer; the task actor is evaluated by Torch on GPU.'}\n" ] }, { "data": { "text/html": [ " $\"GPU$ " ], "text/plain": [ "GIF 预览：../01-任务资料/精选视频/assets/notebook-replays/perturbation_gpu.gif；原始 MP4 未嵌入 Notebook，以控制仓库体积。" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import sys\n", "\n", "GPU_TASK = 'perturbation'\n", "GPU_VIDEO_STEPS = int(os.getenv(\"MICRODUCK_GPU_VIDEO_STEPS\", \"250\"))\n", "GPU_VIDEO_OUTPUT = OUTPUT_ROOT / \"gpu_videos\" / f\"{GPU_TASK}_gpu_latest.mp4\" if GPU_TASK else None\n", "GPU_ONNX_DEFAULTS = {\n", " \"basketball\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/basketball.opset11.onnx\",\n", " \"walking\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx\",\n", " \"perturbation\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/walking.opset11.onnx\",\n", " \"stilt\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/stilts25.opset11.onnx\",\n", " \"swing\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/swing.opset11.onnx\",\n", " \"ball_balance\": \"/home/ubuntu/workspaces/microduck_bpu_models/converted/motrix_ball_balance.opset11.onnx\",\n", "}\n", "GPU_ONNX = Path(os.getenv(\"MICRODUCK_GPU_ONNX\", GPU_ONNX_DEFAULTS.get(GPU_TASK, \"\"))).expanduser() if GPU_TASK else None\n", "GPU_LADDER_CLIMBER = Path(os.getenv(\"MICRODUCK_GPU_LADDER_CLIMBER\", \"/home/ubuntu/workspaces/microduck_bpu_models/converted/official_ladder_climber.opset11.onnx\")).expanduser()\n", "GPU_LADDER_GETUP = Path(os.getenv(\"MICRODUCK_GPU_LADDER_GETUP\", \"/home/ubuntu/workspaces/microduck_bpu_models/converted/official_ladder_getup.opset11.onnx\")).expanduser()\n", "MOTRIX_ROOT = Path(os.getenv(\"MOTRIX_ROOT\", \"/home/ubuntu/workspaces/MotrixLab\")).expanduser() if GPU_TASK == \"ball_balance\" else None\n", "MOTRIX_PYTHON = Path(os.getenv(\"MOTRIX_PYTHON\", str(MOTRIX_ROOT / \".venv/bin/python\"))).expanduser() if MOTRIX_ROOT else None\n", "MOTRIX_RUNNER = PLAYGROUND_ROOT / \"scripts\" / \"motrix_runner.py\" if GPU_TASK == \"ball_balance\" else None\n", "GPU_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / (\"motrix_gpu_inloop_video.py\" if GPU_TASK == \"ball_balance\" else \"gpu_inloop_video.py\")\n", "GPU_LADDER_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / \"bpu_official_ladder_pair_video.py\"\n", "\n", "if GPU_TASK == \"ladder\":\n", " gpu_ready = GPU_LADDER_SCRIPT.is_file() and GPU_LADDER_CLIMBER.is_file() and GPU_LADDER_GETUP.is_file()\n", " gpu_cmd = [\n", " sys.executable, str(GPU_LADDER_SCRIPT), \"--backend\", \"gpu\",\n", " \"--official-root\", \"/home/ubuntu/workspaces/microduck-playground-official/experiments/desk-climb\",\n", " \"--climber-onnx\", str(GPU_LADDER_CLIMBER), \"--getup-onnx\", str(GPU_LADDER_GETUP),\n", " \"--output\", str(GPU_VIDEO_OUTPUT), \"--seconds\", \"24\", \"--num-envs\", \"64\", \"--record-env\", \"17\",\n", " ]\n", "else:\n", " gpu_ready = GPU_SCRIPT.is_file() and GPU_ONNX is not None and GPU_ONNX.is_file()\n", " gpu_cmd = [sys.executable, str(GPU_SCRIPT)] if GPU_TASK != \"ball_balance\" else [str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"video\", str(GPU_SCRIPT)]\n", " if GPU_TASK != \"ball_balance\":\n", " gpu_cmd += [\"--task\", GPU_TASK]\n", " gpu_cmd += [\"--onnx\", str(GPU_ONNX), \"--output\", str(GPU_VIDEO_OUTPUT), \"--steps\", str(GPU_VIDEO_STEPS)]\n", "\n", "print(\"GPU 推理任务:\", GPU_TASK)\n", "print(\"GPU 模型:\", GPU_ONNX if GPU_TASK != \"ladder\" else (GPU_LADDER_CLIMBER, GPU_LADDER_GETUP))\n", "if not gpu_ready:\n", " print(\"GPU 回放暂不可运行：请检查任务专属 ONNX、脚本和 Motrix 环境。\")\n", "elif GPU_TASK == \"ball_balance\" and (MOTRIX_PYTHON is None or not MOTRIX_PYTHON.is_file()):\n", " print(\"找不到 MotrixLab Python 环境:\", MOTRIX_PYTHON)\n", "else:\n", " GPU_VIDEO_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " print(\"开始生成 GPU 推理视频。\")\n", " gpu_result = subprocess.run(gpu_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(), capture_output=True, text=True)\n", " safe_lines = [line for line in (gpu_result.stdout + \"\\n\" + gpu_result.stderr).splitlines() if \"GPU\" in line or \"PASS-\" in line or \"Video\" in line]\n", " print(\"\\n\".join(safe_lines[-20:]))\n", " if gpu_result.returncode != 0 or not GPU_VIDEO_OUTPUT.is_file():\n", " raise RuntimeError(\"GPU 推理视频生成失败，请查看脚本日志。\")\n", " GPU_REPORT = GPU_VIDEO_OUTPUT.with_suffix(\".json\")\n", " print(\"PASS-GPU-VIDEO:\", GPU_VIDEO_OUTPUT)\n", " if GPU_REPORT.is_file():\n", " print(json.loads(GPU_REPORT.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(GPU_VIDEO_OUTPUT), embed=True))" ] }, { "cell_type": "markdown", "id": "5a976527", "metadata": {}, "source": [ "## 6. RDK X5 BPU-in-the-loop 视频\n", "\n", "本单元只展示新生成的板端闭环 MP4，不读取参考 GIF 或关键帧。Ubuntu 运行 MuJoCo 物理和 EGL 渲染；每一个控制步把任务观测发送到 RDK，RDK 用当前任务对应的 HBM 在 BPU 上计算 14 维动作，再返回给 Ubuntu。视频因此是“板端 BPU 产生动作、Ubuntu 负责仿真画面”的真实闭环。\n", "\n", "运行前会自动重启 RDK 上的任务策略服务。切换 Notebook 时必须重新运行本单元，让板端 HBM 与当前任务匹配。" ] }, { "cell_type": "code", "execution_count": 8, "id": "7c12d022", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:41.127227Z", "iopub.status.busy": "2026-09-19T08:19:41.127146Z", "iopub.status.idle": "2026-09-19T08:19:47.301564Z", "shell.execute_reply": "2026-09-19T08:19:47.301263Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RDK BPU 控制服务不可用，尝试 SSH: [Errno 113] No route to host\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RDK 策略服务启动失败: ssh: connect to host 192.168.8.128 port 22: No route to host\n", "BPU-CACHED-VIDEO: /home/ubuntu/workspaces/microduck-teaching/microduck-topic/16-专题组队学习/05-MicroDuck运动策略与任务开发专题/03-Notebook/outputs/bpu_videos/perturbation_bpu_latest.mp4\n", "说明: 本轮板端不可达，保留此前真实 RDK X5 BPU 任务视频。\n", "{'task': 'perturbation', 'task_id': 'Mjlab-Velocity-Flat-MicroDuck', 'display_name': 'physical-perturbation', 'backend': 'RDK X5 BPU via hbm_runtime', 'bpu_host': '192.168.8.128', 'bpu_port': 8765, 'hbm_file': '/home/ubuntu/workspaces/microduck_bpu_models/compiled/walking/model_output/walking.opset11.bin', 'hbm_sha256': 'fc44b395300aa6b45e4b8e3677f669f6e275e5ce6d6c26e2a7c08c6e27129110', 'observation_dim_sent': 61, 'action_dim': 14, 'recurrent': False, 'source_root': '.', 'steps': 250, 'terminated_or_truncated_resets': 1, 'reset_steps': [98], 'first_termination_step': 98, 'first_termination_time_s': 1.96, 'longest_uninterrupted_steps': 152, 'longest_uninterrupted_time_s': 3.04, 'video_nominal_time_s': 5.0, 'video_actual_time_s': 5.0, 'mean_round_trip_ms': 10.761883068829775, 'p95_round_trip_ms': 37.828994262963505, 'camera': {'distance': 1.2, 'elevation': -8.0, 'azimuth': 90.0}, 'note': 'Ubuntu MuJoCo renders; RDK X5 BPU produces every policy action.'}\n" ] }, { "data": { "text/html": [ " $\"历史缓存$ " ], "text/plain": [ "GIF 预览：../01-任务资料/精选视频/assets/notebook-replays/perturbation_bpu_cached.gif；原始 MP4 未嵌入 Notebook，以控制仓库体积。" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "MuJoCo 回放命令模板: python 任务资料/02 的 serve_mjswan.py\n" ] } ], "source": [ "import hashlib\n", "import sys\n", "\n", "BPU_TASK = 'perturbation'\n", "BPU_LADDER_LOCAL_OPT_IN = os.getenv(\"MICRODUCK_RUN_LOCAL_LADDER_BPU\", \"0\") == \"1\"\n", "BPU_VIDEO_STEPS = int(os.getenv(\"MICRODUCK_BPU_VIDEO_STEPS\", \"250\"))\n", "BPU_VIDEO_WIDTH = int(os.getenv(\"MICRODUCK_BPU_VIDEO_WIDTH\", \"640\"))\n", "BPU_VIDEO_HEIGHT = int(os.getenv(\"MICRODUCK_BPU_VIDEO_HEIGHT\", \"480\"))\n", "BPU_CAMERA_DISTANCE = os.getenv(\"MICRODUCK_BPU_CAMERA_DISTANCE\", \"1.2\")\n", "BPU_CAMERA_ELEVATION = os.getenv(\"MICRODUCK_BPU_CAMERA_ELEVATION\", \"-8\")\n", "BPU_CAMERA_AZIMUTH = os.getenv(\"MICRODUCK_BPU_CAMERA_AZIMUTH\", \"90\")\n", "BPU_PORT = int(os.getenv(\"MICRODUCK_BPU_PORT\", \"8765\"))\n", "RDK_CONTROL_PORT = int(os.getenv(\"MICRODUCK_BPU_CONTROL_PORT\", \"8766\"))\n", "RDK_HOST = os.getenv(\"RDK_HOST\", \"192.168.8.128\")\n", "BPU_HBM_DEFAULTS = {\n", " \"ball_balance\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/ball_balance/model_output/ball_balance.opset11.bin\",\n", " \"basketball\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/basketball/model_output/basketball.opset11.bin\",\n", " \"walking\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/walking/model_output/walking.opset11.bin\",\n", " \"perturbation\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/walking/model_output/walking.opset11.bin\",\n", " \"stilt\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/stilts25/model_output/stilts25.opset11.bin\",\n", " \"swing\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/swing/model_output/swing.opset11.bin\",\n", " \"ladder\": \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/ladder/model_output/ladder.opset11.bin\",\n", "}\n", "RDK_HBM_DEFAULTS = {\n", " \"ball_balance\": \"/home/sunrise/microduck_policy/hbm/ball_balance.bin\",\n", " \"basketball\": \"/home/sunrise/microduck_policy/hbm/basketball.bin\",\n", " \"walking\": \"/home/sunrise/microduck_policy/hbm/walking.bin\",\n", " \"perturbation\": \"/home/sunrise/microduck_policy/hbm/walking.bin\",\n", " \"stilt\": \"/home/sunrise/microduck_policy/hbm/stilts25.bin\",\n", " \"swing\": \"/home/sunrise/microduck_policy/hbm/swing.bin\",\n", " \"ladder\": \"/home/sunrise/microduck_policy/hbm/ladder.bin\",\n", "}\n", "BPU_HBM = Path(os.getenv(\"MICRODUCK_BPU_HBM\", BPU_HBM_DEFAULTS.get(BPU_TASK, \"\"))) if BPU_TASK else None\n", "RDK_HBM = RDK_HBM_DEFAULTS.get(BPU_TASK, \"\") if BPU_TASK else \"\"\n", "RDK_SERVER_SCRIPT = os.getenv(\"RDK_BPU_SERVER_SCRIPT\", \"/home/sunrise/microduck_policy/rdk_bpu_policy_server.py\")\n", "MOTRIX_ROOT = Path(os.getenv(\"MOTRIX_ROOT\", \"/home/ubuntu/workspaces/MotrixLab\")).expanduser() if BPU_TASK == \"ball_balance\" else None\n", "MOTRIX_PYTHON = Path(os.getenv(\"MOTRIX_PYTHON\", str(MOTRIX_ROOT / \".venv/bin/python\"))).expanduser() if MOTRIX_ROOT else None\n", "MOTRIX_RUNNER = PLAYGROUND_ROOT / \"scripts\" / \"motrix_runner.py\" if BPU_TASK == \"ball_balance\" else None\n", "BPU_VIDEO_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / (\"motrix_bpu_inloop_video.py\" if BPU_TASK == \"ball_balance\" else \"bpu_inloop_video.py\")\n", "BPU_VIDEO_OUTPUT = OUTPUT_ROOT / \"bpu_videos\" / f\"{BPU_TASK}_bpu_latest.mp4\" if BPU_TASK else None\n", "ALLOW_CACHED_BPU = os.getenv(\"MICRODUCK_ALLOW_CACHED_BPU\", \"0\") == \"1\"\n", "BPU_LADDER_PAIR_SCRIPT = PLAYGROUND_ROOT / \"scripts\" / \"bpu_official_ladder_pair_video.py\"\n", "BPU_LADDER_CLIMBER_HBM = Path(os.getenv(\"MICRODUCK_BPU_LADDER_CLIMBER_HBM\", \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/official_ladder_climber/model_output/official_ladder_climber.bin\")).expanduser()\n", "BPU_LADDER_GETUP_HBM = Path(os.getenv(\"MICRODUCK_BPU_LADDER_GETUP_HBM\", \"/home/ubuntu/workspaces/microduck_bpu_models/compiled/official_ladder_getup/model_output/official_ladder_getup.bin\")).expanduser()\n", "BPU_LADDER_PAIR_OUTPUT = OUTPUT_ROOT / \"bpu_videos\" / \"ladder_official_pair_bpu_latest.mp4\"\n", "\n", "def show_cached_bpu_video(path, reason):\n", " if not ALLOW_CACHED_BPU or path is None or not path.is_file():\n", " return False\n", " print(\"BPU-CACHED-VIDEO:\", path)\n", " print(\"说明:\", reason)\n", " report = path.with_suffix(\".json\")\n", " if report.is_file():\n", " print(json.loads(report.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(path), embed=True))\n", " return True\n", "\n", "def restart_rdk_policy_server():\n", " if not BPU_TASK or not RDK_HBM:\n", " return False\n", " try:\n", " if BPU_HBM is not None and BPU_HBM.is_file():\n", " payload = BPU_HBM.read_bytes()\n", " request = {\n", " \"op\": \"upload_and_switch\",\n", " \"task\": BPU_TASK,\n", " \"size\": len(payload),\n", " \"sha256\": hashlib.sha256(payload).hexdigest(),\n", " }\n", " else:\n", " payload = b\"\"\n", " request = {\"op\": \"switch\", \"task\": BPU_TASK}\n", " with socket.create_connection((RDK_HOST, RDK_CONTROL_PORT), timeout=8) as control:\n", " control.sendall((json.dumps(request) + \"\\n\").encode(\"utf-8\"))\n", " if payload:\n", " for offset in range(0, len(payload), 1024 * 1024):\n", " control.sendall(payload[offset:offset + 1024 * 1024])\n", " response = json.loads(control.makefile(\"rb\").readline().decode(\"utf-8\"))\n", " if response.get(\"ok\"):\n", " action = \"已上传并切换\" if payload else \"已切换\"\n", " print(\"RDK 任务策略服务\" + action + \":\", BPU_TASK, \"->\", response.get(\"model\"), \"pid=\", response.get(\"pid\"))\n", " return True\n", " print(\"RDK BPU 控制服务拒绝切换:\", response)\n", " except (OSError, ValueError, json.JSONDecodeError) as exc:\n", " print(\"RDK BPU 控制服务不可用，尝试 SSH:\", exc)\n", " remote = \"sunrise@\" + RDK_HOST\n", " command = (\n", " \"pkill -f '^python3 .*rdk_bpu_policy_server.py' >/dev/null 2>&1 || true; \"\n", " \"sleep 1; \"\n", " \"nohup python3 \" + shlex.quote(RDK_SERVER_SCRIPT) +\n", " \" --model \" + shlex.quote(RDK_HBM) +\n", " \" --port \" + str(BPU_PORT) +\n", " \" >/tmp/microduck_bpu_server.log 2>&1 \", RDK_HBM)\n", " return True\n", "\n", "if BPU_TASK == \"ladder\":\n", " pair_ready = BPU_LADDER_PAIR_SCRIPT.is_file() and BPU_LADDER_CLIMBER_HBM.is_file() and BPU_LADDER_GETUP_HBM.is_file()\n", " pair_cmd = [\n", " sys.executable, str(BPU_LADDER_PAIR_SCRIPT), \"--backend\", \"bpu\",\n", " \"--climber-hbm\", str(BPU_LADDER_CLIMBER_HBM), \"--getup-hbm\", str(BPU_LADDER_GETUP_HBM),\n", " \"--output\", str(BPU_LADDER_PAIR_OUTPUT), \"--seconds\", os.getenv(\"MICRODUCK_LADDER_SECONDS\", \"24\"),\n", " \"--num-envs\", \"64\", \"--record-env\", \"17\",\n", " ]\n", " print(\"BPU 官方双模型:\", BPU_LADDER_CLIMBER_HBM, BPU_LADDER_GETUP_HBM)\n", " if not pair_ready:\n", " print(\"官方爬梯 BPU 双模型视频暂不可运行：请检查 climber/get-up HBM 和脚本。\")\n", " else:\n", " BPU_LADDER_PAIR_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " pair_result = subprocess.run(pair_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(), capture_output=True, text=True)\n", " print(\"\\n\".join((pair_result.stdout + \"\\n\" + pair_result.stderr).splitlines()[-35:]))\n", " if pair_result.returncode != 0 or not BPU_LADDER_PAIR_OUTPUT.is_file():\n", " if not show_cached_bpu_video(BPU_LADDER_PAIR_OUTPUT, \"本轮板端不可达，保留此前真实 RDK X5 BPU 双模型视频。\"):\n", " raise RuntimeError(\"官方爬梯 BPU 双模型视频生成失败。\")\n", " else:\n", " print(\"PASS-BPU-VIDEO:\", BPU_LADDER_PAIR_OUTPUT)\n", " pair_report = BPU_LADDER_PAIR_OUTPUT.with_suffix(\".json\")\n", " if pair_report.is_file():\n", " print(json.loads(pair_report.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(BPU_LADDER_PAIR_OUTPUT), embed=True))\n", "elif not BPU_TASK:\n", " print(\"该任务目前没有可验证的任务专属 HBM；不会用其他任务视频替代。\")\n", "elif BPU_TASK == \"ladder\" and not BPU_LADDER_LOCAL_OPT_IN:\n", " print(\"已跳过本地 14 级 ladder HBM 视频：它不是社区 climber/get-up 方案。\")\n", " print(\"只有明确设置 MICRODUCK_RUN_LOCAL_LADDER_BPU=1 才会运行本地实验，不得将结果标为社区复刻。\")\n", "elif BPU_HBM is None or not BPU_HBM.is_file():\n", " print(\"Ubuntu 端 HBM 不存在:\", BPU_HBM)\n", "elif not BPU_VIDEO_SCRIPT.is_file():\n", " print(\"找不到板端闭环录制器:\", BPU_VIDEO_SCRIPT)\n", "elif BPU_TASK == \"ball_balance\" and (MOTRIX_PYTHON is None or not MOTRIX_PYTHON.is_file()):\n", " print(\"找不到 MotrixLab Python 环境:\", MOTRIX_PYTHON)\n", "elif BPU_TASK == \"ball_balance\" and (MOTRIX_RUNNER is None or not MOTRIX_RUNNER.is_file()):\n", " print(\"找不到 MotrixLab 启动器:\", MOTRIX_RUNNER)\n", "elif restart_rdk_policy_server():\n", " BPU_VIDEO_OUTPUT.parent.mkdir(parents=True, exist_ok=True)\n", " if BPU_TASK == \"ball_balance\":\n", " video_cmd = [str(MOTRIX_PYTHON), str(MOTRIX_RUNNER), \"video\", str(BPU_VIDEO_SCRIPT)]\n", " else:\n", " video_cmd = [sys.executable, str(BPU_VIDEO_SCRIPT)]\n", " if BPU_TASK != \"ball_balance\":\n", " video_cmd += [\"--task\", BPU_TASK]\n", " video_cmd += [\n", " \"--bpu-host\", RDK_HOST, \"--bpu-port\", str(BPU_PORT),\n", " \"--hbm\", str(BPU_HBM), \"--output\", str(BPU_VIDEO_OUTPUT),\n", " \"--steps\", str(BPU_VIDEO_STEPS), \"--width\", str(BPU_VIDEO_WIDTH),\n", " \"--height\", str(BPU_VIDEO_HEIGHT),\n", " ]\n", " video_cmd += [\n", " \"--camera-distance\", BPU_CAMERA_DISTANCE,\n", " \"--camera-elevation\", BPU_CAMERA_ELEVATION,\n", " \"--camera-azimuth\", BPU_CAMERA_AZIMUTH,\n", " ]\n", " print(\"开始生成板端闭环视频:\", \" \".join(shlex.quote(x) for x in video_cmd))\n", " video_result = subprocess.run(\n", " video_cmd, cwd=PLAYGROUND_ROOT, env=os.environ.copy(),\n", " capture_output=True, text=True,\n", " )\n", " print(\"\\n\".join((video_result.stdout + \"\\n\" + video_result.stderr).splitlines()[-35:]))\n", " if video_result.returncode != 0 or not BPU_VIDEO_OUTPUT.is_file():\n", " raise RuntimeError(\"BPU-in-the-loop 视频生成失败，请先查看上面最后 35 行日志。\")\n", " BPU_REPORT = BPU_VIDEO_OUTPUT.with_suffix(\".json\")\n", " print(\"PASS-BPU-VIDEO:\", BPU_VIDEO_OUTPUT)\n", " if BPU_REPORT.is_file():\n", " print(json.loads(BPU_REPORT.read_text(encoding=\"utf-8\")))\n", " from IPython.display import Video, display\n", " display(Video(str(BPU_VIDEO_OUTPUT), embed=True))\n", "else:\n", " show_cached_bpu_video(BPU_VIDEO_OUTPUT, \"本轮板端不可达，保留此前真实 RDK X5 BPU 任务视频。\")\n", "\n", "SIM_COMMAND = 'python 任务资料/02 的 serve_mjswan.py'\n", "print(\"MuJoCo 回放命令模板:\", SIM_COMMAND)" ] }, { "cell_type": "markdown", "id": "03f9137b", "metadata": {}, "source": [ "## 7. RDK X5 / BPU 审计\n", "\n", "上一单元已经生成任务专属闭环视频。本单元只检查板端连通、HBM 元数据和服务日志；通用 MobileNet 样例只能证明系统运行时存在，不能代替任务视频。" ] }, { "cell_type": "code", "execution_count": 9, "id": "e1d64462", "metadata": { "execution": { "iopub.execute_input": "2026-09-19T08:19:47.304643Z", "iopub.status.busy": "2026-09-19T08:19:47.304578Z", "iopub.status.idle": "2026-09-19T08:19:50.367767Z", "shell.execute_reply": "2026-09-19T08:19:50.367482Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "默认 BPU 样例模型: /opt/tros/humble/lib/dnn_benchmark_example/config/X5/mobilenetv1_224x224_nv12.bin\n", "默认 BPU 样例输入: 75264 bytes uint8\n", "MicroDuck 策略 HBM: 该任务尚未提供\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RDK 暂不可达，跳过板端验收: ssh: connect to host 192.168.8.128 port 22: No route to host\n" ] } ], "source": [ "import time\n", "\n", "RDK_HOST = os.getenv(\"RDK_HOST\", \"192.168.8.128\")\n", "BPU_SMOKE_MODEL = os.getenv(\n", " \"RDK_BPU_SMOKE_MODEL\",\n", " \"/opt/tros/humble/lib/dnn_benchmark_example/config/X5/mobilenetv1_224x224_nv12.bin\",\n", ")\n", "BPU_SMOKE_INPUT_BYTES = int(os.getenv(\"RDK_BPU_SMOKE_INPUT_BYTES\", \"75264\"))\n", "BPU_HBM = os.getenv(\"RDK_BPU_HBM\", \"\")\n", "print(\"默认 BPU 样例模型:\", BPU_SMOKE_MODEL)\n", "print(\"默认 BPU 样例输入:\", BPU_SMOKE_INPUT_BYTES, \"bytes uint8\")\n", "print(\"MicroDuck 策略 HBM:\", BPU_HBM or \"该任务尚未提供\")\n", "probe_script = \"for x in hrt_model_exec hb_mapper hrt_bin_dump hrt_bin_info python3; do if command -v $x >/dev/null 2>&1; then echo $x=$(command -v $x); else echo $x=MISSING; fi; done\"\n", "remote = \"sunrise@\" + RDK_HOST\n", "probe = subprocess.run([\"ssh\", \"-o\", \"BatchMode=yes\", \"-o\", \"ConnectTimeout=5\", remote, \"bash -lc \" + shlex.quote(probe_script)], capture_output=True, text=True)\n", "if probe.returncode:\n", " print(\"RDK 暂不可达，跳过板端验收:\", probe.stderr.strip() or \"ssh failed\")\n", "else:\n", " print(\"RDK 已连通:\", RDK_HOST)\n", " print(probe.stdout)\n", " if not BPU_HBM:\n", " print(\"没有任务 HBM：不执行通用样例冒充任务推理。\")\n", " smoke_model = BPU_SMOKE_MODEL\n", " smoke_bytes = BPU_SMOKE_INPUT_BYTES\n", " smoke_info_cmd = \"hrt_model_exec model_info --model_file \" + shlex.quote(smoke_model)\n", " smoke_info = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(smoke_info_cmd)], capture_output=True, text=True)\n", " print(\"BPU 样例 model_info exit:\", smoke_info.returncode)\n", " print(smoke_info.stdout[-4000:])\n", " print(smoke_info.stderr[-1000:])\n", " if smoke_info.returncode == 0:\n", " import numpy as np\n", " local_input = OUTPUT_ROOT / \"rdk_bpu_runtime_smoke.bin\"\n", " np.zeros(smoke_bytes, dtype=np.uint8).tofile(local_input)\n", " remote_dir = \"/tmp/microduck_notebook\"\n", " remote_input = remote_dir + \"/rdk_bpu_runtime_smoke.bin\"\n", " subprocess.run([\"ssh\", remote, \"mkdir\", \"-p\", remote_dir], check=True)\n", " subprocess.run([\"scp\", str(local_input), remote + \":\" + remote_input], check=True)\n", " dump_dir = remote_dir + \"/runtime_dump\"\n", " infer_cmd = \"rm -rf \" + shlex.quote(dump_dir) + \" && mkdir -p \" + shlex.quote(dump_dir) + \" && hrt_model_exec infer --model_file \" + shlex.quote(smoke_model) + \" --input_file \" + shlex.quote(remote_input) + \" --enable_dump true --dump_format txt --dump_path \" + shlex.quote(dump_dir)\n", " started = time.perf_counter()\n", " infer = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(infer_cmd)], capture_output=True, text=True)\n", " elapsed_ms = (time.perf_counter() - started) * 1000.0\n", " print(\"BPU runtime sample exit:\", infer.returncode, \"round-trip ms:\", round(elapsed_ms, 2))\n", " print(infer.stdout[-4000:])\n", " print(infer.stderr[-1000:])\n", " if infer.returncode == 0:\n", " print(\"PASS-rdk-bpu-runtime-sample: X5 BPU 样例推理已返回 0。\")\n", " else:\n", " info_cmd = \"hrt_model_exec model_info --model_file \" + shlex.quote(BPU_HBM)\n", " info = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(info_cmd)], capture_output=True, text=True)\n", " print(\"HBM model_info exit:\", info.returncode)\n", " print(info.stdout[-6000:])\n", " print(info.stderr[-2000:])\n", " if info.returncode == 0:\n", " # hrt_model_exec consumes binary tensors; this zero observation is\n", " # only a transport/BPU smoke input, not a task-quality evaluation.\n", " import numpy as np\n", " local_input = OUTPUT_ROOT / \"zero_obs_61_f32.bin\"\n", " np.zeros((1, 61), dtype=np.float32).tofile(local_input)\n", " remote_dir = \"/tmp/microduck_notebook\"\n", " remote_input = remote_dir + \"/zero_obs_61_f32.bin\"\n", " subprocess.run([\"ssh\", remote, \"mkdir\", \"-p\", remote_dir], check=True)\n", " subprocess.run([\"scp\", str(local_input), remote + \":\" + remote_input], check=True)\n", " dump_dir = remote_dir + \"/dump\"\n", " infer_cmd = \"rm -rf \" + shlex.quote(dump_dir) + \" && mkdir -p \" + shlex.quote(dump_dir) + \" && hrt_model_exec infer --model_file \" + shlex.quote(BPU_HBM) + \" --input_file \" + shlex.quote(remote_input) + \" --enable_dump true --dump_format txt --dump_path \" + shlex.quote(dump_dir)\n", " started = time.perf_counter()\n", " infer = subprocess.run([\"ssh\", remote, \"bash -lc \" + shlex.quote(infer_cmd)], capture_output=True, text=True)\n", " elapsed_ms = (time.perf_counter() - started) * 1000.0\n", " print(\"BPU infer exit:\", infer.returncode, \"round-trip ms:\", round(elapsed_ms, 2))\n", " print(infer.stdout[-6000:])\n", " print(infer.stderr[-2000:])\n", " if infer.returncode == 0:\n", " print(\"PASS-rdk-bpu: hrt_model_exec infer 已返回 0。\")\n", " else:\n", " print(\"未通过：优先以第 5 节 BPU-in-the-loop 报告为准。\")" ] }, { "cell_type": "markdown", "id": "819d8867", "metadata": {}, "source": [ "## 8. 直播结论\n", "\n", "- `PASS-local-onnx`：ONNX checker 和本地 ONNX Runtime 通过。\n", "- `PASS-GPU-VIDEO`：GPU actor 推理视频已生成，且排在 BPU 视频之前。\n", "- `PASS-BPU-VIDEO`：Ubuntu MuJoCo 视频中的每一个动作都由 RDK X5 对应 HBM 产生。\n", "- `PASS-rdk-bpu-runtime-sample`：板端已安装的 X5 BPU 样例推理退出码为 0。\n", "- `PASS-rdk-bpu`：SSH 连通、板端明确选择 BPU、MicroDuck HBM 推理命令退出码为 0，并记录输入输出形状与延迟。\n", "\n", "没有 `PASS-BPU-VIDEO` 的任务，只能算“已有素材/接口说明”，不能算板端视频已完成。\\n\\nBPU status rule: PASS-BPU-VIDEO means a current online RDK run. BPU-CACHED-VIDEO means a previous real BPU video is shown while the board is offline; rerun this cell after RDK is online." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.14" } }, "nbformat": 4, "nbformat_minor": 5 }