{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 10. \u7528 \u03c00.5 \u5fae\u8c03 OMY \u673a\u68b0\u81c2\u5e76\u5728 MuJoCo \u4e2d\u8bc4\u6d4b\n", "\n", "\u8fd9\u4efd Notebook\uff3b\u4ea4\u4e92\u5f0f\u7b14\u8bb0\u672c\uff3d\u63a5\u7eed [1.collect_data.ipynb](1.collect_data.ipynb) \u548c [2.visualize_data-omy.ipynb](2.visualize_data-omy.ipynb)\uff0c\u5b8c\u6210\uff1a\u68c0\u67e5\u793a\u6559 \u2192 \u8fc1\u79fb\u6570\u636e \u2192 \u8865\u5145\u5206\u4f4d\u6570 \u2192 \u4e94\u6b65\u8bad\u7ec3\u68c0\u67e5 \u2192 \u6b63\u5f0f\u5fae\u8c03 \u2192 \u79bb\u7ebf\u52a8\u4f5c\u5bf9\u6bd4 \u2192 \u4eff\u771f\u6293\u653e\u3002\n", "\n", "\u5bf9\u8c61\u662f **ROBOTIS OMY \u516d\u8f74\u673a\u68b0\u81c2\u4e0e\u5939\u722a\u7684\u5355\u676f\u6293\u653e\u573a\u666f** `asset/example_scene_y.xml`\u3002\u4f7f\u7528\u72ec\u7acb\u7684 **Python 3.12 / LeRobot 0.6.1 / PyTorch 2.7.1 CUDA 12.8** \u73af\u5883\uff0c\u672c\u7ae0\u7b2c 1\uff5e9 \u8282\u7ee7\u7eed\u4f7f\u7528\u5404\u81ea\u7684\u65e7\u73af\u5883\u3002\u4e0d\u8981\u5728\u65e7\u73af\u5883\u5185\u5347\u7ea7\u4f9d\u8d56\uff0c\u4e5f\u4e0d\u8981\u7528\u65e7 `train_model.py` \u542f\u52a8\u672c\u6d41\u7a0b\u3002\n", "\n", "| \u8f93\u5165/\u8f93\u51fa | \u672c\u6d41\u7a0b\u5b9a\u4e49 |\n", "| --- | --- |\n", "| \u4e24\u8def\u56fe\u50cf | \u5916\u90e8\u76f8\u673a\u3001\u8155\u90e8\u76f8\u673a\uff0c\u91c7\u96c6\u4e3a 256\u00d7256 RGB\uff3b\u7ea2\u7eff\u84dd\u5f69\u8272\u56fe\uff3d |\n", "| `observation.state` | 6 \u7ef4\u672b\u7aef\u4f4d\u59ff `[x,y,z,roll,pitch,yaw]`\uff0c\u4f4d\u7f6e\u4e3a\u7c73\uff0c\u65cb\u8f6c\u4e3a\u5f27\u5ea6 |\n", "| `action` | 7 \u7ef4\u7edd\u5bf9\u76ee\u6807\u5173\u8282\u89d2 `[joint1,\u2026,joint6,gripper]`\uff0c\u89d2\u5ea6\u4e3a\u5f27\u5ea6\uff0c\u5939\u722a 0 \u5f00\u30011 \u95ed |\n", "| \u4efb\u52a1 | \u6bcf\u5e27\u7684 `task` \u8bed\u8a00\u6307\u4ee4\uff0c\u6765\u81ea\u539f\u6570\u636e\u4efb\u52a1\u7d22\u5f15 |\n", "| `obj_init` | \u676f\u5b50\u4e0e\u76d8\u5b50\u7684\u4e24\u4e2a\u521d\u59cb\u4f4d\u7f6e\uff0c\u4ec5\u7528\u4e8e\u56de\u653e\u548c\u4eff\u771f\u91cd\u7f6e |\n", "| \u9891\u7387 | 20 Hz\uff3b\u6bcf\u79d2 20 \u6b21\u63a7\u5236\uff3d\uff0c\u4fdd\u7559\u91c7\u96c6\u9891\u7387 |\n", "\n", "**\u9002\u7528\u8303\u56f4\uff1a** \u7b2c 5 \u8282\u8bed\u8a00\u573a\u666f\u7684\u516d\u7ef4\u72b6\u6001\u662f\u5173\u8282\u89d2\uff0c\u7269\u4f53\u548c\u91cd\u7f6e\u63a5\u53e3\u4e5f\u4e0d\u540c\uff0c\u4e0d\u80fd\u76f4\u63a5\u5957\u7528\u8fd9\u91cc\u7684\u8f6c\u6362\u548c\u8bc4\u6d4b\u3002\u8fd9\u91cc\u4f7f\u7528\u7b2c 1 \u8282\u672b\u7aef\u4f4d\u59ff\u6570\u636e\u3002\u6a21\u578b\u5185\u90e8\u5904\u7406\u52a8\u4f5c\u586b\u5145\uff08\u9ed8\u8ba4 32 \u7ef4\uff09\uff1b\u65e0\u9700\u624b\u5de5\u628a\u4e03\u7ef4\u52a8\u4f5c\u6539\u4e3a 32 \u7ef4\u3002\u672b\u7aef\u4f4d\u59ff\u4e0e\u5173\u8282\u76ee\u6807\u4e0d\u662f\u540c\u4e00\u79cd\u91cf\uff0c\u56e0\u6b64\u660e\u786e\u8bbe\u7f6e `use_relative_actions=False`\u3002\n", "\n", "\u03c00.5 \u7684\u901a\u7528\u9884\u8bad\u7ec3\u6743\u91cd\u662f\u5fae\u8c03\u8d77\u70b9\uff0c\u8fd8\u4e0d\u662f\u5df2\u5b66\u4f1a OMY \u6293\u653e\u7684\u7b56\u7565\u3002\u9ed8\u8ba4\u53ea\u8bad\u7ec3\u52a8\u4f5c\u4e13\u5bb6\u5e76\u51bb\u7ed3\u89c6\u89c9\u7f16\u7801\u5668\uff0c\u6279\u91cf\u5927\u5c0f\u4e3a 1\uff0c\u542f\u7528\u68af\u5ea6\u68c0\u67e5\u70b9\u3002\u5b98\u65b9\u5b8c\u6574\u8bad\u7ec3\u793a\u4f8b\u4f7f\u7528\u5927\u663e\u5b58\u786c\u4ef6\uff1b\u8fd9\u4e9b\u8bbe\u7f6e\u4e5f\u4e0d\u4fdd\u8bc1\u80fd\u5728 6 GB \u663e\u5b58\u4e0a\u8bad\u7ec3\u3002\u5efa\u8bae\u5728\u5b9e\u9a8c\u5ba4\u7684 Linux\uff3b\u64cd\u4f5c\u7cfb\u7edf\uff3d\u5de5\u4f5c\u7ad9\u4e0a\u6267\u884c\uff0c\u4ee5\u4e94\u6b65\u68c0\u67e5\u65f6\u7684\u5b9e\u9645\u5cf0\u503c\u663e\u5b58\u4e3a\u51c6\u3002\u6570\u636e\u3001\u6a21\u578b\u7f13\u5b58\u548c\u8f93\u51fa\u653e\u5230\u5927\u5bb9\u91cf\u78c1\u76d8\u3002\n" ], "id": "pi05-omy-00" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1. \u521b\u5efa\u72ec\u7acb\u73af\u5883\n", "\n", "\u4ee5\u4e0b\u547d\u4ee4\u5728 **\u672c\u7ae0\u76ee\u5f55\u7684\u7ec8\u7aef**\u6267\u884c\u4e00\u6b21\uff0c\u7136\u540e\u5728\u7b14\u8bb0\u672c\u83dc\u5355\u9009\u62e9 `OMY pi05 (Python 3.12)` \u5185\u6838\u3002\u5148\u5b89\u88c5\u4e0e\u663e\u5361\u9a71\u52a8\u5339\u914d\u7684 PyTorch\uff3b\u5f20\u91cf\u8ba1\u7b97\u6846\u67b6\uff3d\u9884\u7f16\u8bd1\u5305\uff0c\u518d\u5b89\u88c5\u672c\u6d41\u7a0b\u4f9d\u8d56\u3002\u9700\u8981\u7cfb\u7edf\u5df2\u6709 `ffmpeg`\uff3b\u89c6\u9891\u7f16\u7801\u5de5\u5177\uff3d\u548c\u56fe\u5f62\u684c\u9762\uff1b\u672c\u7ae0 MuJoCo\uff3b\u7269\u7406\u4eff\u771f\u5668\uff3d\u7a97\u53e3\u4f9d\u8d56\u663e\u793a\u670d\u52a1\u3002\u65e0\u684c\u9762\u670d\u52a1\u5668\u53ef\u7528 `xvfb-run`\uff3b\u865a\u62df\u663e\u793a\u5de5\u5177\uff3d\u542f\u52a8\u7b14\u8bb0\u672c\u670d\u52a1\u3002\n", "\n", "```bash\n", "conda create -n omy-pi05 python=3.12 -y\n", "conda activate omy-pi05\n", "python -m pip install torch==2.7.1 torchvision==0.22.1 --index-url https://download.pytorch.org/whl/cu128\n", "python -m pip install -r requirements_pi05.txt\n", "python -m ipykernel install --user --name omy-pi05 --display-name 'OMY pi05 (Python 3.12)'\n", "jupyter lab .\n", "```\n", "\n", "\u5de5\u4f5c\u7ad9\u5982\u4f7f\u7528\u5176\u4ed6\u73af\u5883\u7ba1\u7406\u5de5\u5177\uff0c\u53ef\u5c06\u73af\u5883\u653e\u5728 `/data/Data14TB/envs/`\u3002\u5f00\u59cb\u524d\u5728\u7ec8\u7aef\u8bbe\u7f6e\u5de5\u4f5c\u76ee\u5f55\uff0c\u4f8b\u5982\uff1a\n", "\n", "```bash\n", "export OMY_PI05_ROOT=/data/Data14TB/robotics_shared/experiments/omy-pi05\n", "export HF_HOME=/data/Data14TB/robotics_shared/cache/huggingface\n", "```\n", "\n", "\u573a\u666f\u8d44\u6e90\u6cbf\u7528\u672c\u7ae0\u5b89\u88c5\u6b65\u9aa4\uff1a\u5c06 `asset/objaverse/plate_11.zip` \u89e3\u538b\u5230\u540c\u76ee\u5f55\u3002\u4e0d\u8981\u4fee\u6539\u539f\u91c7\u96c6\u6570\u636e\u76ee\u5f55\u3002\u9996\u6b21\u4e0b\u8f7d\u9884\u8bad\u7ec3\u6743\u91cd\u9700\u8981\u7f51\u7edc\u548c\u8db3\u591f\u78c1\u76d8\uff1b\u77ed\u6b65\u68c0\u67e5\u3001\u6b63\u5f0f\u8bad\u7ec3\u5404\u81ea\u4fdd\u5b58\u5b8c\u6574\u6a21\u578b\u4e0e\u8bad\u7ec3\u72b6\u6001\uff0c\u5408\u8ba1\u7a7a\u95f4\u53ef\u80fd\u8fbe\u5230\u6570\u5341 GB\u3002\n" ], "id": "pi05-omy-01" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import json\n", "import shlex\n", "import subprocess\n", "from pathlib import Path\n", "from datetime import datetime\n", "\n", "CHAPTER = Path.cwd().resolve()\n", "assert (CHAPTER / \"pi05_omy_utils.py\").is_file(), \"Start Jupyter in this chapter directory\"\n", "SOURCE = CHAPTER / \"demo_data\" # Newly collected chapter-1 data; not demo_data_language.\n", "RUN_ROOT = Path(os.environ.get(\"OMY_PI05_ROOT\", CHAPTER / \"pi05_outputs\")).resolve()\n", "V3_ROOT = RUN_ROOT / \"omy_v3\"\n", "REPO_ID = \"local/omy-pi05-20hz\"\n", "RUN_ID = datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n", "SMOKE_OUTPUT = RUN_ROOT / f\"smoke-{RUN_ID}\"\n", "TRAIN_OUTPUT = RUN_ROOT / f\"train-{RUN_ID}\"\n", "BASE_MODEL = os.environ.get(\"OMY_PI05_PRETRAINED\", \"lerobot/pi05_base\")\n", "SMOKE_STEPS = 5\n", "CHUNK_SIZE = 50\n", "N_ACTION_STEPS = 5\n", "TRAIN_STEPS = 10000\n", "BATCH_SIZE = 1\n", "\n", "# Set True only after checking the current collector and replaying your demonstrations.\n", "ACTION_LABELS_VERIFIED = False\n", "RUN_SMOKE = False\n", "RUN_TRAIN = False\n", "\n", "RUN_ROOT.mkdir(parents=True, exist_ok=True)\n", "os.environ.setdefault(\"HF_HOME\", str(RUN_ROOT / \"hf_cache\"))\n", "from pi05_omy_utils import (check_version, read_v21, prepare_dataset,\n", " verify_alignment, training_command, load_policy, rollout)\n", "print(\"source:\", SOURCE)\n", "print(\"outputs:\", RUN_ROOT)\n" ], "id": "pi05-omy-02" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import importlib.metadata as metadata\n", "import torch\n", "\n", "check_version()\n", "assert sys.version_info[:2] == (3, 12), sys.version\n", "assert torch.__version__.split(\"+\")[0] == \"2.7.1\", torch.__version__\n", "assert torch.cuda.is_available(), \"Select the CUDA workstation kernel\"\n", "assert torch.cuda.is_bf16_supported(), \"This configuration requires BF16 support\"\n", "assert metadata.version(\"mujoco\") == \"3.1.6\"\n", "free, total = torch.cuda.mem_get_info()\n", "print(\"Python:\", sys.version.split()[0])\n", "print(\"LeRobot:\", metadata.version(\"lerobot\"))\n", "print(\"PyTorch / CUDA:\", torch.__version__, torch.version.cuda)\n", "print(\"device:\", torch.cuda.get_device_name(0))\n", "print(f\"free / total VRAM: {free/2**30:.1f} / {total/2**30:.1f} GiB\")\n", "if free < 20 * 2**30:\n", " print(\"Low free VRAM: reduce workload or use the larger workstation before training\")\n" ], "id": "pi05-omy-03" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. \u91c7\u96c6\u3001\u56de\u653e\u5e76\u786e\u8ba4\u52a8\u4f5c\u6807\u7b7e\n", "\n", "\u5148\u5728\u65e7\u7248\u91c7\u96c6\u73af\u5883\u8fd0\u884c\u7b2c 1 \u8282\uff0c\u589e\u52a0 `NUM_DEMO`\uff0c\u91c7\u96c6\u4e0d\u540c\u676f\u5b50\u4f4d\u7f6e\u7684\u5b8c\u6574\u6210\u529f\u793a\u6559\uff0c\u518d\u7528\u7b2c 2 \u8282\u56de\u653e\u3002\u5efa\u8bae\u5148\u79ef\u7d2f 30\uff5e50 \u4e2a\u56de\u5408\u68c0\u67e5\u6d41\u7a0b\uff0c\u6240\u9700\u6570\u91cf\u6700\u7ec8\u53d6\u51b3\u4e8e\u4efb\u52a1\u8986\u76d6\u548c\u8bc4\u6d4b\u7ed3\u679c\u3002\n", "\n", "**2026-10-02 \u524d\u7684\u65e7\u91c7\u96c6\u5165\u53e3\u66fe\u628a\u6d4b\u91cf\u5173\u8282\u72b6\u6001\u5b58\u6210\u52a8\u4f5c\u6807\u7b7e\u3002\u683c\u5f0f\u8f6c\u6362\u4e0d\u80fd\u6062\u590d\u4e22\u5931\u7684\u76ee\u6807\u547d\u4ee4\u3002** \u8bf7\u4f7f\u7528\u4fee\u590d\u540e\u7684 `get_commanded_joint_action()` \u63a5\u53e3\u91cd\u65b0\u91c7\u96c6\uff0c\u6216\u786e\u8ba4\u6570\u636e\u6765\u6e90\u5df2\u7ecf\u91c7\u7528\u8be5\u63a5\u53e3\u3002\u4ed3\u5e93 `demo_data_example` \u662f\u5386\u53f2\u7ed3\u6784\u793a\u4f8b\uff0c\u4e0d\u4f5c\u4e3a\u8fd9\u91cc\u7684\u8bad\u7ec3\u9ed8\u8ba4\u503c\u3002\n", "\n", "\u4e0b\u9762\u7684\u68c0\u67e5\u53ef\u4ee5\u53d1\u73b0\u7f3a\u5931\u6587\u4ef6\u3001\u9519\u8bef\u7ef4\u5ea6\u3001\u65f6\u95f4\u6233\u3001\u5939\u722a\u8303\u56f4\u3001\u56fe\u50cf\u548c\u56de\u5408\u7d22\u5f15\u95ee\u9898\uff0c**\u4e0d\u80fd\u4ec5\u51ed\u6570\u7ec4\u5224\u65ad\u52a8\u4f5c\u6807\u7b7e\u6765\u6e90\u6216\u72b6\u6001\u542b\u4e49**\u3002\u786e\u8ba4\u7b2c 1 \u8282\u672b\u7aef\u4f4d\u59ff\u3001\u5b9e\u9645\u4e0b\u53d1\u7684\u5173\u8282\u76ee\u6807\u4e0e\u56de\u653e\u5747\u6b63\u786e\u540e\uff0c\u628a\u914d\u7f6e\u5355\u5143\u7684 `ACTION_LABELS_VERIFIED` \u8bbe\u4e3a `True`\u3002\n" ], "id": "pi05-omy-04" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "info, episodes = read_v21(SOURCE)\n", "print({key: info[key] for key in [\"codebase_version\", \"fps\", \"total_episodes\", \"total_frames\"]})\n", "print(\"tasks:\", sorted({row[\"task\"] for rows in episodes for row in rows}))\n", "print(\"first state:\", episodes[0][0][\"observation.state\"])\n", "print(\"first action:\", episodes[0][0][\"action\"])\n", "print(\"action semantics must be verified from the collector and replay\")\n" ], "id": "pi05-omy-05" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from pi05_omy_utils import _image\n", "\n", "first = episodes[0][0]\n", "fig, axes = plt.subplots(1, 2, figsize=(8, 4))\n", "for ax, key in zip(axes, [\"observation.image\", \"observation.wrist_image\"]):\n", " ax.imshow(_image(first[key], SOURCE.resolve(), (256, 256, 3)))\n", " ax.set_title(key)\n", " ax.axis(\"off\")\n", "plt.show()\n", "actions = np.stack([row[\"action\"] for row in episodes[0]])\n", "plt.figure(figsize=(10, 3))\n", "plt.plot(np.arange(len(actions))/20, actions)\n", "plt.xlabel(\"simulation time (s)\")\n", "plt.ylabel(\"joint target (rad) / normalized gripper\")\n", "plt.legend([f\"joint{i}\" for i in range(1, 7)] + [\"gripper\"], ncol=4)\n", "plt.show()\n" ], "id": "pi05-omy-06" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. \u8f6c\u6362\u4e3a\u72ec\u7acb\u7684 LeRobot v3 \u6570\u636e\n", "\n", "\u672c\u7ae0 v2.1 \u56fe\u50cf\u5b58\u50a8\u5728 Parquet\uff3b\u5217\u5f0f\u6570\u636e\u6587\u4ef6\uff3d\u4e2d\u3002\u8f85\u52a9\u7a0b\u5e8f\u7528\u65b0\u7248\u5b98\u65b9\u6570\u636e\u5199\u5165\u63a5\u53e3\u9010\u56de\u5408\u91cd\u5efa v3 \u6570\u636e\uff0c\u4fdd\u7559\u56fe\u50cf\u3001\u516d\u7ef4\u72b6\u6001\u3001\u4e03\u7ef4\u52a8\u4f5c\u3001\u4efb\u52a1\u3001\u521d\u59cb\u4f4d\u7f6e\u548c 20 Hz\uff3b\u6bcf\u79d2 20 \u6b21\u63a7\u5236\uff3d\u9891\u7387\uff0c\u4ec5\u89c4\u8303\u76f8\u673a\u952e\u540d\u4e0e\u7d22\u5f15\u3002\n", "\n", "\u76ee\u6807\u76ee\u5f55\u5fc5\u987b\u4e0d\u5b58\u5728\uff1b\u91cd\u590d\u6267\u884c\u65f6\u9009\u62e9\u65b0\u76ee\u5f55\uff0c\u4e0d\u4f1a\u81ea\u52a8\u5220\u9664\u539f\u6570\u636e\u6216\u5df2\u6709\u8f6c\u6362\u7ed3\u679c\u3002\u5df2\u6709\u672c\u6d41\u7a0b\u7684\u8f6c\u6362\u7ed3\u679c\u53ef\u8df3\u8fc7\u8f6c\u6362\u5355\u5143\uff0c\u76f4\u63a5\u8bfb\u53d6 `omy_manifest.json` \u540e\u7ee7\u7eed\u6821\u9a8c\u3002\n", "\n", "\u4e09\u4e2a\u4ee5\u4e0a\u56de\u5408\u65f6\uff0c\u6309\u5b8c\u6574\u56de\u5408\u968f\u673a\u7559\u51fa\u7ea6 20% \u4f5c\u8bc4\u6d4b\uff0c\u56fa\u5b9a\u79cd\u5b50\u4e3a 42\u3002\u4ec5\u7528\u8bad\u7ec3\u56de\u5408\u8ba1\u7b97\u72b6\u6001\u548c\u52a8\u4f5c\u7684\u7cbe\u786e `q01/q99`\uff3b1%/99% \u5206\u4f4d\u6570\uff3d\u53ca\u5747\u503c\u3001\u6807\u51c6\u5dee\u7b49\u7edf\u8ba1\u91cf\uff0c\u4e0d\u628a\u7559\u51fa\u56de\u5408\u7528\u4e8e\u5f52\u4e00\u5316\u3002\u4e00\u4e24\u6761\u793a\u6559\u53ea\u9002\u5408\u63a5\u53e3\u68c0\u67e5\u6216\u8fc7\u62df\u5408\u8c03\u8bd5\uff0c\u4e0d\u62a5\u544a\u6cdb\u5316\u6210\u529f\u7387\u3002\n" ], "id": "pi05-omy-07" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "manifest = prepare_dataset(SOURCE, V3_ROOT, REPO_ID,\n", " labels_verified=ACTION_LABELS_VERIFIED, split_seed=42)\n", "print(json.dumps(manifest, indent=2, ensure_ascii=False))\n" ], "id": "pi05-omy-08" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# This cell can also reopen an existing conversion produced by this notebook.\n", "manifest = json.loads((V3_ROOT / \"omy_manifest.json\").read_text(encoding=\"utf-8\"))\n", "print(verify_alignment(V3_ROOT, chunk_size=CHUNK_SIZE))\n", "stats = json.loads((V3_ROOT / \"meta/stats.json\").read_text(encoding=\"utf-8\"))\n", "for key in [\"observation.state\", \"action\"]:\n", " print(key, \"q01:\", stats[key][\"q01\"], \"q99:\", stats[key][\"q99\"])\n", "print(\"train episodes:\", manifest[\"train_episodes\"])\n", "print(\"held-out episodes:\", manifest[\"heldout_episodes\"])\n" ], "id": "pi05-omy-09" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. \u68c0\u67e5\u6a21\u578b\u8bbf\u95ee\u5e76\u51c6\u5907\u8bad\u7ec3\u547d\u4ee4\n", "\n", "\u5fae\u8c03\u9ed8\u8ba4\u4f7f\u7528 [lerobot/pi05_base](https://huggingface.co/lerobot/pi05_base)\u3002\u5df2\u6709\u517c\u5bb9\u7684\u672c\u5730\u9884\u8bad\u7ec3\u76ee\u5f55\u65f6\uff0c\u5728\u542f\u52a8\u524d\u8bbe\u7f6e `OMY_PI05_PRETRAINED`\uff0c\u6216\u76f4\u63a5\u4fee\u6539\u914d\u7f6e\u5355\u5143\u7684 `BASE_MODEL`\uff0c\u53ef\u590d\u7528\u78c1\u76d8\u4e0a\u7684\u6743\u91cd\u3002\u8be5\u76ee\u5f55\u9700\u8981\u6a21\u578b\u3001LeRobot \u914d\u7f6e\u4ee5\u53ca\u8f93\u5165\u3001\u8f93\u51fa\u5904\u7406\u5668\u914d\u7f6e\uff1b\u539f\u59cb OpenPi \u76ee\u5f55\u9700\u8981\u5148\u51c6\u5907\u8fd9\u4e9b\u9002\u914d\u6587\u4ef6\u3002\u8f93\u5165\u5904\u7406\u8fd8\u9700\u8981 `google/paligemma-3b-pt-224` \u7684\u5206\u8bcd\u5668\u3002\u82e5\u63d0\u793a\u8bbf\u95ee\u53d7\u9650\uff0c\u5148\u5728\u5176\u6a21\u578b\u9875\u9762\u63a5\u53d7\u8bb8\u53ef\uff0c\u518d\u5728\u7ec8\u7aef\u6267\u884c `hf auth login`\uff3b\u6a21\u578b\u5e73\u53f0\u8d26\u53f7\u767b\u5f55\uff3d\uff1b\u4e0d\u8981\u628a\u8bbf\u95ee\u4ee4\u724c\u5199\u8fdb\u7b14\u8bb0\u672c\u3002\n", "\n", "\u8bad\u7ec3\u901a\u8fc7\u672c\u7ae0 `pi05_omy_train.py` \u8c03\u7528\u5b98\u65b9\u8bad\u7ec3\u5165\u53e3\uff0c\u5e76\u68c0\u67e5\u5168\u90e8\u9884\u8bad\u7ec3\u53c2\u6570\u52a0\u8f7d\u6210\u529f\u3002\u56fa\u5b9a\u7248\u672c\u7684\u5b98\u65b9\u52a0\u8f7d\u5668\u53ef\u80fd\u5728\u8bfb\u53d6\u6743\u91cd\u5931\u8d25\u540e\u8fd4\u56de\u968f\u673a\u521d\u59cb\u5316\u6a21\u578b\uff0c\u672c\u7ae0\u4f1a\u628a\u8fd9\u79cd\u60c5\u51b5\u4f5c\u4e3a\u9519\u8bef\u505c\u6b62\uff1b\u63a8\u7406\u52a0\u8f7d\u4e5f\u4f7f\u7528\u540c\u4e00\u68c0\u67e5\u3002\n", "\n", "\u76f8\u673a\u952e\u6620\u5c04\u4e3a `observation.images.image` \u548c `observation.images.image2`\uff0c\u6a21\u578b\u7279\u5f81\u4ece\u8f6c\u6362\u540e\u7684\u5143\u6570\u636e\u53d6\u5f97\u3002\u8bed\u8a00\u4e0e\u91cf\u5316\u540e\u7684\u672b\u7aef\u72b6\u6001\u4e00\u8d77\u9001\u5165\u5206\u8bcd\u5668\u3002\u8bad\u7ec3\u548c\u63a8\u7406\u90fd\u4f7f\u7528\u5b98\u65b9\u8f93\u5165\u3001\u8f93\u51fa\u5904\u7406\u5668\uff0c\u4e0d\u80fd\u53ea\u8bfb\u53d6\u6743\u91cd\u540e\u76f4\u63a5\u628a\u539f\u59cb\u6570\u7ec4\u9001\u8fdb\u6a21\u578b\u3002\n", "\n", "`chunk_size=50`\uff3b\u9884\u6d4b 50 \u6b65\u52a8\u4f5c\u5757\uff3d\u4fdd\u6301\u9884\u8bad\u7ec3\u9ed8\u8ba4\u503c\uff1b`n_action_steps=5`\uff3b\u8fde\u7eed\u6267\u884c\u5176\u4e2d 5 \u6b65\uff3d\u4f7f\u7b56\u7565\u7ea6\u6bcf 0.25 \u79d2\u4eff\u771f\u65f6\u95f4\u91cd\u65b0\u89c2\u5bdf\u3002\u77ed\u6b65\u68c0\u67e5\u53ea\u9a8c\u8bc1\u6570\u636e\u3001\u524d\u5411\u3001\u53cd\u5411\u3001\u4fdd\u5b58\u4e0e\u52a0\u8f7d\u94fe\u8def\uff0c\u4e0d\u4ee3\u8868\u5b66\u4f1a\u6293\u653e\u3002\n" ], "id": "pi05-omy-10" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from huggingface_hub import hf_hub_download\n", "from transformers import AutoTokenizer\n", "\n", "# Only config/tokenizer files are requested here; weights load when training starts.\n", "if Path(BASE_MODEL).is_dir():\n", " required = [\"config.json\", \"model.safetensors\", \"policy_preprocessor.json\", \"policy_postprocessor.json\"]\n", " assert all((Path(BASE_MODEL) / name).is_file() for name in required), \"Incomplete local pretrained directory\"\n", " model_config = json.loads((Path(BASE_MODEL) / \"config.json\").read_text(encoding=\"utf-8\"))\n", "else:\n", " model_config = json.loads(Path(hf_hub_download(BASE_MODEL, \"config.json\")).read_text(encoding=\"utf-8\"))\n", "assert model_config[\"type\"] == \"pi05\", model_config.get(\"type\")\n", "tokenizer = AutoTokenizer.from_pretrained(\"google/paligemma-3b-pt-224\")\n", "print(\"model configuration and tokenizer access: OK\")\n", "smoke_cmd = training_command(V3_ROOT, SMOKE_OUTPUT, steps=SMOKE_STEPS,\n", " batch_size=BATCH_SIZE, chunk_size=CHUNK_SIZE, n_action_steps=N_ACTION_STEPS,\n", " pretrained=BASE_MODEL)\n", "train_cmd = training_command(V3_ROOT, TRAIN_OUTPUT, steps=TRAIN_STEPS,\n", " batch_size=BATCH_SIZE, chunk_size=CHUNK_SIZE, n_action_steps=N_ACTION_STEPS,\n", " pretrained=BASE_MODEL)\n", "print(f\"{SMOKE_STEPS}-step check:\\n\", shlex.join(smoke_cmd))\n", "print(\"full training:\\n\", shlex.join(train_cmd))\n" ], "id": "pi05-omy-11" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def run_training(command, log_path):\n", " # Stream logs; interrupting the cell stops only the process started by this cell.\n", " with Path(log_path).open(\"w\", encoding=\"utf-8\") as log:\n", " process = subprocess.Popen(command, cwd=CHAPTER, stdout=subprocess.PIPE,\n", " stderr=subprocess.STDOUT, text=True, bufsize=1)\n", " try:\n", " for line in process.stdout:\n", " log.write(line)\n", " log.flush()\n", " print(line, end=\"\")\n", " returncode = process.wait()\n", " except BaseException:\n", " process.terminate()\n", " try:\n", " process.wait(timeout=15)\n", " except subprocess.TimeoutExpired:\n", " process.kill()\n", " process.wait()\n", " raise\n", " if returncode:\n", " raise RuntimeError(f\"Training failed ({returncode}); inspect {log_path}\")\n", "\n", "if RUN_SMOKE:\n", " run_training(smoke_cmd, RUN_ROOT / f\"smoke-{RUN_ID}.log\")\n", " smoke_checkpoint = SMOKE_OUTPUT / \"checkpoints/last/pretrained_model\"\n", " assert (smoke_checkpoint / \"config.json\").is_file(), smoke_checkpoint\n", "else:\n", " print(\"Set RUN_SMOKE=True in the configuration cell when ready\")\n" ], "id": "pi05-omy-12" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. \u6b63\u5f0f\u5fae\u8c03\u4e0e\u65ad\u70b9\u6062\u590d\n", "\n", "\u4e94\u6b65\u68c0\u67e5\u6b63\u5e38\u7ed3\u675f\u540e\uff0c\u68c0\u67e5\u65e5\u5fd7\u4e2d\u7684\u635f\u5931\u3001\u68af\u5ea6\u8303\u6570\u3001\u5cf0\u503c\u663e\u5b58\uff0c\u4ee5\u53ca\u6700\u540e\u4fdd\u5b58\u7684\u5b8c\u6574\u68c0\u67e5\u70b9\u3002\u5c06 `RUN_TRAIN` \u8bbe\u4e3a `True`\uff0c\u91cd\u65b0\u751f\u6210\u6b63\u5f0f\u8bad\u7ec3\u547d\u4ee4\u5e76\u6267\u884c\u4e0b\u9762\u5355\u5143\u3002\n", "\n", "\u521d\u59cb\u53c2\u6570\u4e3a 10000 \u6b65\u3001\u6279\u91cf\u5927\u5c0f 1\uff0c\u6bcf 2000 \u6b65\u4fdd\u5b58\u3002\u6570\u636e\u91cf\u5c0f\u65f6\u5148\u89c2\u5bdf\u662f\u5426\u8fc7\u62df\u5408\uff0c\u6700\u7ec8\u4ee5\u7559\u51fa\u573a\u666f\u6293\u653e\u6210\u529f\u7387\u51b3\u5b9a\u6a21\u578b\u662f\u5426\u53ef\u7528\u3002\u8bad\u7ec3\u6b65\u6570\u548c\u6279\u91cf\u5927\u5c0f\u4e0d\u662f\u5df2\u7ecf\u9a8c\u8bc1\u7684\u6700\u4f18\u503c\u3002\n", "\n", "\u4e2d\u65ad\u540e\u53ef\u4ee5\u7528\u4fdd\u5b58\u7684\u5b8c\u6574\u8bad\u7ec3\u914d\u7f6e\u6062\u590d\u6a21\u578b\u3001\u4f18\u5316\u5668\u548c\u8c03\u5ea6\u5668\u3002\u6062\u590d\u547d\u4ee4\u5f62\u5982\uff1a\n", "\n", "```bash\n", "python -m pi05_omy_train --config_path=/absolute/path/train-*/checkpoints/last/pretrained_model/train_config.json --resume=true\n", "```\n", "\n", "\u8bf7\u628a\u5360\u4f4d\u8def\u5f84\u66ff\u6362\u6210\u5b9e\u9645\u6587\u4ef6\uff0c\u4e0d\u8981\u628a\u65b0\u8bad\u7ec3\u8f93\u51fa\u76ee\u5f55\u6307\u5411\u5df2\u6709\u7ed3\u679c\u3002\u77ed\u6b65\u68c0\u67e5\u4fdd\u5b58\u7684\u6a21\u578b\u53ea\u80fd\u7528\u4e8e\u94fe\u8def\u68c0\u67e5\uff1b\u4e0b\u9762\u8bc4\u6d4b\u9ed8\u8ba4\u8bfb\u53d6\u6b63\u5f0f\u8bad\u7ec3\u76ee\u5f55\u3002\n" ], "id": "pi05-omy-13" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "if RUN_TRAIN:\n", " # Rebuild to ensure the output directory is still new before launching.\n", " train_cmd = training_command(V3_ROOT, TRAIN_OUTPUT, steps=TRAIN_STEPS,\n", " batch_size=BATCH_SIZE, chunk_size=CHUNK_SIZE, n_action_steps=N_ACTION_STEPS,\n", " pretrained=BASE_MODEL)\n", " run_training(train_cmd, RUN_ROOT / f\"train-{RUN_ID}.log\")\n", "else:\n", " print(\"Set RUN_TRAIN=True after the five-step check passes\")\n" ], "id": "pi05-omy-14" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6. \u52a0\u8f7d\u6b63\u5f0f\u68c0\u67e5\u70b9\u5e76\u5bf9\u6bd4\u7559\u51fa\u52a8\u4f5c\n", "\n", "\u5982\u5728\u7ec8\u7aef\u8bad\u7ec3\u6216\u91cd\u65b0\u6253\u5f00\u7b14\u8bb0\u672c\uff0c\u5148\u628a `CHECKPOINT` \u6539\u6210\u5b9e\u9645\u7684 `pretrained_model` \u76ee\u5f55\u3002\u52a0\u8f7d\u65f6\u540c\u65f6\u8bfb\u53d6\u4fdd\u5b58\u7684\u5f52\u4e00\u5316\u3001\u5206\u8bcd\u4e0e\u8bbe\u5907\u5904\u7406\u914d\u7f6e\uff0c\u4e0d\u91cd\u65b0\u4f30\u8ba1\u7edf\u8ba1\u91cf\u3002\n", "\n", "\u79bb\u7ebf\u6bd4\u8f83\u7684\u662f\u9996\u6b65\u7684\u516d\u4e2a\u5173\u8282\u76ee\u6807\u53ca\u5939\u722a\u547d\u4ee4\u3002\u4e24\u8005\u5355\u4f4d\u4e0d\u540c\uff0c\u5206\u522b\u62a5\u544a\u8bef\u5dee\u3002\u79bb\u7ebf\u8bef\u5dee\u4e0d\u80fd\u4ee3\u66ff\u4eff\u771f\u6210\u529f\u7387\uff1b\u8fd9\u91cc\u6700\u591a\u68c0\u67e5\u516b\u5e27\uff0c\u4fbf\u4e8e\u5148\u68c0\u67e5\u8f93\u51fa\u5f62\u72b6\u548c\u6570\u503c\u3002\n" ], "id": "pi05-omy-15" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "CHECKPOINT = TRAIN_OUTPUT / \"checkpoints/last/pretrained_model\"\n", "policy, preprocessor, postprocessor = load_policy(CHECKPOINT)\n", "print(\"state:\", policy.config.input_features[\"observation.state\"])\n", "print(\"action:\", policy.config.output_features[\"action\"])\n" ], "id": "pi05-omy-16" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from lerobot.datasets.lerobot_dataset import LeRobotDataset\n", "import pyarrow.parquet as pq\n", "\n", "assert manifest[\"heldout_episodes\"], \"Collect at least three demos for a held-out evaluation\"\n", "dataset = LeRobotDataset(REPO_ID, root=V3_ROOT,\n", " delta_timestamps={\"action\": [i / 20 for i in range(policy.config.chunk_size)]})\n", "index_rows = []\n", "for path in sorted((V3_ROOT / \"data\").rglob(\"*.parquet\")):\n", " index_rows.extend(pq.read_table(path).select([\"index\", \"episode_index\", \"frame_index\", \"obj_init\"]).to_pylist())\n", "eval_indices = [r[\"index\"] for r in index_rows if r[\"episode_index\"] in manifest[\"heldout_episodes\"]]\n", "chosen = np.linspace(0, len(eval_indices) - 1, min(8, len(eval_indices)), dtype=int)\n", "errors = []\n", "for offset in chosen:\n", " sample = dataset[eval_indices[offset]]\n", " observation = {key: sample[key] for key in [\"observation.images.image\", \"observation.images.image2\", \"observation.state\", \"task\"]}\n", " policy.reset()\n", " preprocessor.reset()\n", " postprocessor.reset()\n", " with torch.inference_mode():\n", " predicted = postprocessor(policy.predict_action_chunk(preprocessor(observation)))[0, 0].cpu().numpy()\n", " target = sample[\"action\"][0].numpy()\n", " assert predicted.shape == (7,) and np.isfinite(predicted).all()\n", " errors.append(np.abs(predicted - target))\n", "errors = np.stack(errors)\n", "print(\"joint target MAE (rad):\", errors[:, :6].mean(axis=0))\n", "print(\"gripper MAE:\", errors[:, 6].mean())\n" ], "id": "pi05-omy-17" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 7. \u5728 OMY \u5355\u676f\u573a\u666f\u95ed\u73af\u8bc4\u6d4b\n", "\n", "\u8bad\u7ec3\u73af\u5883\u6ca1\u6709\u767b\u8bb0\u4e3a\u5b98\u65b9\u8bc4\u6d4b\u73af\u5883\uff0c\u56e0\u6b64\u8bad\u7ec3\u65f6\u5173\u95ed\u5b98\u65b9\u73af\u5883\u8bc4\u6d4b\uff0c\u4f7f\u7528\u672c\u8282\u7684\u672c\u7ae0\u573a\u666f\u3002\u5148\u6309\u7559\u51fa\u56de\u5408 `obj_init` \u91cd\u7f6e\u676f\u5b50\u3001\u76d8\u5b50\u4f4d\u7f6e\uff0c\u518d\u8ba9\u7b56\u7565\u72ec\u7acb\u6267\u884c\u6293\u653e\uff0c\u8bb0\u5f55\u771f\u5b9e\u6210\u529f\u5224\u5b9a\u3002\n", "\n", "\u6bcf\u6b21\u91c7\u96c6\u5f53\u524d\u4e24\u8def\u56fe\u50cf\u4e0e\u672b\u7aef\u4f4d\u59ff\uff0c\u9884\u6d4b\u7edd\u5bf9\u5173\u8282\u76ee\u6807\uff0c\u7528 `action_type='joint_angle'` \u4e0b\u53d1\u3002\u6bcf\u4e2a\u63a7\u5236\u5468\u671f\u63a8\u8fdb 0.05 \u79d2\u7269\u7406\u4eff\u771f\uff0c\u9891\u7387\u4e0e 20 Hz\uff3b\u6bcf\u79d2 20 \u6b21\u63a7\u5236\uff3d\u793a\u6559\u4e00\u81f4\u3002\u63a8\u7406\u8017\u65f6\u53ef\u80fd\u4f7f\u5899\u4e0a\u65f6\u949f\u4e0b\u7684\u64ad\u653e\u66f4\u6162\uff0c\u4f46\u4e0d\u4f1a\u901a\u8fc7\u6539\u6570\u636e\u9891\u7387\u6765\u52a0\u901f\u3002\u6b64\u5904\u540c\u6b65\u63a7\u5236\u4e0d\u7b49\u4e8e\u6ee1\u8db3\u771f\u673a\u5b9e\u65f6\u63a7\u5236\u8981\u6c42\u3002\n", "\n", "\u73af\u5883\u4f1a\u88c1\u526a\u5939\u722a\u547d\u4ee4\u5e76\u9650\u5236\u5f00\u5408\u901f\u7387\uff0c\u6240\u4ee5\u540c\u65f6\u4fdd\u5b58\u6a21\u578b\u539f\u59cb\u9884\u6d4b\u548c\u5b9e\u9645\u4e0b\u53d1\u52a8\u4f5c\u3002\u673a\u68b0\u81c2\u63a7\u5236\u6cbf\u7528\u672c\u7ae0\u7b80\u5316\u4eff\u771f\u63a5\u53e3\uff1b\u7ed3\u679c\u7528\u4e8e\u8be5\u573a\u666f\u7684\u7b56\u7565\u68c0\u67e5\u3002\u6bcf\u56de\u5408\u6700\u591a 600 \u6b65\uff0c\u5373 30 \u79d2\u4eff\u771f\u65f6\u95f4\u3002\u89c6\u9891\u3001\u52a8\u4f5c\u4e0e\u6210\u529f\u7387\u5199\u5230\u65b0\u7684\u8bc4\u6d4b\u76ee\u5f55\uff0c\u4e0d\u8986\u76d6\u65e7\u7ed3\u679c\u3002\n" ], "id": "pi05-omy-18" }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import cv2\n", "assert os.environ.get(\"DISPLAY\"), \"Run on the Ubuntu desktop or launch Jupyter with xvfb-run\"\n", "from mujoco_env.y_env import SimpleEnv\n", "EVAL_OUTPUT = RUN_ROOT / f\"eval-{datetime.now().strftime('%Y%m%d-%H%M%S')}\"\n", "EVAL_OUTPUT.mkdir(exist_ok=False)\n", "EVAL_EPISODES = manifest[\"heldout_episodes\"][:5]\n", "assert EVAL_EPISODES, \"No held-out episodes; collect more demonstrations\"\n", "records = []\n", "env = SimpleEnv(xml_path=str(CHAPTER / \"asset/example_scene_y.xml\"),\n", " action_type=\"joint_angle\", state_type=\"ee_pose\", seed=42)\n", "try:\n", " for episode in EVAL_EPISODES:\n", " initial_row = next(r for r in index_rows if r[\"episode_index\"] == episode and r[\"frame_index\"] == 0)\n", " initial = np.asarray(initial_row[\"obj_init\"], dtype=np.float32)\n", " env.reset(seed=42 + episode)\n", " env.set_obj_pose(initial[:3], initial[3:])\n", " for _ in range(100):\n", " env.step_env()\n", " task = dataset[initial_row[\"index\"]][\"task\"]\n", " torch.manual_seed(42 + episode)\n", " result = rollout(policy, preprocessor, postprocessor, env, task=task, fps=20, max_steps=600)\n", " frames = result.pop(\"frames\")\n", " video_path = EVAL_OUTPUT / f\"episode-{episode:03d}.mp4\"\n", " height, width = frames[0].shape[:2]\n", " writer = cv2.VideoWriter(str(video_path), cv2.VideoWriter_fourcc(*\"mp4v\"), 20, (width, height))\n", " assert writer.isOpened(), \"Video encoder could not open the output\"\n", " try:\n", " for frame in frames:\n", " writer.write(cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))\n", " finally:\n", " writer.release()\n", " result[\"predicted_actions\"] = [a.tolist() for a in result[\"predicted_actions\"]]\n", " result[\"applied_actions\"] = [a.tolist() for a in result[\"applied_actions\"]]\n", " result.update({\"episode\": episode, \"seed\": 42 + episode, \"task\": task, \"video\": str(video_path), \"checkpoint\": str(CHECKPOINT)})\n", " (EVAL_OUTPUT / f\"episode-{episode:03d}.json\").write_text(json.dumps(result, indent=2), encoding=\"utf-8\")\n", " records.append(result)\n", " print(\"episode:\", episode, \"success:\", result[\"success\"], \"steps:\", result[\"steps\"])\n", "finally:\n", " env.env.close_viewer()\n", "\n", "summary = {\"evaluated_episodes\": EVAL_EPISODES,\n", " \"successes\": sum(r[\"success\"] for r in records), \"episodes\": len(records),\n", " \"success_rate\": sum(r[\"success\"] for r in records) / len(records),\n", " \"checkpoint\": str(CHECKPOINT)}\n", "(EVAL_OUTPUT / \"summary.json\").write_text(json.dumps(summary, indent=2), encoding=\"utf-8\")\n", "print(summary)\n" ], "id": "pi05-omy-19" }, { "cell_type": "markdown", "metadata": {}, "source": [ "## \u68c0\u67e5\u4e0e\u5e38\u89c1\u95ee\u9898\n", "\n", "- \u5728\u72ec\u7acb\u73af\u5883\u4e2d\u8fd0\u884c `python -m unittest -v test_pi05_omy`\u3002\u68c0\u67e5\u4f7f\u7528\u771f\u5b9e\u65b0\u7248\u6570\u636e\u5199\u5165\u548c\u8bfb\u53d6\u63a5\u53e3\uff0c\u8986\u76d6\u56fe\u50cf/\u52a8\u4f5c\u4fdd\u771f\u3001\u6253\u4e71\u884c\u987a\u5e8f\u3001\u56de\u5408\u672b\u5c3e\u52a8\u4f5c\u586b\u5145\u3001\u8bad\u7ec3\u56de\u5408\u5206\u4f4d\u6570\u3001\u539f\u6570\u636e\u4fdd\u7559\u548c 20 Hz\uff3b\u6bcf\u79d2 20 \u6b21\u63a7\u5236\uff3d\u4e0b\u53d1\u65f6\u5e8f\u3002\n", "- \u76ee\u6807\u76ee\u5f55\u5df2\u7ecf\u5b58\u5728\uff1a\u6539\u7528\u65b0\u7684 `V3_ROOT`\uff0c\u6216\u8df3\u8fc7\u8f6c\u6362\u5355\u5143\u5e76\u8bfb\u53d6\u5df2\u6709\u6e05\u5355\uff1b\u4e0d\u8981\u5220\u9664\u539f\u793a\u6559\u3002\n", "- \u6ca1\u6709 `q01/q99`\uff3b1%/99% \u5206\u4f4d\u6570\uff3d\uff1a\u786e\u8ba4\u4f7f\u7528\u672c\u6d41\u7a0b\u8f6c\u6362\u540e\u7684\u6570\u636e\uff0c\u68c0\u67e5 `meta/stats.json`\uff1b\u4e0d\u8981\u628a\u5747\u503c\u548c\u6807\u51c6\u5dee\u6539\u540d\u5f53\u4f5c\u5206\u4f4d\u6570\u3002\n", "- \u56fe\u50cf\u6216\u6a21\u578b\u65e0\u6cd5\u8bfb\u53d6\uff1a\u68c0\u67e5\u6570\u636e\u662f\u5426\u5b8c\u6574\u4e0b\u8f7d\uff0c\u8bbf\u95ee\u8bb8\u53ef\u662f\u5426\u5df2\u63a5\u53d7\uff0c\u4ee5\u53ca\u72ec\u7acb\u5185\u6838\u662f\u5426\u5df2\u9009\u4e2d\u3002\n", "- \u8bad\u7ec3\u663e\u5b58\u4e0d\u8db3\uff1a\u5148\u4fdd\u6301\u6279\u91cf\u5927\u5c0f 1 \u4e0e\u51bb\u7ed3\u8bbe\u7f6e\uff0c\u5728\u5927\u663e\u5b58\u5de5\u4f5c\u7ad9\u8fd0\u884c\u4e94\u6b65\u68c0\u67e5\u3002\u7f29\u77ed\u52a8\u4f5c\u5757\u4f1a\u6539\u53d8\u65f6\u57df\u8986\u76d6\uff0c\u5e94\u540c\u6b65\u68c0\u67e5\u52a8\u4f5c\u5757\u914d\u7f6e\u548c\u8bc4\u6d4b\u6267\u884c\u6b65\u6570\u3002\n", "- \u4f4e\u79bb\u7ebf\u8bef\u5dee\u4f46\u4eff\u771f\u5931\u8d25\uff1a\u5148\u56de\u653e\u7559\u51fa\u793a\u6559\uff0c\u68c0\u67e5\u72b6\u6001\u5b9a\u4e49\u3001\u76f8\u673a\u987a\u5e8f\u3001\u7edd\u5bf9\u5173\u8282\u52a8\u4f5c\u3001\u521d\u59cb\u4f4d\u7f6e\u548c\u5939\u722a\u65b9\u5411\uff1b\u518d\u589e\u52a0\u793a\u6559\u8986\u76d6\u4e0e\u8bad\u7ec3\u3002\u4e0d\u8981\u6839\u636e\u635f\u5931\u4e0b\u964d\u5ba3\u79f0\u6293\u653e\u590d\u73b0\u6210\u529f\u3002\n", "\n", "\u672c\u6d41\u7a0b\u7684\u652f\u6301\u4f9d\u636e\uff1a\u5b98\u65b9 [\u03c00.5 \u4f7f\u7528\u8bf4\u660e](https://huggingface.co/docs/lerobot/main/en/pi05)\u3001[0.6.1 \u56fa\u5b9a\u7248\u4f9d\u8d56](https://github.com/huggingface/lerobot/blob/v0.6.1/pyproject.toml)\u3001[\u03c00.5 \u914d\u7f6e](https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/policies/pi05/configuration_pi05.py) \u548c [\u5b98\u65b9\u8bad\u7ec3\u5165\u53e3](https://github.com/huggingface/lerobot/blob/v0.6.1/src/lerobot/scripts/lerobot_train.py)\u3002\u6d41\u7a0b\u56fa\u5b9a\u4f7f\u7528 0.6.1\uff0c\u6700\u65b0\u6587\u6863\u7684\u5176\u4ed6\u7248\u672c\u547d\u4ee4\u4e0d\u4e00\u5b9a\u517c\u5bb9\u3002\n", "\n", "\u8fd0\u884c\u672c\u7b14\u8bb0\u672c\u4e0d\u4f1a\u81ea\u52a8\u4e0a\u4f20\u6570\u636e\u3001\u6a21\u578b\u6216\u5b9e\u9a8c\u7ed3\u679c\u3002\u6b63\u5f0f\u590d\u73b0\u8bb0\u5f55\u5e94\u6ce8\u660e\u793a\u6559\u56de\u5408\u6570\u3001\u8bad\u7ec3/\u7559\u51fa\u5212\u5206\u3001\u8bad\u7ec3\u8bbe\u7f6e\u3001\u68c0\u67e5\u70b9\u548c\u5b9e\u9645\u6210\u529f\u7387\u3002\n" ], "id": "pi05-omy-20" } ], "metadata": { "kernelspec": { "display_name": "OMY pi05 (Python 3.12)", "language": "python", "name": "omy-pi05" }, "language_info": { "name": "python", "version": "3.12" } }, "nbformat": 4, "nbformat_minor": 5 }