Deep Analysis of LIBERO Dataset: Architecture, Statistics, and Engineering Implementation

1. Introduction

In the field of robot learning, LIBERO (Lifelong Learning Benchmark for Robot Manipulation) is a milestone dataset. It is designed specifically for studying lifelong learning and knowledge transfer. Unlike traditional single-task datasets, LIBERO aims to evaluate how robot policies overcome catastrophic forgetting and utilize prior knowledge to accelerate learning when facing new task sequences.

This chapter based on actual dataset exploration (EDA) conducts detailed quantitative and qualitative analysis on the four core subsets (Suits) of the LIBERO dataset, and explores the differences in data formats under the two different training frameworks: LeRobot and OpenVLA.

Figure 1: Introduction to LIBERO data

2. Dataset Architecture & Taxonomy

The LIBERO dataset is not a random collection of tasks, but is carefully divided into four task suites with specific semantic objectives. Each suite contains 10 core tasks designed to test the generalization ability of the robot across different dimensions.

2.1 LIBERO-Object: Generalization at the Object Level

• Design objective: To test the robustness of the robot to the visual features (color, shape, texture) of different objects.
• Core logic: The action sequence is highly similar (“pick up X and place in basket”), but the manipulation object $X$ changes.
• Typical tasks:
  • Pick up Alphabet Soup can
  • Pick up Cream Cheese
  • Pick up Ketchup
  • Note: All actions are Pick-and-Place, and the variables refer only to the objects themselves.

2.2 LIBERO-Spatial: Robustness of Spatial Layout

• Design objective: To test the robot’s spatial perception and planning capabilities when the relative position of objects changes.
• Core logic: The object to be manipulated is fixed (usually a black bowl), but its initial position and surrounding environment (positions of obstacles) change.
• Typical tasks:
  • Pick up the black bowl from between the plate and the mold.
  • Pick up the black bowl from the center of the table.
  • Pick up the black bowl from the top drawer of the wooden cabinet.
  • Note: This requires the Agent to have strong spatial semantic understanding capabilities, rather than simple coordinate memory.

2.3 LIBERO-Goal: Diversity of target states

• Design objective: To test the robot’s ability to achieve a specific state change, involving complex interactions.
• Core logic: The task goal is no longer just grasping, but includes various physical interactions such as opening, pushing, pulling, and placing.
• Typical tasks:
  • Open the middle drawer of the cabinet.
  • Push a plate in front of the stove.
  • Place a wine bottle on the wine rack.
  • Stack: Put bowls on plates.

2.4 LIBERO-10 (Long-Horizon): Long-term temporal reasoning

• Design objective: To test the robot’s ability to execute long sequences and multi-step combined tasks. This is the most challenging subset.
• Core logic: Usually involves two or more consecutive actions, or actions triggered by conditions.
• Typical tasks:
  • Put both the letter soup can and the cream cheese box into the basket (multi-object sequence).
  • Put the black bowl into the bottom drawer of the cabinet and close it (complex interaction sequence).
  • Turn on the stove and place the mocha maker on top (causal sequence).

3. Quantitative Analysis Statistics (Quantitative Analysis)

Through the traversal and statistics of the Parquet file, we have obtained the following precise data distribution. This is crucial for setting the Batch Size and planning the Epoch during training.

3.1 Overall Scale

• Total Episodes: 1,693
• Total Sample Frames: 273,465 frames
• Average Sampling Rate: Approximately 161.5 frames/Episode

3.2 Detailed Statistics Table of Subset

3.3 Data Distribution Insights

1. Sample balance: Except for libero_10, which has a significantly larger sample size due to task complexity, the sample sizes of the other three datasets remain at the 50,000–60,000 level. The number of Episodes for each sub-task is basically between 35 and 50. This balance helps avoid overfitting during model training.
2. Temporal features: The average step size of 267.7 in libero_10 indicates that training this subset requires the model to have a longer Context Window or better memory management capabilities.

4. Data Engineering: Alignment of LeRobot and OpenVLA Formats

In practical engineering implementation, the heterogeneity of data formats is a major obstacle to training. The LIBERO dataset currently has two main formats: the LeRobot (Parquet) format for the HuggingFace ecosystem, and the OpenVLA (RLDS/TFRecord) format for the TensorFlow/Google ecosystem.

4.1 Format Comparison Matrix

4.2 Challenges and Solutions for Data Alignment

According to the analysis script, both are in a “incompatible” state. To use data in LeRobot format for fine-tuning the OpenVLA model, an ETL (Extract-Transform-Load) process must be performed:

1. Extract (Extract): Use Pandas or PyArrow to read the .parquet file, and parse the flattened observation.image and action columns.
2. Transform (Transform):
   • Structural reorganization: Reorganize flat data into a hierarchical structure of (episode, step).
   • Type conversion: Convert image byte streams or Numpy arrays into TF Feature format.
   • Metadata injection: Add fields such as is_first, is_last, and reward required by RLDS.
3. Load (Load): Serialize tf.io.TFRecordWriter into a .tfrecord file, and generate the corresponding RLDS signature file.

5. Conclusion

The LIBERO dataset clearly decouples the challenges in the four dimensions of robot manipulation: visual (Object), spatial (Spatial), interaction (Goal), and timing (10/Long-Horizon), through structured task design.

• For algorithm researchers: LIBERO provides an excellent benchmark for evaluating the weaknesses of a model in specific capabilities (for example, if a model performs well on Object tasks but not on Spatial tasks, it indicates a defect in its spatial attention mechanism).
• For engineering developers: Understanding the differences between LeRobot and OpenVLA formats is essential for building efficient data pipelines. The significantly increased sequence length in libero_10 also reminds us to pay attention to inference latency and memory usage during model deployment.

Attachment: Statistical Analysis Report and Analysis Code for Libero Dataset

python#!/usr/bin/env python3
"""
分析Libero数据集，统计每个大任务下的小任务数量和样本数量
"""

import pandas as pd
import os
from pathlib import Path
import json
from collections import defaultdict

def analyze_libero_dataset():
    """分析Libero数据集"""

    # 数据集路径
    datasets = [
        "libero_10_no_noops_lerobot_v21",
        "libero_goal_no_noops_lerobot_v21",
        "libero_object_no_noops_lerobot_v21",
        "libero_spatial_no_noops_lerobot_v21"
    ]

    base_path = Path("/data1/DATA")

    results = {}

    for dataset_name in datasets:
        print(f"\n分析数据集: {dataset_name}")
        print("=" * 50)

        dataset_path = base_path / dataset_name
        data_path = dataset_path / "data" / "chunk-000"

        if not data_path.exists():
            print(f"数据路径不存在: {data_path}")
            continue

        # 获取所有parquet文件
        parquet_files = list(data_path.glob("episode_*.parquet"))
        print(f"总episode数量: {len(parquet_files)}")

        # 分析每个episode
        task_stats = defaultdict(lambda: {"episodes": 0, "total_samples": 0})
        episode_info = []

        for parquet_file in parquet_files:
            try:
                # 读取parquet文件
                df = pd.read_parquet(parquet_file)

                # 获取任务信息
                task_name = "unknown"
                if 'task_index' in df.columns and len(df) > 0:
                    task_index = df['task_index'].iloc[0]
                    task_name = f"task_{task_index}"
                elif 'task' in df.columns:
                    task_name = df['task'].iloc[0] if len(df) > 0 else "unknown"
                elif 'task_name' in df.columns:
                    task_name = df['task_name'].iloc[0] if len(df) > 0 else "unknown"
                else:
                    # 打印前几列看看数据结构
                    if len(episode_info) < 3:  # 只打印前3个文件的信息
                        print(f"文件 {parquet_file.name} 的列名: {list(df.columns)}")
                        if len(df) > 0:
                            print(f"样本数: {len(df)}")
                            print(f"前几行数据:")
                            print(df.head(2))
                            print("-" * 30)

                # 统计信息
                sample_count = len(df)
                task_stats[task_name]["episodes"] += 1
                task_stats[task_name]["total_samples"] += sample_count

                episode_info.append({
                    "file": parquet_file.name,
                    "task": task_name,
                    "samples": sample_count
                })

            except Exception as e:
                print(f"处理文件 {parquet_file.name} 时出错: {e}")

        # 输出统计结果
        print(f"\n{dataset_name} 统计结果:")
        print(f"{'任务名称':<15} {'Episode数量':<12} {'总样本数':<12} {'平均样本数':<12}")
        print("-" * 60)

        total_episodes = 0
        total_samples = 0

        # 按任务索引排序
        sorted_tasks = sorted(task_stats.items(), key=lambda x: x[0])

        for task_name, stats in sorted_tasks:
            avg_samples = stats["total_samples"] / stats["episodes"] if stats["episodes"] > 0 else 0
            print(f"{task_name:<15} {stats['episodes']:<12} {stats['total_samples']:<12} {avg_samples:<12.1f}")
            total_episodes += stats["episodes"]
            total_samples += stats["total_samples"]

        print("-" * 60)
        print(f"{'总计':<15} {total_episodes:<12} {total_samples:<12} {total_samples/total_episodes if total_episodes > 0 else 0:<12.1f}")

        # 保存结果
        results[dataset_name] = {
            "task_stats": dict(task_stats),
            "total_episodes": total_episodes,
            "total_samples": total_samples,
            "episode_info": episode_info
        }

    return results

if __name__ == "__main__":
    results = analyze_libero_dataset()

    # 保存详细结果到JSON文件
    with open("libero_analysis_results.json", "w", encoding="utf-8") as f:
        json.dump(results, f, ensure_ascii=False, indent=2)

    print("\n" + "=" * 80)
    print("总体统计:")
    print("=" * 80)

    for dataset_name, result in results.items():
        print(f"\n{dataset_name}:")
        print(f"  - 任务类型数量: {len(result['task_stats'])}")
        print(f"  - 总Episode数量: {result['total_episodes']}")
        print(f"  - 总样本数量: {result['total_samples']}")
        print(f"  - 平均每Episode样本数: {result['total_samples']/result['total_episodes'] if result['total_episodes'] > 0 else 0:.1f}")

        # 显示每个任务的详细统计
        print(f"  - 任务分布:")
        sorted_tasks = sorted(result['task_stats'].items(), key=lambda x: x[0])
        for task_name, stats in sorted_tasks:
            print(f"    * {task_name}: {stats['episodes']} episodes, {stats['total_samples']} samples")

The above is data analysis in Lerobot format.

The following is in openvla format.

python#!/usr/bin/env python3
"""
分析Libero数据集，统计每个大任务下的小任务数量和样本数量
"""

import os
import tensorflow_datasets as tfds
import tensorflow as tf

def analyze_dataset(dataset_path):
    """
    Analyzes a single dataset to extract unique language instructions using tfds.
    """
    try:
        # Suppress verbose logging
        tf.get_logger().setLevel('ERROR')

        builder = tfds.builder_from_directory(builder_dir=dataset_path)
        dataset = builder.as_dataset(split='train')

        unique_instructions = set()

        for episode in dataset:
            # The language instruction is the same for all steps in an episode
            # We need to iterate through the steps dataset to get the first step
            for step in episode['steps'].take(1):
                instruction = step['language_instruction'].numpy().decode('utf-8')
                unique_instructions.add(instruction)

        return sorted(list(unique_instructions))

    except Exception as e:
        print(f"Error analyzing {dataset_path}: {e}")
        return []

def main():
    base_dir = "/data/DATA/modified_libero_rlds"
    task_categories = [d for d in os.listdir(base_dir) if os.path.isdir(os.path.join(base_dir, d)) and not d.startswith('.')]

    for category in task_categories:
        print(f"--- Task Category: {category} ---")

        version_dir = os.path.join(base_dir, category, "1.0.0") # Assuming version 1.0.0

        if not os.path.exists(version_dir):
            print(f"Version directory not found for {category}")
            continue

        subtasks = analyze_dataset(version_dir)

        print(f"  Found {len(subtasks)} subtasks:")
        for i, task in enumerate(subtasks):
            print(f"    {i+1}. {task}")
        print("\n")

if __name__ == "__main__":
    main()

Dataset Overview

Four Libero datasets were analyzed, and each dataset contains 10 different task types (task_0 to task_9).

Detailed Statistics Results

1. libero_10_no_noops_lerobot_v21

• Total number of episodes: 379
• Total sample count: 101,469
• Average number of samples per episode: 267.7
• Number of task types: 10 tasks

Detailed statistics for each task:

2. libero_goal_no_noops_lerobot_v21

• Total number of episodes: 428
• Total sample count: 52,042
• Average number of samples per episode: 121.6
• Number of task types: 10 tasks

Detailed statistics for each task:

3. libero_object_no_noops_lerobot_v21

• Total number of episodes: 454
• Total sample count: 66,984
• Average number of samples per episode: 147.5
• Number of task types: 10 tasks

Detailed statistics for each task:

4. libero_spatial_no_noops_lerobot_v21

• Total episode count: 432
• Total sample count: 52,970
• Average sample per episode: 122.6
• Number of task types: 10 tasks

Detailed statistics for each task:

Overall Summary

Key Findings

1. Uniform dataset structure: All 4 datasets contain 10 task types (task_0 to task_9)
2. Sample quantity differences: The libero_10 dataset has the most samples (101,469), followed by libero_object (66,984)
3. Episode length differences: The average Episode length of libero_10 is the longest (267.7), while libero_goal and libero_spatial have relatively shorter Episodes (about 122)
4. Task distribution: The task distribution within each dataset is relatively uniform, with approximately 30–50 episodes per task
5. Data scale: There are a total of 1,693 episodes and 273,465 sample points

Data Characteristics

• Data format: All data are stored in parquet format, with one file per episode.
• Data structure: Each sample contains fields such as observation.state, action, and timestamp.
• Task identification: Different task types are identified by the task_index field.
• Timeliness: Each episode represents a complete task execution sequence.

Discussion on Data Alignment of lerobot and openvla

# OpenVLA vs. LeRobot 数据格式对比分析

## 1. 核心差异总结

| 对比项 | OpenVLA (modified_libero_rlds) | LeRobot (lerobot_v21) | 对齐状态 |
| :--- | :--- | :--- | :--- |
| **存储格式** | `TFRecord` | `Parquet` | 🔴 **不兼容** |
| **特征元数据** | `features.json` (结构不同) | `info.json` | 🟡 **需映射** |
| **特征结构** | 嵌套在 `steps` 和 `episode_metadata` | 扁平化结构 | 🟡 **需转换** |

**结论**: OpenVLA 和 LeRobot 的数据格式**没有直接对齐**。要使其兼容，需要进行数据转换和特征映射。

---

## 2. 详细对比

### 存储格式

-   **OpenVLA**: 使用 `TFRecord` 格式 (`.tfrecord`)。这是一种序列化的二进制格式，在 TensorFlow 生态中广泛使用，适合高效的 `I/O` 操作。
-   **LeRobot**: 使用 `Apache Parquet` 格式 (`.parquet`)。这是一种列式存储格式，在 `Pandas` 和 `Spark` 等大数据处理框架中很流行，提供了高效的压缩和编码。

👉 **对齐分析**: 两种格式在二进制层面完全不同，无法直接通用。

### 元数据和特征定义

#### OpenVLA

-   **`dataset_info.json`**: 包含数据集的版本、描述、分片信息等高级元数据。
-   **`features.json`**: 定义了数据的具体结构。从分析脚本的初步运行结果看，其根级别没有`features`键，这表明特征定义可能嵌套在更深层次，或者需要结合 `TensorFlow Datasets (TFDS)` 的加载器来完整解析。通常，RLDS/OpenVLA 格式会将数据组织成 `steps`，每个 `step` 包含 `observation`, `action`, `reward`, `is_first`, `is_last` 等。

#### LeRobot

-   **`info.json`**: 集中包含了数据集的元数据和详细的特征定义。
-   **`features` 键**: 在 `info.json` 中直接定义了所有数据字段，包括：
    -   `observation.images.wrist_image` (视频)
    -   `observation.images.image` (视频)
    -   `observation.state` (机器人状态)
    -   `action` (动作)
    -   `timestamp`, `frame_index`, `episode_index` (元数据)
    -   `task_index` (任务ID)

👉 **对齐分析**:
-   `LeRobot` 的特征是**扁平化**的，每个字段（如 `action`, `observation.state`）都作为 `Parquet` 文件中的一列。
-   `OpenVLA` 的特征是**层级化**的，通常封装在 `steps` 序列中。

---

## 3. 如何对齐？

要将 `LeRobot` 的数据格式对齐到 `OpenVLA`，需要执行以下步骤：

1.  **读取 Parquet 数据**: 使用 `pandas` 或 `pyarrow` 逐个读取 `LeRobot` 的 `episode_*.parquet` 文件。
2.  **重构数据**:
    -   将每个 `episode` 的数据帧（`DataFrame` 的行）转换为 `OpenVLA` 的 `step` 结构。
    -   每个 `step` 应该包含 `observation`（包含图像和状态）、`action` 等字段。
    -   在 `episode` 级别添加 `episode_metadata`，如 `task_id`。
3.  **序列化为 TFRecord**:
    -   将重构后的 `episode` 序列化为 `tf.train.Example` 协议缓冲区。
    -   将序列化后的数据写入 `.tfrecord` 文件。
4.  **创建元数据文件**:
    -   生成符合 `OpenVLA` 规范的 `dataset_info.json` 和 `features.json`。

这个过程本质上是一个 `ETL` (Extract, Transform, Load) 流程，需要编写专门的转换脚本。

如果需要，我可以帮编写一个初步的转换脚本。

Detailed task analysis

This openvla-format libero dataset is divided into 4 major category tasks, with exactly 10 different sub-tasks under each category.

Detailed Analysis Results

1. libero_object_no_noops

This category mainly focuses on picking up specific items and placing them into a basket.

• Sub-task list:
  1. Pick up the letter soup can and put it in the basket.
  2. Pick up the barbecue sauce and put it in the basket.
  3. Pick up the butter and put it in the basket.
  4. Pick up the chocolate pudding and put it in the basket.
  5. Pick up the cream cheese and put it in the basket.
  6. Pick up the ketchup and put it in the basket.
  7. Pick up the milk and put it in the basket.
  8. Pick up the orange juice and put it in the basket.
  9. Pick up the salad dressing and put it in the basket.
  10. Pick up the ketchup and put it in the basket.

2. libero_goal_no_noops

This category mainly focuses on achieving specific target states, such as opening and closing drawers, placing items in specific locations, etc.

• Sub-task list:
  1. Open the middle drawer of the cabinet.
  2. Open the top drawer and put the bowl inside.
  3. Push the plate in front of the stove.
  4. Place the bowl on the plate.
  5. Put the bowl on the stove.
  6. Place the bowl on the top of the cabinet.
  7. Add cream cheese into the bowl.
  8. Place the wine bottle on the wine rack.
  9. Place the wine bottle on the top of the cabinet.
  10. Turn on the stove.

3. libero_10_no_noops

This category contains a series of manipulation tasks that require manipulating multiple items simultaneously.

• Sub-task list:
  1. Pick up the book and put it into the caddy’s rear compartment.
  2. Place both mocha cups on the stove.
  3. Put the letter soup cans and the cream cheese box into the basket.
  4. Put the letter soup cans and ketchup into the basket.
  5. Put the cream cheese box and butter into the basket.
  6. Put the black bowl in the bottom drawer of the cabinet and close it.
  7. Place the white cup on the left plate, and the yellowish-white cup on the right plate.
  8. Put the white cup on the plate, and the chocolate pudding on the right side of the plate.
  9. Put the yellowish-white cup in the microwave and turn off the door.
  10. Turn on the stove and place the mocha cups on top.

4. libero_spatial_no_noops

This category focuses primarily on spatial relationships, that is, performing manipulation based on the relative positions between items.

• Sub-task list:
  1. Pick up the black bowl from between the plate and the mold, then place it on the plate.
  2. Pick up the black bowl from the center of the table, then place it on the plate.
  3. Pick up the black bowl from the top drawer of the wooden cabinet, then place it on the plate.
  4. Pick up the black bowl from next to the cookie box, then place it on the plate.
  5. Pick up the black bowl from next to the plate, then place it on the plate.
  6. Pick up the black bowl from next to the mold, then place it on the plate.
  7. Pick up the black bowl from the cookie box, then place it on the plate.
  8. Pick up the black bowl from the mold, then place it on the plate.
  9. Pick up the black bowl from the stove, then place it on the plate.
  10. Pick up the black bowl from the wooden cabinet, then place it on the plate.