DM0.5: High-Performance Embodied VLA and OpenDM Open Source Introduction

Purpose of this article: To understand the public world model structure, inference path, and high-performance optimization announcements of DM0.5, and to provide the installation, inference, fine-tuning, and evaluation interfaces that are currently available in OpenDM.

Important note: DM0.5 is a VLA, not a world model. This article is included within the existing VLA section; its inference optimization will be discussed separately, but the term “low-latency VLA inference” will not be mistakenly referred to as a world model rollout.

1. First, get the conclusion: what exactly is this update?

DM0.5 is a embodied vision-language-action model from Dexmal, designed for open-world applications. The officially published OpenDM repository includes model weights, training scripts, inference services, data registration examples, and LIBERO/RoboTwin2.0 evaluation guides; among the currently available models, Dexmal/DM05, Dexmal/DM05-libero, Dexmal/DM05-robotwin2, and Dexmal/DM05-SO101-Pick-Cube are accessible as model entry points.

The focus of this update is not on “another larger VLM,” but on compressing the online execution links step by step:

text图像/状态/指令
    -> Vision TensorRT
    -> Gemma3 VLM prefix + KV cache
    -> Action Expert suffix / Flow Matching
    -> action chunk
    -> 机器人控制接口

The core data provided in the official website performance announcement are as follows: on the RTX 4090D, the model core latency decreased from 534.04 ms to 57.49 ms, with a total acceleration of 9.29×; the API latency is p50 67.75 ms and p90 70.01 ms. The announcement also states that under the same checkpoint and LIBERO input conditions, the success rate for 2000 episodes increased from 98.45% to 98.40%. These are the numbers disclosed in the official announcement, not the results measured by this tutorial on the local machine.

The real area that needs to be learned is: how to combine VLA’s fixed shape calculations, low-precision matrix multiplication, fusion operators, and GPU graph execution into a low-latency online service while keeping the model behavior largely unchanged.

2. Official resources and current open-source status

2.1 Accurate Answer to “Is there code on the official website?”

Yes, but it needs to be understood in two layers:

1. DM0.5 model and training/inferencing code have been made public. OpenDM can be used to download checkpoints, start the inference server, register custom datasets, perform SFT/LoRA, and evaluate LIBERO and RoboTwin2.0 according to the official guidelines.
2. The 9.29× dedicated optimization stack mentioned in the announcement is not fully present in the current OpenDM. We reviewed the currently public source code: model-level components such as FlexAttention, fused attention, and fixed action chunks can be found, but there is no DM0.5 deployment implementation that fully integrates Vision TensorRT + FP8 E4M3 Prefix MLP + Triton Suffix fused kernels + startup CUDA Graph from the announcement, nor are the scripts corresponding to the performance benchmarks 57.49, 534.04, or 9.29 available.

Therefore, this tutorial will not pretend that the DM0 Triton code of realtime-vla is equivalent to the DM0.5 9.29× code. realtime-vla is very useful for reference; it reveals the DM0 Triton kernel, weight conversion, and the CUDA Graph inference entry point. However, it only proves that “there are public references for related low-latency technical approaches,” but it does not prove that “all the optimizations in the DM0.5 announcement have been made open-sourced.”

3. Observe the diagram: DM0.5’s product signals and parent route

Figure 1: Header image of the DM0.5 official project page. The image uses actions such as wiping, grasping, winding, pushing, rolling, and extracting to convey DM0.5’s product positioning as a open-world-oriented, multi-task-capable, and generalizable product. It is the visual for the promotion/project page, not a network structure diagram.

Source: OpenDM official repository docs/image/header-zh.png.

Figure 2: The parent architecture diagram of DM0 in the Dexbotic warehouse. This figure presents the public parent reference for the DM0 pathway: the visual encoder converts observations into visual tokens, which then enter the LLM along with the robot state and commands; ActionExpert is responsible for converting multimodal context into subtasks, goal boxes, end effector trajectories, and discrete/continuous actions. It helps us understand the “VLM + Action Expert” pathway of Dexmal, but it cannot be regarded as the official exact diagram of DM0.5.

Source: Dexbotic official repository resources/dm0_arch.png.

4. Is DM0.5 actually pi0.5?

No. There are two names that can be easily confused:

• pi0.5 is another VLA model path of Physical Intelligence.
• DM0.5 is a DM series model from Dexmal. The OpenDM open-source code indicates that it is built based on Gemma3 VLM and custom Action Expert.

Therefore, just because both names contain 0.5, they cannot be considered the same base; nor can the Pi0/Pi05 codes in realtime-vla be directly applied to DM0.5. For DM0.5, one should first read DM05Config, DM05ActionExpert, DM05Model.forward(), and inference_action() from OpenDM.

5. Architecture breakdown of the public source code

The following diagram is a teaching diagram compiled based on the currently public source code of OpenDM. It is not an official paper diagram; its purpose is to present the prefix, suffix, KV cache, and Flow Matching process in the code all in one picture.

Figure 3: The public architecture chain organized according to OpenDM dm05_arch.py. The key is the reuse relationship between prefix conditional encoding and suffix action generation: the prefix first encodes images, instructions, and states into conditional context and caches it, while during each time step of action diffusion, the action suffix is mainly executed.

5.1 Prefix: Transform observations into reusable conditional contexts

The input for DM0.5 is not a single image, but a robot decision sample:

text多视角图像 + 任务文本 + proprioception/state + robot type

Visual input first passes through the visual encoder and then enters the multimodal processing path of Gemma3 VLM. Text instructions and robot states are organized according to the processor, dataset transform, and prompt conventions of OpenDM, ultimately forming a prefix token sequence.

The prefix output has two uses:

1. Generate the multi-modal hidden states for current observations, which serve as conditional information for the action expert.
2. Generate and store the KV cache of the VLM, so that subsequent action generation steps do not need to re-process the same image and text through the complete VLM repeatedly.

This step is the key to understanding low-latency optimization. Suppose an action chunk requires 10 Flow Matching steps, and if visual encoding and language prefix are performed repeatedly for each step, the computational cost will be very high. If the prefix is performed only once and the action suffix is processed later, the action sampling cycle can be reduced to a few dozen milliseconds.

5.2 Suffix: Action Expert is not a simple linear action head

DM05ActionExpert in the OpenDM source code inherits the text model structure of Gemma3, but it is responsible for handling the action suffix. It does not simply pass the last hidden state through a MLP layer; instead, the action token/action vector interacts with the prefix condition in a transformer action expert.

Modules that can be confirmed by public code include:

• action_in_proj: Project the continuous action vector onto the language model hidden dimension.
• time_mlp_in and time_mlp_out: Encode the Flow Matching time step as a modulation signal.
• Time modulator for each layer: Let the action expert know which time position on the noisy trajectory currently exists.
• Prefix KV cache: Allow the suffix to access the contextual conditions formed by images, text, and state.
• action_out_proj: Project the hidden states of the action expert back into the action dimension to obtain a velocity field prediction.

Therefore, the action generation of DM0.5 can be understood as:

text连续动作 chunk x_t
    -> action_in_proj
    -> 时间调制的 Action Expert
    -> action_out_proj
    -> 速度场 v_theta(x_t, t | c)

Among them, c represents the image/language/status condition obtained from the prefix encoding.

5.3 Flow Matching: What is Training and Inference Doing Differently

Let the real action chunk be a, and the random noise be epsilon. The public implementation uses linear interpolation to construct the intermediate state:

textx_t = t * epsilon + (1 - t) * a
u_t = epsilon - a

The model receives x_t, time step t, and observation condition c, and predicts the velocity field v_theta(x_t, t | c). The training objective can be summarized as velocity field regression with an action mask:

textL = MSE(v_theta(x_t, t | c), u_t)

action mask here is very important: the training loss is calculated only for the effective action dimension and effective time slices, avoiding padding or invalid dimensions that could interfere with the action expert learning.

During inference, there is no actual action a. Thus, starting from a random x_1, the action suffix is called repeatedly to estimate the velocity, and then Euler updates integrate the noise gradually into the trajectory:

textx_{t-dt} = x_t + v_theta(x_t, t | c) * dt

The public implementation defaults to diffusion_steps=10, and ultimately returns an action chunk. In other words, the VLA outputs a sequence of actions during one inference, rather than individual joint movements; after the robot executes part of it, it reads new observations for re-inference. This is the common closed loop of receding-horizon / action chunking.

6. 9.29× acceleration optimized which aspects

The approach outlined in the announcement can be divided into four layers based on the online reasoning chain. Note that the “Technical Explanation” below corresponds to the description in the announcement; only those parts of the publicly available OpenDM source code that can be directly verified are marked as “Public Source Code Visible”.

6.1 Vision TensorRT: Why checkpoint-specific?

The TensorRT engine does not become faster automatically just by installing TensorRT. It often requires building or selecting the engine based on checkpoint, input size, number of images, and data type.

The four words emphasized in the announcement have actual meanings:

• checkpoint-specific: Weight, layer structure, quantization/accuracy policy, and engine binding; the engine of another model cannot be used arbitrarily.
• FP16 execution: Half-precision execution is used during visual encoding to reduce memory bandwidth and Tensor Core computing costs.
• static I/O: The input/output buffer shape is fixed, facilitating memory reuse, stable addresses, and CUDA Graph capture.
• fixed image-count contract: At the deployment level, the number of images per input must be fixed; if it varies between 2 and 3 images, both the static engine and CUDA Graph will fail or require multiple profiles.

Therefore, the inspiration for online robot systems is not “replace everything with TensorRT”, but rather: fix the observation protocol first, and then build an engine for that fixed protocol.

6.2 FlexAttention: What is optimized is not the model semantics, but the execution shape of attention

The prefix and suffix of VLA have different attention structures. The prefix needs to handle image tokens, text tokens, and state tokens; the suffix needs to access the prefix cache while processing the tokens/vectors of the current action chunk. If a general attention mechanism is used for each call, it may incur additional overhead for mask, shape, KV layout, and different stages.

FlexAttention allows implementers to describe attention access patterns that better fit the model structure, which are then compiled/generated by PyTorch into efficient execution paths. The current source code of OpenDM has retained the entry points for VLM LLM and action expert implementations of flex_attention/sdpa, but this does not mean that all the tuned kernels announced are already public.

6.3 FP8 E4M3: Why Only Training MLP May Also Yield Great Results

In Transformer, the MLP typically involves a large-scale gate/up/down projection matrix multiplication, which requires significant computational power and memory access. The announcement states that for the prefix MLP’s gate/up/down operations, FP8 E4M3 Tensor Core GEMM is used. The key benefit is:

1. Weights and activations consume less bandwidth.
2. The Tensor Core in GPUs such as 4090D offers higher throughput for low-precision matrix multiplication.
3. Concentrate FP8 on the matrix multiplications that are most sensitive to throughput, rather than quantizing all modules uniformly.

However, FP8 is not simply by writing dtype=torch.float8_e4m3fn and stopping. It is also necessary to handle scale, dynamic range, accumulation precision, weight layout, calibration data, and output error; without official calibration and kernel implementation, readers should not consider this part as a directly reproducible step in current OpenDM.

6.4 Triton Suffix fused kernels: The reduction is in scheduling and memory access costs

The action suffix is often executed repeatedly within each Flow Matching step. Even though each sub-operator takes a short time alone, after multiple kernel launches, temporary tensor writes, and Python/framework scheduling, the delay will significantly increase.

The typical goal of the Triton fused kernel is to combine a sequence of fixed manipulations into a GPU kernel, for example:

text读取输入 -> 归一化/逐元素调制 -> 投影或 GEMM -> 激活 -> 写回

The actual fusion boundary needs to be determined by considering the model dimension, register occupancy, occupancy, L2 hit rate, and numerical error, rather than just the number of kernels. The officially published realtime-vla/dm0_infer.py provides a Triton reference for DM0, which is suitable for learning how to break down the GEMM-like operations of the visual encoder, LLM, and action expert; it is not a substitute for the official optimization package for DM0.5.

6.5 CUDA Graph: Why Capture at Startup, Static Address, and One Replay

CUDA Graph is suitable for inference cycles with “fixed shape, fixed control flow, and fixed memory addresses”. During deployment, a dummy buffer of fixed shape is used to run once and capture the graph. In the online phase, only new observations are transferred into the allocated static buffer, and then replay is performed.

The online phase specified in the announcement can be compressed to:

text静态数据搬运 -> TensorRT engine 执行 -> 一次 graph replay

This will reduce the large runtime launch overhead, but it also brings constraints:

• The number of images, resolution, prompt/token length, and action chunk shape need to be fixed or bucketed.
• The CUDA stream, buffer address, and execution path used during graph capture must remain stable.
• Diffusion steps cannot be changed temporarily online; changing the step count usually requires a new capture.
• Complex dynamic Python branches, dynamic memory allocation, or exception handling cannot be easily included in the graph.

The publicly available DM0 realtime documentation for Dexbotic also reflects the same type of engineering constraints: when the service starts, a fixed number of steps of the graph are captured, and the sampling steps at the call side must be consistent with the capabilities return values. This can serve as an understanding reference for CUDA Graph practices, but the 5.5× benchmark of DM0 cannot be directly compared with the 9.29× benchmark of DM0.5.

7. How to run OpenDM officially and publicly

This section provides the entry where “readers can continue with the manipulation on the official repository”, without wrapping it up as a complete training reproduction that has already been completed for the readers in this article.

7.1 Environmental Prerequisites

The current README for OpenDM recommends Linux, NVIDIA GPU, and Docker/NVIDIA Container Toolkit. Training typically requires multiple GPUs, while inference can be performed with a single GPU. For specific CUDA, PyTorch, memory, and image versions, refer to the current README in the repository.

It is recommended to confirm first:

bashnvidia-smi
docker --version
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Do not download the 11 GB checkpoint first, and then check the environment. First, verify that the GPU, drivers, container runtime, and Hugging Face network are available.

7.2 Getting Code and Models

bashgit clone https://github.com/dexmal/opendm.git
cd opendm

# 使用 Hugging Face CLI 下载公开权重，实际文件较大，请预留磁盘空间
hf download Dexmal/DM05 --local-dir ./checkpoints/DM05

The repository also provides Docker-related files and scripts. It is recommended to use the official container/startup scripts instead. Do not upgrade the versions of PyTorch, FlashAttention, xformers, or CUDA from the beginning; the versions used in VLA inference for attention, image processing, and custom kernels are sensitive to changes.

7.3 Starting the DM0.5 Inference Service

The service entry in the official README is similar to:

bashscript/dm05_launcher.sh \
  --task inference \
  --nproc_per_node 1 \
  --model-config.model-name-or-path ./checkpoints/DM05 \
  --model-config.chunk-size 50 \
  --inference-config.port 7891

Then use the curl demo in the repository:

bashbash tests/curl_demo.sh http://127.0.0.1:7891/process_frame

Requests typically require task text, robot status, and image fields. Whether fields such as robot_type, control_mode, and speed are required depends on the model and deployment mode. Do not directly copy the JSON from old versions: first check the current tests/curl_demo.sh, processor, and README to ensure that the number of images, key order, and state dimension match the checkpoint normalization stats.

7.4 Public Entry Points for SFT/LoRA

OpenDM currently provides public data registration examples and multiple fine-tuning guides. The general training entry is similar to:

bashscript/dm05_launcher.sh \
  --exp playground/dm05_sft_demo.py \
  --task train \
  --nproc_per_node 8 \
  --data-config.dataset-name my_robot \
  --model-config.model-name-or-path ./checkpoints/DM05 \
  --model-config.chunk-size 50 \
  --trainer-config.num-train-steps 50000

What needs to be completed before starting the training:

1. Register the dataset according to the example opendm/dataset/.
2. Verify the image fields, status, actions, and language instruction formats of the episode.
3. Ensure the automatically generated normalization stats match the checkpoint, robot body, and action dimensions.
4. First run the data loading and forward smoke test with a very small number of steps, then scale up the training.
5. Use the current configurations of docs/en/dm05_finetuning.md, docs/en/dm05_libero_lora_training.md, and docs/en/dm05_so101_lora_training.md, instead of the old commands mentioned in the network article.

The link can reproduction the public training/fine-tuning capabilities of DM0.5; it does not equal being able to reproduce the deployment kernel of 9.29× from the official website.

8. What can be reproduced, what cannot

9. How should the performance numbers be read

9.1 Core latency and API latency are not a single metric

The announcement also provides the core model latency and API latency, indicating that their measurement ranges are different:

• Model core latency: Closer to the actual model computation on the GPU.
• API latency: May also include overhead such as request parsing, data transfer, image preprocessing, tokenization, post-processing, and network/service scheduling.

Therefore, 57.49 ms and p50 67.75 ms are not contradictory. When writing an experimental report, it is necessary to specify the timing boundaries, number of warm-ups, batch size, number of images, prompt length, diffusion steps, chunk size, GPU model, and driver/CUDA/PyTorch version.

9.2 98.45% -> 98.40% indicates that the optimization has no significant impact on behavior, but this does not prove general performance.

The success rate of LIBERO 2000 episodes remains largely unchanged, providing valuable evidence that “reasoning optimization causes no significant impact on task performance.” However, it still only covers specific checkpoints, input protocols, and benchmarks. To evaluate the actual deployment performance of robots, additional tests are also needed:

• Sensor frame rate and latency jitter.
• Observational freshness during the execution of action chunks.
• p50/p90/p99, rather than just looking at the average value.
• Data transfer from CPU to GPU, image preprocessing, and action postprocessing.
• Frame loss, camera blocking, GPU temperature/frequency variation, and long-term operation stability.
• Different numbers of images, prompt length, and shape contract of the robot body.

10. Relationship with DM0 realtime and OpenDM

These two repositories can be read together: first, use OpenDM to understand “what DM0.5 needs to calculate”, and then use realtime-vla to understand “how VLA like DM0 breaks down the calculation into a Triton kernel for real-time inference”. However, when porting, it is necessary to reconfirm the number of model layers, hidden dimensions, visual resolution, action dimensions, weight layout, norm stats, and processor protocol.

11. Recommended Reading Order

If you do not train for now, it is recommended to read in the following order:

1. First, read OpenDM README to confirm the model, weights, and evaluation entry points.
2. Then read Figure 2 in this document to understand the VLM + Action Expert parent pipeline of Dexmal.
3. Open dm05_arch.py and focus on searching for DM05ActionExpert, action_in_proj, action_out_proj, forward, and inference_action.
4. Combining with Figure 3, manually trace the prefix KV cache, action noise, Flow Matching, and Euler steps.
5. Next, read the DM0 realtime documentation to understand the static shape and startup CUDA Graph.
6. Finally, read realtime-vla for dm0_infer.py and benchmark.py, and observe how a real Triton runtime organizes vision, LLM, action expert, and graph.

12. Common Pitfalls to Avoid When Reproducing

12.1 Mixing DM0.5 and pi0.5 Together

Similar names do not mean the same base. DM0.5 should be implemented using Gemma3 + Action Expert from OpenDM, and pi0.5 should be implemented using OpenPI/Physical Intelligence.

12.2 Focus only on the core latency of the model, not the closed loop latency

Whether a robot can operate in real time depends on the entire process from camera exposure to the issuance of control commands. It is essential to record the image acquisition, preprocessing, reasoning, inverse normalization, control cycle, network, and actuator latency separately.

12.3 Dynamic Input Destroys Static Optimization

Any change in the number of images, resolution, token length, chunk size, and diffusion steps will cause the TensorRT profile, kernel specialization, or CUDA Graph to fail. The deployment design should first determine the input contract, and then decide whether to perform static compilation and graph capture.

12.4 Normalization stats and action dimensions do not match

Action normalization is the foundation for the proper execution of VLA. For custom datasets, it is not sufficient to simply include image and action files; you must also verify the action dimension, robot state dimension, effective action mask, and the source of norm_stats.json.

12.5 Running New Repository with Commands from Old Tutorials

OpenDM is still being updated rapidly. Training scripts, model directories, weight names, and documentation will change. For reproduction, refer to the current repository README, docs/en/, and script help information, and record the commit hash.

13. Final Summary

DM0.5 This update is worth including in the VLA tutorial. The reason is not that “9.29× is a new VLA algorithm,” but because it explains in detail the most overlooked aspect when implementing embodied models: the model structure, input protocol, low-precision matrix multiplication, operator fusion, static memory, and CUDA Graph must be designed together.

For learners, it can be broken down into three levels:

1. Algorithm layer: Gemma3 VLM prefix + Action Expert suffix + Flow Matching action chunk.
2. Model layer: prefix KV cache, FlexAttention, time modulation, action projection, and Euler sampling.
3. Deployment layer: TensorRT, FP8, Triton fused kernels, fixed input shape, startup CUDA Graph, and end-to-end latency measurement.

Current OpenDM already supports reproduction of DM0.5 models for reading, weight downloading, inference services, SFT/LoRA, and public benchmark entries. The 9.29× dedicated optimization stack mentioned in the announcement should still be labeled as an official performance update. Reproduction will be carried out strictly after the corresponding deployment code and complete benchmark harness are made available. This boundary itself is an engineering fact that the tutorial needs to convey to readers.