logo Mr.Robota 具身智能学习手册 中文
Chapter 12: 12 Interview Questions

1. Architecture Logic and High-Precision Control

  1. Separation of Perception and Execution: In your solution, why choose to decouple the vision-language model from the diffusion policy instead of adopting a unified end-to-end architecture? If the large model is directly used to output action space coordinates, where are the main performance bottlenecks in high-precision micro-manipulation scenarios?
  2. Real-time Closed Loop Correction: Diffusion models typically output in the form of Action Chunk. However, when dealing with real-time dynamic environments or non-linearly deformed objects, if the action sequence has not been executed yet but the target state has shifted, how does the system achieve efficient closed-loop real-time correction?
  3. Multimodal Distribution Planning: Please provide an example of how the diffusion model handles multimodal distribution path planning in tasks. When facing two paths that are both logically valid, how can the model avoid “wobble” at the decision boundary or output invalid mean actions?

II. Long Sequence Modeling (Transformer vs. Mamba)

  1. Reliability of long-term memory: When performing tasks that take a long time, the KV CacheKV\ Cache in Transformer increases quadratically with the sequence length. While Mamba reduces computational overhead through linear compression, it is still a lossy compression method. How can we ensure that the model accurately remembers previous logical nodes after running for a long time without generating “hallucinations” due to state compression?
  2. Comparison of underlying structures: Please explain in detail the core differences between the Transformer architecture and the Mamba architecture from the perspectives of information bottleneck and computational flow.
  3. Derivation of computational complexity: Why is the complexity of the attention mechanism in Transformer O(N2)O(N^2), while Mamba (State Space Model SSM) can achieve a linear complexity of O(N)O(N)?
  4. Trade-off between accuracy and performance: In your quantitative experiments, does Mamba suffer any loss in execution accuracy compared to Transformer? If so, how is this achieved through algorithmic optimization?

III. High-level Planning and Model Training

  1. Hierarchical Planning Architecture: What is the specific structure of the High-level Planner you mentioned? How does it transmit commands to the underlying controller?
  2. Context Understanding Limitations: If the High-level Planner receives only 3–5 frames of historical images as input, what mechanism enables it to maintain a coherent understanding of the entire long-term task state?
  3. Training Process: Please describe the training pipeline of the High-level Planner, including data augmentation methods, loss function design, and how to address positive-negative sample imbalance.
  4. Selective Scan Mechanism: In Selective Scan, how does the model learn “which information should be forgotten and which should be included in the state vector”? What fault-tolerant mechanisms exist at the system level if a critical transient feature (such as a very small displacement) is missed?

IV. Robustness and Sim-to-Real

  1. Visual occlusion handling: When the manipulation area experiences severe occlusion that causes visual feedback loss, does the system maintain a “blind operation” capability based on historical states, or does it directly trigger a safety pause?
  2. Physical parameter alignment: The dynamics parameters in the simulation environment differ significantly from those in the real physical world (such as elastic modulus and friction force of objects). How do you use technical methods to reduce the gap between Sim-to-Real?
  3. Cognitive inertia in OOD scenarios: The state update in Mamba is recursive. If a sudden abnormal condition not seen in the training set occurs after long-term operation, will the hidden state exhibit inertia due to “too much historical memory,” forcing the current anomaly to be fitted into past normal trajectories?
  4. Real-machine validation evaluation: What is the success rate of the algorithm in real scenarios? Is the current bottleneck in perception accuracy or control frequency?

5. Mathematical Foundations and Algorithm Implementation

  1. Probability Geometry: What is the probability that a unit-length wooden stick, when randomly broken into three segments, will form a triangle? Provide the derivation logic.
  2. Graph Theory and Connectivity: How can we efficiently count the number of “independent connected regions” in a given 2D grid (0/1 matrix)?
  3. Mathematical Equations: Write down the discrete expression or state update equation for the core algorithm in your model.