Most teleoperation systems are unilateral: the operator sees video and robot state, and sends commands, but nothing physical comes back through the input device. Bilateral teleoperation closes a second loop — force measured at the robot (from a force/torque sensor, joint current, or a load cell) is rendered back through the input device, so the operator's hand feels what the robot's end-effector is pushing against. That second loop is what lets an operator "feel" an insertion align, a part seat, or a surface make contact, but it is also what turns a comfortably delay-tolerant control problem into a much harder one.
Position-position vs position-force architectures
The two return channels can be closed in different ways, and the choice shapes both feel and stability margin.
- Position-position (PP). Each side sends its own position to the other and renders the position error as a spring-like force. Simple to implement and inherently stable for a wide range of stiffnesses, but it renders contact indirectly — the operator feels a spring, not the robot's actual contact force — and free-space motion feels artificially heavy.
- Position-force (PF) / force-position. The leader sends position, the follower returns measured force directly. This is more transparent — the operator feels something closer to the true environment force — but it is more sensitive to noise and delay because measured force, unlike a computed position error, is not naturally bounded or smooth.
- Four-channel architectures. Both position and force are exchanged in both directions, tuned so the operator feels the environment's actual mechanical impedance rather than a proxy for it. Four-channel schemes achieve the best theoretical transparency but carry the most terms that must each stay passive.
None of these is free; every architecture trades some transparency for some robustness to delay and noise, and the right one depends on whether the task needs precise force perception (deformable handling) or just reliable contact detection (peg insertion, connector mating).
Why stability is the hard problem
A bilateral loop couples two mechanical systems — the operator's arm and the robot's end-effector — through a communication channel. Feed a mechanical system its own delayed reflection and it can behave like a lightly damped oscillator: small forces get amplified on each round trip until the system rings or diverges. This is not a tuning inconvenience; a PID gain that is perfectly stable on a LAN can make a teleoperated arm oscillate visibly once WAN delay is added, because the delay itself is what destabilizes the loop, independent of gain.
The standard analytical tool is passivity: a system is passive if it never supplies more energy at its port than it absorbs. Two widely used passivity-based techniques:
- Wave variables, introduced by Niemeyer and Slotine, transform the position/force signals exchanged across the channel into a pair of "wave" signals. Transmitting wave variables instead of raw power variables guarantees passivity of the communication channel under any constant time delay, which is why wave-variable teleoperation was one of the first schemes proven stable over long-haul links.
- Time-domain passivity control (TDPC), from Hannaford and Ryu, takes a more active approach: a passivity observer continuously measures net energy flow at each port, and a passivity controller injects damping only when the observer detects the port has gone active (producing more energy than it received). This adapts to real, time-varying delay and jitter rather than assuming a fixed worst case.
Both approaches trade some transparency for a stability guarantee — a theme that runs through the entire field.
The transparency-versus-stability trade-off
Transparency is how faithfully the operator perceives the remote environment's actual mechanical impedance — ideally, a rigid wall feels rigid and a soft gasket feels soft. Every mechanism that buys stability margin — added virtual damping, wave-variable transformation, passivity-controller energy dissipation — also blurs that perception, because it inserts artificial dynamics between the operator's hand and the true contact. Push for perfect transparency and small delays or model errors can destabilize the loop; push for guaranteed stability and the operator's felt sense of contact softens, sometimes to the point of "everything feels like it's underwater." Tuning a bilateral system is choosing where on that curve a given task needs to sit — connector mating cares more about clean force onset than perfect stiffness rendering, while surgical and deformable-material work cares about both.
Why delay hurts a force loop faster than a video loop
Video lag degrades teleoperation gracefully: an operator working past roughly 100–150ms of round-trip delay just gets slower, pausing after each motion to let the picture catch up, and throughput drops while the system stays controllable throughout. A force loop has no equivalent graceful mode. Because it is a closed loop coupling two physical systems, added delay reduces the maximum stable gain roughly in proportion to the delay itself — double the round-trip delay and you may need to roughly halve the stiffness or damping the operator can feel before the loop rings. Push past the stable region and the failure isn't sluggishness, it's oscillation or a controller safety trip. This is precisely why long-haul bilateral teleoperation is a much narrower engineering problem than long-haul video teleoperation, and why most production fleet-teleoperation setups run video-only over the wide area and reserve force feedback for local or edge-relayed links with tightly bounded delay.
What force feedback adds to a training dataset
For imitation learning, a force-reflecting session captures information a vision-and-position-only session cannot: the operator's own corrective response to contact. In a peg-in-hole insertion, an operator with force feedback feels misalignment as resistance and corrects the angle before it shows up as a large vision-visible error; without force feedback, they often only discover misalignment after the peg has already jammed, which produces jerkier, less consistent demonstrations. The same applies to handling deformables — cloth, cable, food — where the "right" grip force is defined by feel, not geometry, and cannot be inferred reliably from video alone.
Recording force-feedback demonstrations well means capturing the measured contact force and the rendered operator-side force on the same clock as video and joint state — a session where force and vision drift apart in time is worse than no force channel at all, since the policy learns a spurious relationship between contact and the wrong video frame.