visual servoing for deformable objects with pre-planned trajectory-guided geometric primitives

This paper presents a novel technique for the rigid alignment of a perspective camera to the deformations of a non-rigid object, using an adaptive visual servoing approach. Unlike existing methods, the proposed approach does not rely on any parametric or model-based priors regarding the deformation. Assuming the availability of a pre-planned camera trajectory observing the non-rigid object, the method aims to align this trajectory during the execution phase, utilizing only the most relevant landmarks as prior information. Importantly, the approach does not depend on any parametric or non-parametric models of the underlying deformation physics. Instead, it is formulated as a tracking problem embedded within an optimal visual control framework. This tracking process involves the visual servoing of geometric features from the deformable object, bridging the gap between the planning and execution stages. The optimal visual control is defined using a weighted least-squares criterion, which minimizes the distance between the reference features and those observed in real-time. The weights are time-dependent, smooth functions that encode the relevance of the visible object features. The experimental results demonstrate the method?s ability to adapt to various pre-planned trajectories and different types of deformations, without requiring prior knowledge, while also exhibiting resilience to noise in the detection of image features.