Logo image
Geometry-Aware Visual-Tactile Image Representation Learning with Uniform Proxies on the Hypersphere
期刊文章   同儕審查

Geometry-Aware Visual-Tactile Image Representation Learning with Uniform Proxies on the Hypersphere

Chen Guo, Wenxi Liu, Aiping Huang, Chia-Wen Lin 和 Tiesong Zhao
ACM transactions on multimedia computing communications and applications
15/09/2026

摘要

Computing methodologies Information retrieval Information systems Learning paradigms
Understanding the physical properties of object surfaces is crucial for robotic systems to perform delicate operations and perceive the physical environment. Recent advances in optical sensor-based tactile imaging enable the capture of high-resolution contact information, demonstrating significant potential for multimodal robotic perception. Despite this progress, visual-tactile image representation learning remains under-studied. Our analysis explores two key challenges: visual and tactile images exhibit significant spatial inconsistencies, and tactile data shows more severe inter-class similarity and intra-class variation than common modalities. These factors limit the effectiveness of existing approaches based on direct instance-level alignment. To address these challenges, we propose a geometry-aware visual-tactile representation learning framework on a hyperspherical space. Specifically, we introduce a set of modality-agnostic class proxies that serve as shared semantic anchors to guide cross-modal alignment. Instead of enforcing direct pairwise correspondence, both modalities are organized around these proxies, enabling more stable and consistent semantic grouping. Furthermore, we impose a uniformity constraint on the proxies to explicitly regularize the global structure of the hyperspherical space, improving class separability. Extensive experiments on multiple visual-tactile benchmarks demonstrate that the proposed method consistently outperforms existing approaches and leads to well-structured and discriminative hyperspherical representations.

相關連結

指標

1 檢視次數

詳細資料

Logo image