FeDepth: Federated Learning for Depth Estimation under Robot Heterogeneity

Although recent robot perception research emphasizes train-ing on data from diverse environments to improve generalization, mostexisting methods still rely on centralized learning, which is inefficient anddifficult to scale across heterogeneous robot platforms. Federated learn-ing (FL) offers an alternative by enabling distributed training withoutraw data transfer, but it suffers from severe performance degradation un-der domain shifts caused by heterogeneity across clients. In real roboticdeployments, data distributions often overlap across platforms, environ-ments, and sensing conditions, making it difficult to partition clientsinto clearly separated domains. However, this characteristic breaks theassumption of clearly separable client domains commonly used in clus-tered FL. To address this gap in robot perception, particularly in depthestimation, we introduce two realistic and unexplored non-IID scenariosthat reflect heterogeneity in terms of platform, environment, and depthdistribution. We then propose FeDepth, a descriptor-based clusteredFL framework that models client relationships through soft clustering.Unlike hard clustering methods that assume clearly separated clusters,FeDepth allows clients to participate in multiple clusters, capturing con-tinuous and ambiguous domain transitions commonly observed in roboticenvironments. Extensive experiments demonstrate that FeDepth con-sistently improves robustness over standard FL and clustered FL base-lines across multiple depth estimation architectures, providing a practicaland effective solution for federated robot perception. Our project pageis available at https://vision3d-lab.github.io/fedepth/.