Pose from Silhouettes (PfS) is the problem of estimating a known object's 6 Degrees of Freedom (DoF) pose from its observed silhouettes - this article solves this problem under orthographic projection. We introduce GlOptiPoS: a globally optimal approach for silhouette-based pose estimation, applicable to arbitrary 3D shapes without correspondences, with applications in medical imaging, robotics, and augmented reality.
We address the problem of 3D pose estimation from silhouettes, termed as the Pose from Silhouette (PfS) problem, within geometric computer vision and 3D reconstruction. Unlike learning-based approaches, we propose a globally optimal method for estimating pose from orthographic silhouettes using analytical properties of silhouette geometry.
We solve the problem of determining the pose of known shapes in $\mathbb{R}^3$ from their unoccluded silhouettes. The pose is determined up to global optimality using a simple yet under-explored property of the area-of-silhouette: its continuity w.r.t trajectories in the rotation space.
The proposed method utilises pre-computed silhouette-signatures, modelled as a response surface of the area-of-silhouettes. Querying this silhouette-signature response surface for pose estimation leads to a strong branching of the rotation search space, making resolution-guided candidate search feasible.
Additionally, we utilise the aspect ratio of 2D ellipses fitted to projected silhouettes as an auxiliary global shape signature to accelerate the pose search. This combined strategy forms the first method to efficiently estimate globally optimal pose from just the silhouettes, without being guided by correspondences, for any shape, irrespective of its convexity and genus.
We validate our method on synthetic and real examples, demonstrating significantly improved accuracy against comparable approaches.
Pose from silhouette, silhouettes, pose estimation, 3D pose estimation, shape from silhouette, orthographic projection, geometric computer vision, 3D reconstruction, global optimization, CVPR 2026
Building upon the framework of Hartley and Kahl, we introduce shape signatures of orthographic silhouettes that vary continuously with an object’s orientation, enabling non-trivial branching strategies that lead to solutions converging to global optima, up to discretisation. Concretely, We propose two intuitive yet powerful silhouette descriptors: the Area-of-Silhouettes (AoS) and the aspect ratio of ellipses fitted to the silhouettes; we show that they capture sufficient geometric information for accurate pose estimation. Orientation is recovered numerically, followed by a post-hoc non-linear refinement on the $\mathbb{SE}(3)$ manifold guaranteeing convergence to a solution that lies in close vicinity of the global optima, up to discretization.
Shown below are some randomly sampled results from the test objects Stanford Bunny (SB), Phlegmatic Dragon (PD), and Pelvic Bone (PB).
Assuming depth priors as input, the same solution methodology as above achieves near-optimal accuracy for perspective silhouettes, demonstrating strong practical performance while preserving an explainable, initialisation-free design. However, the perspective case is not guaranteed to converge to the vicinity of global minima.
Shown below are some randomly sampled results from the asymmetric objects from the BCoT dataset; the average accuracy remains high but being based on an approximation, some results are inaccurate.
@inproceedings{sengupta2026pfs,
title={Globally Optimal Pose from Orthographic Silhouettes},
author={Sengupta, Agniva and Kuş, Dilara and Li, Jianning and Zachow, Stefan},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2026}
}