Design Choices for Zero-Shot Synthetic-to-Real Object Detection in Industrial Environments

Abstract

Training object detectors exclusively on synthetic data eliminates costly site-specific annotation. However, the design choices governing whether these detectors scale to the real world are often made without controlled comparison. This study provides a systematic evaluation of safety-cone detection for a factory-floor task, using scenes rendered with domain randomization in CARLA and Unreal Engine 5.5 and a strict zero-shot protocol in which no real image informs training or thresholding. Our findings identify annotation geometry as a key factor, with silhouette projection clearly outperforming bounding cuboids. A leave-one-out ablation of five randomization factors shows that transfer is object-centric, letting an outdoor driving simulator supervise an indoor task. Sweeping architecture against on-device latency shows that this tolerance is not universal, since lightweight backbones lose far more from frozen background randomization than heavier ones. Randomization and architecture must therefore be chosen jointly. The detector reaches 63.5% mAP@50-95 and 96.8% mAP@50 on a selected real held-out test set. In a separate in-situ deployment, it reaches 99.3% precision and 94.4% recall over 144 cone instances in 118 frames.

Cite

@inproceedings{masood2027designchoices,
  title = {Design Choices for Zero-Shot Synthetic-to-Real Object Detection in Industrial Environments},
  author = {Masood, Wasif and Hadziaganovic, Armin and Muzaffar, Raheeb},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2027)},
  year = {2027},
  url = {https://empirischtech.at/papers/synthetic-to-real-detection}
}