The DoRIAH project aims to investigate the analysis of remote sensing images from a human-in-the-loop perspective. Its goal is to allow the semi-automatic georeferencing, 3D reconstruction and detection of various small-size objects in remote sensing images of any kind, from historical aerial images to modern-day satellite images. This is enabled by implementing a holistic ecosystem where automatic, computational methods are combined with human recognition power by a Visual Analytics approach. DoRIAH has the vision to generalize the analysis of remote sensing images in contrast to commonly encountered, computationally driven attempts where ICT experts develop various solutions for specific application domains without much impact on other domains.