Multi-Modal 3D Neural Representations for Scene Modeling: Towards Building Health Management and Energy Evaluation

Abstract

Prognostics and Health Management (PHM) for buildings requires accurate digital representations that support energy-efficiency assessment and predictive analysis throughout the asset lifecycle. This is particularly important for building envelopes, whose condition directly affects thermal performance, energy efficiency, occupant comfort, and long-term structural health. However, most existing buildings lack simulation-ready digital models, while current inspection and modeling workflows remain labor-intensive, geometrically limited, and inadequate for integrating thermal and physical information. This thesis proposes a cost-efficient and automated framework for reconstructing simulation-ready, multi-modal three-dimensional building representations from visual inputs alone. The research is organized around three main contributions. First, it develops semantic scene reconstruction methods that combine neural implicit surface modeling with semantic prediction to recover detailed 3D building envelopes and estimate key characteristics such as window-to-wall ratio and footprint. Second, it investigates learning-based estimation of thermophysical parameters through implicit thermal field reconstruction and differentiable heat-transfer simulation. Third, it addresses practical sensing constraints by enabling multi-modal reconstruction from limited and unsynchronized thermal observations.