Automated Identification of Visually Similar Construction Materials Using Robotic Hyperspectral Imaging

Abstract

Construction progress tracking relies on visual inspection and RGB-based imaging systems, which are limited when different material states appear visually similar. This challenge is especially relevant in drywall finishing workflows, where untreated and treated surfaces may appear nearly identical under typical indoor lighting although they represent completely different stages of completion. This dissertation investigates the use of hyperspectral imaging and robotics for automated identification of visually similar drywall material states in construction environments. The research developed and evaluated a robotic hyperspectral sensing framework that combined controlled spectral analysis, machine learning-based classification, embedded inference, and dynamic robotic validation. In the first phase, a controlled pilot study was conducted to determine whether hyperspectral imaging could distinguish between different drywall surfaces. The results showed measurable spectral separability among the selected classes and demonstrated that nonlinear machine learning models were effective for this task. The pilot findings also indicated that the strongest distinction occurred between untreated and treated drywall surfaces, while separation among treated subclasses was more difficult. In the second phase, the dataset was expanded and broader benchmarking was conducted to support deployment-oriented model selection. The selected framework was then integrated into a robot-mounted sensing system using a Living Optics hyperspectral camera, an NVIDIA Jetson AGX Orin edge computer, and a ROS-based communication and mapping workflow. Dynamic validation was performed in a mock construction environment using the Boston Dynamics Spot robot under multiple layout, lighting, and navigation conditions. The results showed that hyperspectral material-state classification could operate during robotic motion and be represented spatially within a mapped environment. However, the dynamic experiments also revealed that the most reliable practical boundary remained the distinction between untreated and treated drywall, while fine-grained treated-subclass differentiation remained more challenging. In addition, continuous scene-wide sensing produced false positive material markers on non-target surfaces, highlighting the need for stronger contextual filtering in future systems. Overall, this dissertation demonstrates that robotic hyperspectral sensing is a feasible and promising approach for material-aware construction progress tracking.