Main image: Alex Anikiiev, Lukasz Grzegorczyk, Tariq Adaci, Arshiya Vij, Ashley Costigane, Brian Culver, David Dawson, Gemma Suen, Jose Santano Gonzalez, Kanae Nomura, Karol Zak, Penelope Webb, Scott Cheek, Yenwei Zheng
A digital solution developed during the Microsoft Global Hackathon 2023 to support Ukraine’s reconstruction has been included on the Microsoft Garage Wall of Fame. The project, titled AI-powered Reconstruction of Destroyed Buildings, leverages artificial intelligence and advanced visualization tools to assist local authorities in assessing and planning the rebuilding of war-damaged infrastructure.
The project originated as a response to the destruction caused by Russia’s full-scale invasion. A cross-functional team of engineers, designers, and data scientists sought to answer a central question: Could AI help rebuild what was lost?
From concept to prototype
During the hackathon, the team created a working prototype that used AI models to interpret and visualize 3D blueprints and construction data. The first use case focused on digitally reconstructing the destroyed House of Culture in Irpin.
To do this, the team applied the latest OpenAI language models (hosted on Azure OpenAI Service) alongside 3D visualization, photogrammetry, and construction metadata. The prototype included an Azure-based web app that allows users to interact with building data conversationally—asking questions and receiving structured information about layouts, materials, and reconstruction strategies.
A key technical achievement was the integration of Retrieval-Augmented Generation (RAG) with Azure Machine Learning and Universal Scene Description (USD) data. This allowed the GPT model to operate on complex spatial datasets, enabling semantic search and dynamic AI-driven querying.Technologies used in the prototype:
- OpenAI GPT Models for natural language interaction
- Azure Machine Learning and RAG for AI integration
- 3D models using GLB and USD formats for immersive visualization
- Azure AI Search for semantic search indexing
- Azure Web App with WebGL and ReactJS interface
Strategic pivot: from full reconstruction to planning support
Initially envisioned as a full digital reconstruction tool, the project shifted focus to preconstruction optimization. This decision reduced the data burden and increased practical value, enabling planners to make faster decisions on what and how to rebuild based on available blueprints and damage assessments.
The pivot enabled the team to deliver a functioning prototype by the end of the hackathon, opening new avenues for collaboration with public-sector partners.
Collaboration with Ukraine’s Ministry of Digital Transformation
Following the hackathon, the prototype evolved into a broader collaboration between Microsoft and Ukraine’s Ministry of Digital Transformation. A joint demonstration was presented at the 2024 World Economic Forum in Davos, showcasing the solution’s relevance to digital reconstruction efforts. The project demonstrated how AI design, planning, and simulation workflows can accelerate recovery and restore critical infrastructure.
The solution also became part of «Arina’s Story», a Ministry-led immersive documentary built with Unreal Engine and streamed via Azure Pixel Streaming. Visitors experience a virtual reconstruction of Irpin’s House of Culture while following the story of Arina, an 8-year-old Ukrainian dancer impacted by the war.
Ongoing development and future use
The AI-powered reconstruction tool continues to evolve, with new use cases being explored in historical asset preservation, immersive reconstruction planning, and urban development. According to Microsoft, the project is widely shared internally as a model for purpose-driven innovation, demonstrating how technology can illuminate paths to recovery, even in moments of crisis.
