Generative AI for Immersive Experiences: Integrating Text-to-Image Models in VR-Mediated Co-design Workflows

Text-to-image AI models can generate novel images for design inspiration. Yet, their applications for collaborative design (co-design) purposes and interoperability within simulation-based, immersive settings have been scarcely explored. In this paper, we propose a novel, multi-modal approach for interactive public participation in urban design projects. The main objectives of our research are (a) to describe a methodological workflow of integrating text-to-image AI models into VR-mediated co-design workshops, and (b) to investigate the applicability of the proposed workflow through a set of completed and prospective case studies. Both studies are parts of a broader research project, which aims to revitalize the city of Derby, UK through producing a series of sustainable design visions. Through these case studies, we discuss some preliminary results and introduce our future work.