International Journal of Urban Management and Energy Sustainability

International Journal of Urban Management and Energy Sustainability

Deep Learning-Based Interactive Art: A Data-Driven Approach to Adaptive Aesthetic Experience

Document Type : Case Study

Authors
Maryam Anduz 1
Hamideh Jafari 2
Meyghol Zandieh 3
Sahar Eidy 4
1 Department of Business Administration, Ra.C., Islamic Azad University, Rasht, Iran
2 Department of Art, St.C., Islamic Azad University, Tehran, Iran
3 Department of Public Administration, West-Tehran Branch, Payame Noor University, Tehran, Iran
4 Department of Art, Ferdows institute of higher education, Mashhad, Iran
10.22034/ijumes.2025.735867
Abstract
The integration of deep learning into interactive art represents a paradigm shift in how audiences engage with aesthetic experience, moving from passive observation toward active, data-mediated co-creation. Despite growing interest, few empirical studies have operationalized this convergence within a unified system capable of real-time multimodal interaction. This study addresses a critical gap in the existing literature: the absence of an empirically evaluated, unified system capable of transforming real-time multimodal audience data into personalized artistic experience. To this end, a deep learning-based interactive art system was designed, implemented, and evaluated through a design-based, mixed-methods framework comprising five iterative phases-conceptual design, algorithm selection, software development, pilot testing, and interaction data analysis. The system simultaneously ingested EEG brain signals, gaze trajectories, voice, and text from 15–20 participants aged 18–45, processing these streams through a CLIP–StyleGAN2 pipeline to generate context-responsive visual outputs in real time. Quantitative analysis revealed elevated EEG cognitive engagement indices and sustained purposeful gaze attention across all sessions; generated artworks received high aesthetic scores for composition, chromatic diversity, and semantic coherence. Qualitative thematic analysis further confirmed that participants recognized the outputs as authentic reflections of their momentary inner states, regardless of prior artistic or technical experience. These findings collectively demonstrate that deep learning algorithms can bridge data-driven computation and lived aesthetic experience, and the proposed framework offers a replicable interdisciplinary model for future research at the intersection of digital arts, cognitive science, and adaptive human–machine interaction.

Graphical Abstract

Deep Learning-Based Interactive Art: A Data-Driven Approach to Adaptive Aesthetic Experience

Highlights

●      A novel framework for data-driven adaptive aesthetic experience based on real-time biometric and behavioral audience data is proposed, integrating EEG signals, gaze trajectories, and voice inputs into a unified generative pipeline.

●      A real-time interactive art system was designed and implemented using state-of-the-art deep learning models (CLIP, StyleGAN2), achieving stable, low-latency personalized output generation during live audience interaction.

●      Brain signal and eye-tracking analysis was combined with generative image algorithms to create artwork that semantically reflects the audience's instantaneous cognitive and emotional state.

●      A mixed-methods evaluation framework combining quantitative biometric data (EEG engagement indices, gaze heat maps) with qualitative thematic analysis of user feedback was applied to assess multi-layered aesthetic experience in a digital art context.

●      Deep learning algorithms demonstrated the capacity to semantically interpret human responses such that generated artworks carry both aesthetic value and a cognitive reflection of the individual viewer, bridging data-driven computation and lived aesthetic experience.

Keywords
Adaptive aesthetics
Data-driven experience
Deep learning
Generative neural networks
Interactive art
·         Anadol, R. (2020). Art in the age of machine intelligence [TED Talk]. TED. https://www.ted.com/talks/refik_anadol_art_in_the_age_of_machine_intelligence
·         Aris, S., Aeini, B., & Nosrati, S. (2023). A digital aesthetics? Artificial intelligence and the future of the art. Journal of Cyberspace Studies, 7(2), 219–236. https://doi.org/10.22059/jcss.2023.91316
·         Bian, Y., & North, C. (2021). DeepSI: Interactive deep learning for semantic interaction. In Proceedings of the 26th International Conference on Intelligent User Interfaces (pp. 197–207). ACM. https://doi.org/10.1145/3397481.3450648
·         Bishop, C. (2025). The social turn: Collaboration and its discontents. In The Performance Studies Reader (pp. 250–257). Routledge.
·         Bourriaud, N. (1998). Relational aesthetics. Les Presses du Réel.
·         Brielmann, A. A., & Dayan, P. (2021). Introducing a computational model of aesthetic value. Psychological Review, 129(2), 282–306. https://doi.org/10.1037/rev0000337
·         Candy, L., Edmonds, E. A., & Ascott, R. (2011). Interacting: Art, research and the creative practitioner. Libri Publishing.
·         Castellano, G., & Vessio, G. (2022). A deep learning approach to clustering visual arts. International Journal of Computer Vision, 130(11), 2590–2605. https://doi.org/10.1007/s11263-022-01664-y
·         Chung, S. (2022). Sketching symbiosis: Towards the development of relational systems. In The Language of Creative AI: Practices, Aesthetics and Structures (pp. 259–276). Springer. https://doi.org/10.1007/978-3-031-10960-7_14
·         Deng, Y., & Zhai, Q. (2025). Integrating deep learning in art and design: Computational techniques for enhancing creative expression. International Journal of Advanced Computer Science & Applications, 16(2). https://doi.org/10.14569/IJACSA.2025.0160278
·         Dewey, J. (1934). Art as experience. Minton, Balch & Company.
·         Edmonds, E. (2018). The art of interaction: What HCI can learn from interactive art. Morgan & Claypool Publishers. https://doi.org/10.2200/S00825ED1V01Y201802HCI038
·         Elgammal, A., Liu, B., Elhoseiny, M., & Mazzone, M. (2017). CAN: Creative adversarial networks, generating "art" by learning about styles and deviating from style norms. arXiv. https://arxiv.org/abs/1706.07068
·         Gatys, L. A., Ecker, A. S., & Bethge, M. (2016). Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 2414–2423). IEEE. https://doi.org/10.1109/CVPR.2016.265
·         Giannachi, G. (2009). San Francisco Museum of Modern Art: The art of participation: 1950 to now. San Francisco Museum of Modern Art.
·         Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
·         Jenkins, H. (2006). Convergence culture: Where old and new media collide. New York University Press.
·         Kaprow, A., & Kelley, J. (2004). The art of Allan Kaprow. University of California Press.
·         Kazemian, L. (2023). Long sentences: An international perspective. Federal Sentencing Reporter, 36(1–2), 63–80. https://doi.org/10.1525/fsr.2023.36.1-2.63
·         LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
·         Liu, X., Zhou, H., & Liu, J. (2022). Deep learning-based analysis of the influence of illustration design on emotions in immersive art. Mobile Information Systems, 2022, 3120955. https://doi.org/10.1155/2022/3120955
·         Liu, P., et al. (2024). Virtual dream reliving: Exploring generative AI in immersive environment for dream re-experiencing. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (pp. 1–11). ACM. https://doi.org/10.1145/3613905.3650983
·         Lockhart, E. N. (2025). Creativity in the age of AI: The human condition and the limits of machine generation. Journal of Cultural Cognitive Science, 9(1), 83–88. https://doi.org/10.1007/s41809-024-00158-6
·         Lu, J. (2022). A deep reinforcement learning based emotional state analysis method for online learning. Journal of Artificial Intelligence Practice, 5(2), 71–80. https://doi.org/10.46300/9104.2022.16.14
·         Madhuranga, D., Madushan, R., Siriwardane, C., & Gunasekera, K. (2021). Real-time multimodal ADL recognition using convolution neural networks. The Visual Computer, 37(6), 1263–1276. https://doi.org/10.1007/s00371-020-01864-6
·         Manovich, L. (2001). The language of new media. MIT Press.
·         Manovich, L. (2023). Artificial aesthetics: A critical guide to AI in art, media and design. Self-published.
·         Merleau-Ponty, M. (1945). Phénoménologie de la perception. Gallimard.
·         Mezei, P. (2021). From Leonardo to the Next Rembrandt: The need for AI-pessimism in the age of algorithms. UFITA: Archiv für Medienrecht und Medienwissenschaft, 84(2), 390–429. https://doi.org/10.5771/2568-9185-2021-2-390
·         Nake, F. (2012). Construction and intuition: Creativity in early computer art. In Computers and creativity (pp. 61–94). Springer. https://doi.org/10.1007/978-3-642-31727-9_3
·         Nicholas, P., Chen, Y., Borpujari, N., Bartov, N., & Refsgaard, A. (2021). A chained machine learning approach to motivate retro-cladding of residential buildings. In Proceedings of the 39th eCAADe Conference (pp. 8–10). eCAADe. https://doi.org/10.52842/conf.ecaade.2021.2.111
·         Oliveira, A. M. (2022). Future imaginings in art and artificial intelligence. Journal of Aesthetics and Phenomenology, 9(2), 209–225. https://doi.org/10.1080/20539320.2022.2136847
·         Oliva, A., Chandrakasan, A., & Huttenlocher, D. (2023). Report to the president for year ended June 30, 2023: MIT–IBM Watson AI Lab. MIT.
·         Photchanasavanee, T., & Chentanez, N. (2024). Screening of defects in AI generated portraits. In 2024 21st International Joint Conference on Computer Science and Software Engineering (JCSSE) (pp. 612–617). IEEE. https://doi.org/10.1109/JCSSE61278.2024.10613659
·         PINA, S. D. S. (2022). Prints of Fluxus art: Revisiting the legacy of expanded arts in Portugal (1970–1980). Art Journal, 81(4).
·         Putnam, E. (2024). On (not) becoming machine: Countering algorithmic thinking through digital performance art. Frontiers in Communication, 9, 1352628. https://doi.org/10.3389/fcomm.2024.1352628
·         Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (pp. 8748–8763). PMLR. https://proceedings.mlr.press/v139/radford21a.html
·         Reimann, M., Klingbeil, M., Pasewaldt, S., Semmo, A., Trapp, M., & Döllner, J. (2018). MaeSTrO: A mobile app for style transfer orchestration using neural networks. In 2018 International Conference on Cyberworlds (CW) (pp. 9–16). IEEE. https://doi.org/10.1109/CW.2018.00012
·         Refsgaard, A. (2018). Eye conductor [Interactive installation]. Andreas Refsgaard Studio.
·         Romic, B. (2022). Negotiating anthropomorphism in the Ai-Da robot. International Journal of Social Robotics, 14(10), 2083–2093. https://doi.org/10.1007/s12369-022-00289-8
·         Sawyer, R. K. (1999). The emergence of creativity. Philosophical Psychology, 12(4), 447–469. https://doi.org/10.1080/095150899105656
·         Sola, M. C., & Guljajeva, V. (2024). Visions of destruction: Exploring a potential of generative AI in interactive art. arXiv. https://arxiv.org/abs/2408.14644
·         Stark, L., & Crawford, K. (2019). The work of art in the age of artificial intelligence: What artists can teach us about the ethics of data practice. Surveillance & Society, 17(3/4), 442–455. https://doi.org/10.24908/ss.v17i3/4.13338
·         Wilson, S. (2003). Information arts: Intersections of art, science, and technology. MIT Press.
International Journal of Urban Management and Energy Sustainability
Volume 6, Issue 3 - Serial Number 3
Summer 2025
Pages 256-274

  • Receive Date 03 March 2025
  • Revise Date 25 May 2025
  • Accept Date 29 June 2025