Remote Rhythms: Audience-informed insights for designing remote music performances (2024)

Over the past few decades, music consumption has increasingly shifted to digital formats[18], with technology also transforming audience experiences at live music events, influencing their engagement and preferences[12]. This transformation spans various aspects, from the concert experience itself, to enabling global access to live performances from the comfort of home. The rise of social media, social VR, and livestreaming platforms has reshaped the cultural landscape of music consumption, by introducing new dimensions to how concerts can be experienced and accessed from home and by expanding the ways in which audiences interact with each other and with the artists. The COVID-19 pandemic has further accelerated this shift, catalysing the adoption of these digital solutions and pushing the boundaries of how we define live music experiences[48, 49, 52]. During this period, both musicians and audiences had to rapidly adapt to this new, uncertain environment[9], moving away from traditional practices toward increased digitisation and accelerated acceptance of technology in music consumption and performance[31]. Livestreaming on platforms such as Instagram, YouTube, and Twitch [38] gained momentum as spaces to hold online concerts, providing artists with new means to connect with their audiences and maintain their artistic activities[15, 26]. Some examples are renowned musicians like John Legend, who leveraged these platforms to perform for remote fans directly from their homes1. Meanwhile, other artists such as Travis Scott, Ariana Grande or Lil Nas X took a different approach and collaborated with gaming platforms (e.g., Fortnite, Roblox) to deliver virtual concerts that attracted millions of viewers2. At the grassroots level, lesser-known artists have also found these platforms invaluable. For example, independent musicians went on Instagram Live and Twitch to host intimate performances, engage in real-time interactions with fans, and cultivate a community presence[22, 53]. The continued use of these platforms3 by both high-profile and emerging artists points to a lasting change in how music is performed and consumed post-pandemic[20]. This shift requires further investigation into how digital platforms are reshaping artist-audience interactions, the economic impacts on the music industry, and the potential for new forms of artistic expression. This research can inform future technological advancements and lead to the development of policies that foster a resilient music ecosystem.

For this process to be effective, it is important to acknowledge the diversity within this space. Different platforms serve various purposes, each distinctively influencing what and how music is performed, consumed and monetised. For instance, Twitch offers a platform for interactive and monetisable4 experiences[25, 57], Instagram Live benefits from more versatile and intimate live streaming features, while VR-based platforms like VRChat transport users into immersive 3D environments where they can attend virtual concerts, interact with other concertgoers’ avatars, and even dance to the rhythm of the music[35, 50]. New platforms, dedicated exclusively to music livestreaming (e.g., Veeps5, StageIt6) have emerged, but many have struggled to attract a significant audience and subsequently disappeared. These difficulties could be attributed to a lack of understanding of the nuances and preferences of remote concertgoers, making it difficult to provide a satisfying experience. Thus, the future relevance of both existing and emerging platforms will largely depend on a deep understanding of user needs and demands in the online space.

Existing knowledge of audiences’ behaviour in remote music performances is often limited to specific platforms (i.e., Facebook Live, Twitch, Zoom), events (i.e., rave concert, jazz concert, traditional folk concert) or contexts (i.e., the COVID-19 pandemic). For instance, Vandenberg etal. [58] looked at audiences’ comments during pandemic livestreamed raves on Facebook Live and found that such events fostered a sense of collective consciousness and social solidarity among participants. The focus on the pandemic was maintained by d'Hoop and Pols [14], who offered an ethnographic perspective on a jazz concert, emphasising the spontaneity of the event and the unpredictable interactions that emerged between musicians and their real-time audience. Benford etal. [2] follow, through an ethnographic approach, the pandemic transition of two traditional folk clubs to online experiences via Zoom and Facebook. Rendell [43] expands on what is considered pandemic media and introduces the concept of portal shows which merge traditional live gigs with screen and new media. Their study analysed users’ posts during three portal shows with a focus on online fan subject-positions, affect, performances and communications and suggests there is potential value in expanding the ways audiences can participate in online performances, as well as encouraging a more varied and interactive approach. Despite some commonalities across these platforms, distinct patterns of audience behaviour also emerged. For instance, Vandenberg [57] focused on music consumption on Twitch and found it is particularly effective in facilitating smaller-scale interactions, with many participants engaging with it as a peripheral activity. The complexity and variety of this space necessitate research across a broad spectrum of platforms. By doing so, we can uncover common behavioural patterns and also understand the unique interactions and preferences specific to different environments. This approach can enable the development of more effective engagement strategies, tailored to the evolving dynamics of the online music landscape.

As highlighted earlier, most platforms currently used for remote music experiences were not originally designed for this purpose but were instead appropriated from other contexts. The design space for remote participation in livestreamed music events, especially from an audience engagement perspective, remains largely unexplored in HCI. In contrast, the field of networked music performances, which focuses on connecting musicians over the internet to simulate playing together in the same room, is better understood[10, 44]. Although these performances sometimes include audiences, their role is generally passive, confined to watching the artists on a screen[46, 56]. However, previous research indicates that remote audiences often desire a more interactive concert experience[2]. Achieving this involves overcoming several challenges such as internet latency, sound quality, and managing multi-user participation. These issues are addressed within the Internet of Musical Things, an extension of the Internet of Things concept applied to the musical domain. Its goal is to develop technologies that enhance two-way communication in music, not only between performers but also between performers and audiences, and among the audience members themselves[23, 63].

Recent advancements in the metaverse and immersive technologies have introduced new opportunities in this domain, already demonstrating the potential for enhancing musicians’ practice[27, 39, 40, 47]. Turchet [54] introduced the concept of musical metaverse, a space dedicated to interactive live concerts, immersive technologies and Internet of Musical Things. Turchet [54] provided a comprehensive analysis of the opportunities and challenges in this upcoming field and discussed the potential of web-based playgrounds in this space[4]. Alongside immersive technologies, innovative interfaces were also shown to have potential in this area[5, 6, 11, 13, 25, 28]. Although there have been significant advancements and technological experimentation, research into the design of systems for remote music performances is still in its early stages. Further study is necessary to determine how the musical metaverse can be optimally designed to enhance engagement, accessibility, and the overall quality of remote and hybrid music performances.

To explore audiences’ experiences across various digital platforms during remote music events, our study employed a multifaceted data collection strategy consisting of:

• A 26-question online survey to gather insights into audience demographics, preferences, and attitudes toward remote music events. The survey featured a combination of multiple-choice questions and open-ended queries. Participants were asked about their frequency of attending music events remotely, the platforms they use, etc. Additionally, open-ended questions related to most memorable at-home music performance, detailing both positive and negative aspects and highlighting what made these experiences stand out.
  
• Semi-structured interviews to go deeper into the nuances and complexities of audience engagement through personal narratives, which could not be fully explored through surveys alone. Each interview lasted between 50 to 60 minutes. All interviews were audio-recorded and then transcribed using an online automated transcription tool. To ensure the precision of the transcripts, two student researchers meticulously reviewed and cleaned the data.
  
• Observations of 11 remote music events , totalling approximately 18 hours, across various platforms including Twitch , YouTube , Instagram , Social VR , and Video On Demand (VoD) – see Figure3 for a collection of screenshots from the observed events. The observations allowed us to directly witness audience behaviours and interactions in real-time. During these events, we systematically captured data, which consisted of screenshots and annotations. All observational data were collected and organised on Miro, for visualising and analysing patterns in audience engagement across different digital platforms.
  

Survey participants and interview participants were two distinct groups, formed through recruitment efforts conducted at different times via our personal and professional networks and social media. The timing difference naturally resulted in varying participants for each method, thereby contributing to a broader range of viewpoints. The diversity in participation enriched our study, providing a wide array of perspectives on remote music events. All research methods and participant interactions were reviewed and approved by our Institutional Review Board, to ensure ethical standards were maintained throughout the study.

The survey was completed by 67 participants from 16 different countries, with ages ranging from 18 to over 55 years old (18-24: 27%; 25-34: 29%; 35-44: 12%; 45-54: 19%; and >55: 12%). Participants identified as 33 women, 32 men, one non-binary and one preferred not to disclose their gender. They used a variety of platforms to engage with remote music events, which included YouTube (59%), television / VoD (53%), Instagram / Facebook (30%), teleconferencing platforms such as Zoom (20%), Twitch (19%), VRChat (10%), Other (e.g., Veeps, We-verse) (15%).

We then interviewed 25 participants who had used and experimented with a combination of platforms, including some that were less represented in existing research (e.g., VRChat, Engage, specialised streaming platforms). Interviewees from 8 countries identified as 10 women, 14 men, and one individual who chose not to specify their gender, with and ages ranging from 18 to over 55 (18-24: 28%; 25-34: 52%; 45-54: 16%; and >55: 4%). The platforms they used for remote music events included TV / VoD (24%), YouTube(48%), Instagram / Facebook (28%), Twitch (28%) specialised streaming platforms (28%), social VR tools like VRChat or Engage (20%) and other platforms such as Zoom (8%). All participants used multiple platforms at least once.

The data collected from the survey, interviews, and observations was analysed using reflexive thematic analysis [7, 8], with the researchers being active creators in the analysis. We employed an iterative open-ended coding process, identifying data patterns related to desires, opportunities and challenges. We started this process by analysing the survey data. First, the main researcher familiarised themselves with the data by reading the entire dataset. At this point, they documented their thoughts and took preliminary notes such as Participants want to keep the performer in the spotlight, without overshadowing them with technology; It appears there is a general consensus that current engagement strategies are insufficiently interactive or engaging; The concept of “live” appears fluid among participants within this digital environment, as discussions about live and recorded content often overlap; etc. Some of these notes informed the interview questions and the finalised thematic analysis. This was followed by a preliminary coding 7 of the open-ended questions. After processing the remaining data (such as interviews and observations) in a similar manner, the main researcher refined the coding. Subsequently, another researcher collaborated with the main researcher to review and analyse the coded data, by integrating and comparing initial codes from all data sources. To ensure all other team members were aligned, detailed documentation of the initial coding and clustering processes was shared, followed by a session for discussion and feedback. Through the iterative process, the team was able to determine which codes were most useful for interpreting relevant themes, how they may be combined according to shared meanings, and which could be discarded. This led to the identification of four themes, which were further broken down in 17 design elements, which we present in Section4.

Our analysis indicates that remote music events are catalysts for the emergence of a new type of online community. However, the community's potential for fostering meaningful social connectedness is not yet fully realised, due to a lack of specialised tools designed for this purpose. Despite the utilisation of certain existing features on online platforms, such as live chatrooms and social media integration, there is a clear and pronounced desire among participants for more authentic interpersonal interactions within the musical context. Our interviews and observations revealed a keen interest from audiences in exploring each other's musical preferences, promoting collaborative play, and nurturing spontaneous interactions. The insights complement research conducted by Onderdijk etal. [36], which investigated the role of agency, presence, and social context in enhancing participants’ experiences at virtual concerts. Our study showed the significant impact of agency and shared agency on fostering social connectedness, as well as the importance of visual and social cues. Notably, it was found that platforms enabling visual interactions among participants, like Zoom, significantly boost the sense of social presence and connectedness within the audience. Aligned with these insights, our work points to the inherent community-building potential within current digital platforms, while simultaneously identifying a critical need for innovative approaches. Drawing from our findings, in Table 1 we have identified four elements pivotal for enabling a more profound sense of social connectedness during remote music events.

Remote settings offer unique opportunities to enhance and evolve the relationship between artists and their audiences, as highlighted by Leshem and Schober [30], who showed that empathetic concern allowed listeners of an improvised jazz concert to experience emotions similar to the performer's emotions when listening to the audio recording online, although it did not facilitate shared emotions in the live concert setting. This seems to indicate that empathetic concern may facilitate greater emotional understanding in digital environments. Currently, audiences leverage digital platform features such as live chats, reactions, and social media integration to engage with artists, and this fosters a sense of connection and agency during virtual events. Previous research identified various elements that enhance this connection. For instance, Onderdijk etal. [36] found an increased artist-audience connection when audiences hear their favourite songs or attend VR concerts. Our findings expand on these insights by identifying additional elements that contribute to deepening this bond such as providing opportunities for co-creation, and insider access to behind-the-scenes content. While existing digital features provide a foundation for interaction, there is scope for the development of new tools and features, as revealed by our participants. The four key elements identified for enhancing the connection audience-artist are detailed in the Table 2.

Digital platforms like Twitch, Instagram Live, and VRChat are redefining the concept of ‘live’ in music performances. They offer dynamic participation in live events, unbound by physical presence or traditional temporal constraints. Audience engagement varies: some viewers stay synchronised with real-time broadcasts, while others utilise the flexibility to pause or rewind. This flexibility reshapes what ‘live’ means, allowing to pause a live stream and dissect, discuss, and deeply engage with specific moments as they unfold. Our observation supports the idea from Hammelburg [21] that the meaning of “live” in a mediated, digital context is constantly evolving and being re-defined. The ephemeral nature of remote live music brings unique opportunities as it challenges the limitations of the physical world, but at the same time, it can make remote music events feel less memorable as they could be considered less of an experience. In Table 3, we present four elements that point to potential areas of intervention in the context of temporal and spatial fluidity, as emerging from our data.

Traditional live music at its core is a multisensory experience, immersing attendees in vivid sights, sounds, temperatures and even smells. The scent of a venue, “the kick of the drums in your chest and the bass and the flow of the people ” - all these elements converge to create lasting memories. However, the digital reality of remote music events often grapples with what can be termed as ‘sensory paucity’, underscoring the diminished sensory richness in digital environments compared to real-world experiences. The significance of sensory information in digital environments is highlighted in the study by Phillips and Krause [37], which explores the future of classical music livestreaming, emphasising how sensory experiences enhance digital engagement. Similarly, the research in ecommerce by [24] investigates the impact of sensory language used by virtual streamers (e.g., descriptive words like “tasty” and “smooth”) on consumer reactions, further underscoring the crucial role of engaging the senses in shaping online experiences. Our study's findings highlight a similar craving for immersive experiences in remote music events, where participants yearn for a depth of engagement that goes beyond listening. They seek holistic experiences that captivate their senses and emotions, mirroring the intensity and vibrancy of physical concerts. The desire for sensory and emotional immersion is a testament to the powerful impact music has when it fully engages multiple facets of human perception and feeling, highlighting the importance of creating remote music events that are not just heard and seen, but are felt in a more comprehensive and emotionally resonant manner. Under this theme, we identified five important elements which are presented in Table 4.

As shown in Figure2, in Phase 2 of our project, we conducted a human-centred co-design workshop aimed at exploring and refining the design elements for online music performance technologies that were identified in Phase 1. By engaging stakeholders directly in the ideation process, we presented them with raw data to foster creativity and gather diverse perspectives. To this end, we intentionally chose not to present the pre-identified design elements directly to the participants, in order to foster unbiased engagement with the data, encouraging stakeholders to form fresh perspectives and original interpretations. It was also intended to stimulate creative thinking and minimise confirmation bias, avoiding the influence of our analysis on the stakeholders’ ideation process. More importantly, our methodology allowed us to observe whether certain elements we had identified would naturally emerge in the stakeholders’ discussions, highlighting which elements were most resonant and significant to the stakeholders, and providing a deeper understanding of the most prevalent aspects in designing for experiential engagement in remote music events. By involving stakeholders in this manner, we aimed to value their insights and foster a sense of ownership in the research outcomes, ensuring that the outcomes were organic and reflective of a diverse range of experiences and perspectives.

We recruited 12 participants (ages between 18 and 40 years old; five men, six women, one non-binary) for the co-design workshop through personal and professional networks. All participants self-declared as frequent concert-goers and had experience attending remote music performances. Moreover, their diverse involvement in the music industry, ranging from performing to organising, enriched the depth of perspectives and insights they brought to the workshop (see details about the occupation and relevance to the study of our participants in Table5). Participants originated from eight countries—Taiwan, the UK, South Korea, China, Romania, France, Portugal, and the Philippines—and all currently reside in the UK. Two additional researchers were present at the workshop: the first author as workshop facilitator and a researcher who documented the process.

We designed a deck of cards building on the data we collected and analysed in our study to serve as starting points. The deck included 44 cards of two types and their content emerged from the thematic analysis of our data (as shown in Figure2): (i) Four cards with How might we...? questions (i.e., How might we encourage shared audience experiences during remote music events?; How might we preserve the atmosphere of a live performance during remote music experiences?; How might we enable meaningful interactions between audiences and musicians during remote music events?; How might we make remote music events more memorable?); (ii) Five sub-groups of eight cards each, one for each of the following broad data categories: a) challenges; b) opportunities; c) aspects to consider when designing technology for remote music experiences; d) technology and e) ideas. On the reverse side of each card, we included curated quotes from participants8 – see Figure4 for a sample of the used cards. Besides the deck of cards, various materials were available: pens, post-its and paper for the ideation; and theatre props, projectors, VR headsets, musical instruments and smell stickers for the prototyping stage.

The three-hour workshop began with a 10-minute introduction, where participants signed consent forms and were briefed on the day's objectives. This was followed by a 10-minute ice-breaker activity, with participants brainstorming the worst ideas for remote performances, to encourage open dialogue and reduce self-censorship. Then, we gave a 10-minute presentation on the study itself presenting how the “How might we..?” questions emerged, as well as some of our observations from the data gathered during Phase 1. After this, participants were divided into four groups of three, each group receiving a deck of cards as described in Section5.3. Participants had 10 minutes to browse through the cards, followed by a crazy eights activity where they had eight minutes to sketch eight ideas based on the data in their deck of cards, addressing the “How might we..?” question they received. This was done to get participants into a creative mindset. An hour long ideation and prototyping phase followed, during which participants suggested a design concept and developed it into a Wizard of Oz prototype (see Figure5 for snapshots of this process). Besides the deck of cards, various materials were made available: pens, post-its and paper, theatre props, projectors, VR headsets, musical instruments and smell stickers. The final hour included a 30 minute enactment session where participants enacted their prototypes in front of the group (see Figures6, 7), followed by a short presentation explaining their thought process. The workshop closed with an open ended discussion, for participants to unpack their experience during the workshop, comment on the ideas provided, and point out concerns, challenges and opportunities around the future of remote music audiences. At the end, participants were each given an empty piece of paper where they were asked to write their final thoughts about the day.

Data from the workshop, including transcriptions of videos, post-it notes, drawings, and prototypes, underwent a mapping process. This method aligned the data with previously identified elements, helping to refine them and generate actionable design insights, intended to inform and guide other designers in their projects.

Group 1 (P01 – P03) had as a starting point the question How might we encourage shared audience experiences during remote music events? For this, they proposed a design for enhancing personal connections during the entire concert journey – before, during, and after the show – while also providing a sense of belonging to a broader community. Their concept promoted a fully immersive experience, where the audience participates as avatars. The design integrated the following features, which we detail below and discuss in the light of the elements resulted from Phase 1.

• Private interconnected virtual spaces: Users should have the option to reserve a virtual space that mimics a VIP stadium booth. This feature would allow them to experience the event alongside friends who are in different locations. Participants can attend the performance with their selected group but also have the ability to see the booths of other audiences. This would allow for smaller, more meaningful interactions between audiences and artists while maintaining the feeling of a large crowd attending the event, pointing towards elements #1 and #2. Each space should include a stage where the performance occurs, creating an intimate setting that makes each group feel as if the performance is tailored specifically for them, pointing to elements #5 and #17.
  
• Custom artist designs: The virtual spaces for each performance should be unique and based on the aesthetic of the artist performing, giving the audiences the feeling that they are stepping into the musician's world and strengthening the emotional bond between the artist and their fans (elements #8 and #11).
  
• Audience lobby: Before the performance starts, users would have the opportunity to enter a fan lobby using their avatars (element #4). Here, they can meet other fans from all over the world (element #1) and make new friends whom they might want to share their space with, thereby fostering new friendships and community ties. This space also acts as a social hub for organic discovery, where attendees can learn about new artists or performances from fellow fans (element #10).
  
• Synchronised dining experience: The concept includes a synchronised food delivery system. Before the experience, the artist curates a food menu that attendees could opt to have delivered to their homes from their local restaurants (see Figure 6a). This way even in different locations attendees can share the same food, mimicking the shared dining experience one would have at a physical venue. Moreover, this adds a tangible, real-world element (elements #13, #14).
  
• Merchandise and artist interaction: The concept also incorporates a virtual merchandise shop where audiences can buy items they like, which will then be physically delivered to their place. Here, the artist can hold signing events and meet-and-greet sessions for their fans (elements #6, #8, #13).
  

Group 2 (P04 – P06) approached the How might we preserve the atmosphere of a live performance during remote music experiences? question through an experience meant to engage users on multiple sensory levels. They proposed a physical object designed for home use, which created an immersive concert experience beyond the screen (see Figure6b). In doing so, the design sought to transform the remote music experience from a predominantly passive, observational activity into an interactive, multisensory journey. The following design features were proposed:

• 360-degree environmental projection: The device projects a live, atmospheric environment around the user, offering a panoramic view of the performance. Users can walk around it, engaging with the performance from various angles. They can also pause it, or replay it experiencing the same moment from different perspectives (element #5). In addition, this projection creates an immersive sound backdrop, complementing the visual experience (element #14).
  
• Olfactory stimulation module: Equipped with scent cartridges, the device releases fragrances aligned with the concert's theme, enhancing the thematic immersion. This links to elements #11 and #14.
  
• Haptic feedback system: To replicate the sensation of live music, the device integrates a sophisticated vibration system. This technology allows users to feel the music in a tactile manner, simulating the vibrations one would experience at a live venue, thus adding a physical dimension to the auditory experience and pointing towards elements #13 and #14.
  
• Audience holograms: For users desiring a more communal experience, the device can project holograms of other audience members into their space. This feature fosters a sense of unity and shared experience, crucial to live performances, by visually and virtually connecting individuals in different locations (element #1).
  

Responding to the question How might we enable meaningful interactions between audiences and musicians during remote music events?, Group 3 (P07 – P09) proposed an app-based concept to strengthen the interaction between artists and their audience in virtual music events, by giving audiences a voice in shaping the concert experience. The design includes the following features:

• Audience-driven setlists: Through the app, attendees can use polls to vote for their favourite songs, ensuring the performance reflects their preferences (element #6, #7).
  
• Co-writing music: The chat function allows for real-time collaboration between the artist and audience with attendees being able to contribute lyrics while the musician plays, fostering a playful co-creation process where the song evolves during the performance (elements #3, #7).
  
• Unique merchandise: Unique merchandise, either virtual or physical, is available exclusively through the app. Artists can ‘hand out’ these items during the show, making the experience more memorable and meaningful. Physical merchandise could be delivered to the attendee's home before or after the event as a tangible memento (element #13).
  
• Enhanced pre- and post-event engagement: Leveraging the virtual format, the app offers Q&A sessions with the artist before and after the performance, allowing fans to share the experience beyond just the music, building anticipation and prolonging the post-event connection (elements #1, #6, #16).
  
• Accessibility features: Recognising the importance of inclusivity, the app incorporates features like close captioning, ensuring that the event is accessible to a diverse audience, including those with auditory impairments (element #12).
  

Group's 4 (P10 – P12) idea features an Internet of Things intervention which modifies the home space according to the concert involving the appliances one has at their home. Designed to immerse the audience in the music and enhance the connection to the virtual event, the smart home appliances become dance partners, turning the living room into a pulsating dance floor by receiving information about the show and responding to it (see Figure 7). Moreover, the concept proposes a series of multisensory abilities for the devices and a feature that randomly replays the best moments after the show. Central to this concept is the idea of engaging with things that people have at home, leveraging the physical space rather than transporting the user to a virtual environment. This way the home creates a new experience every time users attend an event, making each concert unique and memorable. The design has the following features:

• The groovy fridge: A fridge that has a built-in sound recognition system and a set of tiny robotic legs. When it detects music playing it starts swaying and displays colourful LED lights that sync with the music, creating an ambient light show that enhances the mood of the concert (element #14, #15).
  
• Rhythmic washer and dryer duo: These appliances are activated during remote music events and are equipped with sensors that detect the rhythm of the music, synchronising their spin cycles to match the beat (element #14).
  
• Pet engagement device: a “DJ Dog Toy" that reacts to the music, providing entertainment for pets and integrating them into the experience. These toys vibrate, light up, or play modified sounds in harmony with the concert (element #15).
  
• Smart vacuum dance mode: Event experiences are saved and whenever the user vacuums the cleaner randomly replays music from past events but also performs a dance routine, turning a mundane cleaning session into an entertaining show (element #9).
  
• Selective replay control for appliances: Each smart appliance can be individually programmed to respond to selected replay moments. For example, the user might want the Groovy Fridge to light up and sway to a particular song's chorus or release a specific scent during a memorable guitar solo (element #9).
  

Liveness [1] is a defining characteristic of in-person events, which captivate audiences with their raw energy. However, within the digital space, liveness becomes mediated and its meaning is constantly being re-defined by audiences [21]. With digital platforms like Twitch, Instagram Live, and VRChat, audiences have greater agency over their viewing experience: some stay synchronised with real-time music broadcasts, while others utilise the flexibility to pause or rewind. This means that not all viewers are experiencing the event in the same temporal frame. In this context, liveness is less about temporal simultaneity and more about the depth of engagement and the shared experience, even if temporally staggered. This transformation has be outlined byBenford etal. [2], who discusses this fluidity of liveness in the context of the ethnographic account of two traditional folk clubs going live.

In addition, we found that even pre-recorded music performances can evoke a strong sense of liveness among viewers. Therefore, the perception of liveness becomes a combination of the viewer's belief, which is anchored by specific markers and cues within the content, and their active interactions with both the performance and other audience members. Nevertheless, this perception can vary. A viewer might feel a performance is ‘live’ due to real-time chat, even if the music is pre-recorded. Conversely, a lack of audience interaction or artist responsiveness might make a real-time performance feel like a replay. Yet, such discrepancies highlight the multifaceted nature of live experiences: pointing to the need to explore how audiences perceive time during a performance, whether they value synchronous interactions with fellow viewers or the artist, or how they might switch roles - from a passive viewer to an active participant. The dynamic interaction with time has the potential to unlock novel creative avenues for performers and designers and help them craft experiences that truly resonate with the audience's evolving definition of live. The sensory music box design idea capitalises on this by proposing a projected environment around the user which they can pause, walk around and replay enabling them to dissect the performance how they see fit and view it multiple times from multiple perspectives. On a more general level we encourage designers to explore and leverage this fluidity of liveness. They have the opportunity to craft experiences that, while not live in the conventional sense, still convey a sense of immediacy, connection, and shared presence. For example, designers could think about embracing the non-linear nature of digital experiences, offering viewers the ability to pause, rewind, or view asynchronously while still maintaining a sense of shared experience.

In in-person music events, the audience's journey is a rich narrative, extending beyond the performance – from the anticipation leading up to it, the rituals of preparation, the travel to the venue, and the shared moments with other audiences. Within the event itself, there are peak experiences[17, 29], the unexpected encores[61], and the quieter reflective moments. Our study revealed that while the convenience of remote events is appreciated, the absence of a journey-like experience results in events feeling less special. The Digital harmony hubs idea further showcased the importance of the event journey with the team focusing on other activities before, during and after the event (e.g., audience lobby, merchandise, eating) rather than the event itself. Participants argued that the importance of the activities surrounding an event are as important as the actual performance.

To recreate the depth and richness of live events in a remote setting, designers should think of the fan experience as an expansive journey, encompassing pre-event, live event, and post-event phases, each offering distinctive opportunities to deepen engagement and foster a sense of community [11]. In the pre-event phase, cultivating anticipation is crucial [62]. Strategies might include augmenting the multisensory aspects of the event by distributing tangible objects associated with it, or involving the target audience in a co-design process to collaboratively develop an event blueprint that can be shared in advance. During the live event phase, designers should focus on triggering peak moments that mirror the highs and reflective pauses of traditional event experiences by accounting for spontaneous interactions or unexpected surprises, coupled with spaces for individual reflection[34]. Following the live event, designers could think of including virtual lounges where attendees can interact, exclusive content releases that recount the highlights of the event, or interactive Q&A sessions with the artist.

The bond formed between the audience and performer is important, and studies have identified it as a significant factor not only for ensuring a pleasant experience but also for enticing audiences to return to future events[32, 41]. Our findings suggest that remote music audiences long for opportunities to interact during the performance and to co-create, wanting to have an impact on the music and the experience. The Concert connect design idea reflected this by proposing interactive set lists and collaborative lyrics writing. Nevertheless, during the discussion at the end of the workshop, one participant asked “But what about the artist? What about their vision?”.

Indeed, when designing technology for remote music performances it is imperative to recognise the diversity in artists’ preferences for audience interaction. While co-creation and audience participation are beneficial, there is also a risk of diluting the artist's original vision. Therefore, when designing technology for remote music performances it is important to ensure that the integration of audience interaction complements, rather than overshadows the artist's creative vision. Design solutions could offer varying levels of interactivity, allowing artists to choose how much they want to involve their audience in the creative process and how [33]. These levels can range from low-interaction formats, where the audience's role is mainly observational, to highly interactive formats, including real-time polls for song choices or collaborative creative processes. Furthermore, implementing mechanisms for feedback from both artists and audiences could help refine and evolve the level and type of interactivity offered, assisting in adapting the platform to align with evolving user needs and artistic preferences.

Our participants expressed that for them, remote music events are distinct opportunities for novel engagement – with the other audiences, the artist and the performance itself. Our findings indicate that technology is not just an enabler, but rather a driving force for delivering extraordinary and seemingly impossible experiences[59]. Moreover, for artists themselves, technology presents an opportunity to overcome the constraints of the physical environments (e.g., venue size, logistical challenges, or budget constraints). This transformation is particularly valuable for smaller artists or those performing from home, who may have previously been restricted in their ability to deliver visually captivating performances.

Despite this, our audiences also yearned for the emotional depth and authenticity of in-person events, frequently making direct comparisons and highlighting the limitations of technology in capturing the feeling of live music. Their perspectives underscore a critical observation: technology, for all its advancements, is still often perceived as intermediary – a lens through which we access experiences rather than a space where authentic, standalone experiences reside[42].

For designers, this brings a multifaceted challenge. While there is merit in curating technologically advanced and highly polished performances, there is also a need to embrace spontaneity and imperfection. As found in [41], authenticity and how “believable” and “real” a performance feels, plays a huge factor in audience engagement and satisfaction. Instead of replicating in-person experiences, designers should aim to craft novel, authentic digital experiences that celebrate these imperfections and spontaneous moments.

Our study's exploration into remote music experiences highlighted the need to redefine the role of physical space in enhancing audience engagement. Traditionally, music events serve as liminal spaces - thresholds between the ordinary and the extraordinary - where attendees transition from their everyday reality into a shared, heightened sensory world. Our participants noted that while remote settings (i.e., homes) offer a comfortable and cocooning environment for music enjoyment[3], they often lack the distinct boundary between the ordinary (e.g., home chores, distractions, daily routines) and the extraordinary (the music event) that in-person events provide.

The shift towards remote music experiences necessitates rethinking how audiences interact with their physical surroundings during a concert, perhaps focusing on transforming the audience's home into an integral component of the performance. The goal is to turn ordinary spaces, like bedrooms or living rooms, into sacred, immersive environments that transcend the typical associations these spaces might have with daily routines or distractions. Interestingly, three of the four design ideas proposed ways to transform this physical space either by fully changing it into a virtual space (Digital harmony hubs); by augmenting it (The sensory music box); or by using existing elements of the environment such as electronic appliances (Dancing home appliances series). By deploying advanced technologies such as augmented reality, designers can enable audiences to dramatically reshape their surroundings, turning the familiar into a dynamic backdrop for extraordinary musical encounters.

Our Phase 1 data showed a need of audiences for deeper immersion that engages multiple sensory modalities beyond traditional audio-visual formats. Moreover, during Phase 2, all four ideas generated by participants in our workshop included various elements of sensory immersion irrespective of the how might we question they were given to address. From a technological standpoint, the quest for deeper immersion suggested by participants can be met by creating innovative technology-mediated audience participation systems that go beyond mere screen interactions. This approach is supported by emerging research in mulsemedia[13], which explores the integration of touch, taste, and smell with sight and sound, thereby transforming the media consumption landscape. Designers are now challenged to leverage this multi-multi perspective, where multiple types of media engage multiple senses, to significantly enhance the quality of experience for audiences. This shift not only promises more engaging and immersive remote music experiences but also sets a new standard for how designers can create environments that fully captivate the audience's sensory faculties. For example, designers could consider integrating immersive lighting control with smart systems that dynamically adjust to the music's tempo or emotional tone, using interactive dance mats that allow users to interact with the performance, and incorporating scent diffusion systems that release fragrances aligned with the mood or theme of the music can add an olfactory dimension, making the event even more immersive and memorable, via systems like the ones described in [45, 55, 60].