Threshold – An idea proposal for interactive multimedia installation
Chengzhi Peng – Stephen Walker – Nadia Mounajjed, School of Architecture – University of Sheffield
We suggest building an interactive installation for the Leonardo international conference (summer 2006). The proposed artwork will act as a ‘threshold’ for the conference and will provide an interactive experience for the conference’ users/visitors. The artwork will add a digital/virtual layer over the real/material site and will provide a framework for creative exchange and productive interaction between the user’s bodies and the architectural context.
The installation work will be designed to capture aspects of interactivity, which can be later analysed to inform interaction design. We would like to bring in interactivity on the level of the body’s everyday experiences in architectural spaces. Depending on the venue of the final Leonardo International Conference, the chosen site will inspire and define the generating design ideas behind the artwork. Not only will the installation be site-specific, but it will also try to re-present/ re-interpret the functionality of an event; the work will question the ‘conferencing activity’ as an ephemeral event offering a unique experience. In conferences, people gather round a topic in a particular site to exchange and discuss ideas. After this, users may not meet up again—very rarely would the same experience be repeated and even in this case then probably in a different architectural or cultural context.
A possible application of the concepts suggested earlier can be explored by observing the users performance as they interact with their shadows in space or the acoustics of their footsteps, for example. We can develop an interface to transform the shadow of the users or to augment the echoes of moving bodies in space. Such work would stimulate interaction between the body and its shadow, reflections or echoes.
We are seeking collaboration from other members of the Leonardo network particularly on the interfacing aspect of the artwork. The previous proposal is mainly based on our research but it has also been inspired by the FACT discussion and is still open to other suggestions and revisions. However, we can generally provide design ideas and conceptual background for the artwork particularly in relation to real and interactive experiences of users in architectural spaces.
The Obstreperous Shadow
John Mateer, University of York
Imagine walking in a hallway with a bright lighting. Your shadow is cast on a wall to your left and all seems normal — the shadow moves as you do. Suddenly, the shadow seems to take on a life of its own, perhaps staggering and falling down drunk or racing ahead, turning back and thumbing its nose at you. You no longer control the shadow.
In actual fact you are not casting a shadow at all; a synthetic shadow is created using Video Augmented Environment technology. A camera captures your motion, feeding it to a computer which calculates your shadow profile and outputs it to a data projector which completes the effect.
Project Goals:
- to create a naturalistic presentation of a shadow that challenges people’s perceptions of reality
- to create a fun, surprising interactive installation
Please let me know if you need any additional information.
Social Interactive Ambient Soundscapes
Nick Bryan-Kinns, QMUL
This piece explores the tensions between interaction and ambience, and personal and social. Usually we think of ambient music as a personal, passive experience which we immerse ourselves in to relax, free of any interaction. This piece asks the question: what does it feel like when we are surrounded by an evolving, low tempo, relaxing soundscape which cocoons and envelopes us, yet we co-create this with others around us. The basic idea is that several people in a space contribute short musical motifs which are used in the generation of an evolving ambient soundscape and which can be edited by others in the space.
People in the space contribute musical motifs through some small artefacts which also provide feedback on others’ contributions to the music – the key is that the artefact acts not only as a way of capturing motifs, but also as a way of sensing and possibly changing others’ motifs. For example, people could use a tennis ball sized object to create motifs through their pattern of squeezing, and feedback on others’ contributions could be provided through heat, vibration, or light – illuminating when others contribute motifs, or even replicating their motifs through vibration. Or, people may contribute motifs by drawing on a small display which also showed others’ motifs and allowed them to be edited.
The motifs people contribute are collected in a central repository which uses them to generatively produce an ambient soundscape audio-visually projected across the whole physical space. When little or no contributions are present, the repository will evolve the soundscape based on previous contributions – in this way themes will re-emerge from previous interaction.
It is envisaged that the piece would be installed in a social space such as a café or bar where a range of people come and go in informal, ad-hoc ways. The piece would then raise questions about how people manage their social interaction through the musical space, what new forms of interaction emerge, how identity is managed and whether identity of contribution is perceptible in the evolving soundscape, the relationship between
Slow time
Alan Dix – Lancaster University
This is partly a proposal for potential installations that explore our own perceptions of time, partly a set of lenses that can be applied to other pieces either post hoc or as an intimate part of their design. Issues of time have long been a part of research in human computer interaction and also permeate other aspects of life from timelines of sport to artistic retrospectives.
In a Newtonian/Cartesian framework time is the matrix in which we live and act, in an Einsteinian/relativistic framework time is our own threads tying the fabric of reality through our constant crossings and re-crossings at our shared moments of here and now.
We live in a world of CD sampling at kilohertz (KHz), radio waves at megahertz (MHz) and PCs running at gigahertz (GHz). In the blink of an eye the computer clock counts to a billion and 97.6 million waves of Radio 1 pass through my body; don’t they tickle so?
Counter to this world of speed and imperceptibly fast motion, this proposal is to investigate slow time including the imperceptibly slow.
We move our hands and eyes and walk in timescales of seconds indeed foot bridge designers try to avoid resonance at 1Hz, the average walking pace … unfortunately the Millennium Thames Bridge designer forgot that the side to side rocking as we walk happens every other pace 1/2 Hz.
When we sit down to do something or eat a meal, we may be stationary for 20 minutes, or an hour – a few thousand seconds – a millihertz phenomena. And the life of man is three score years and ten, a mere two and a third billion seconds; like most buildings our time on earth is a nanohertz phenomena. Pushing back beyond our lifetimes, the oldest buildings stretch a mere few hundred times longer and human kind itself back 60 thousand years – taking us into the picohertz, whilst dinosaurs and younger rock strata ripple in the femtohertz. Finally the earth itself and the very universe pass in a few billions of years – a blip upon the cosmic silence in the attohertz band.
More at: http://www.hcibook.com/alan/projects/slow-time/
Framework for Interactive Art
Wibke Hott, FACT,
David England, John Wai, LMU
The aim of our proposal is to provide a place or framework where interactive art can take place and be discussed and debated.
It should be a place where the tensions and similarities between interactive art, HCI and design can be explored.
It will be web-based to allow the uploading of thoughts, materials, annotations and ideas.
The website will support debate and discussion about interactive artworks (either provided or linked to) via
• Having Games and battles with avatars
• Taking sides and roles in debates
• Voting and judging on artworks
• History of ideas competing between the different disciplines
An artist will be invited to derive the concept for the framework and respond to LeonardoNet node members’ needs in the design. The framework should evolve with the network. Other partners are invited to shadow the dialogues and discussions on the website for meta-analysis. Software from existing sites, e.g. www.hci-fun.org.uk and www.culturecastles.co.uk will be re-used to form the basis of the new website.
Initial ideas for the website include
• A re-mixing place where existing ideas from HCI, art and design are presented as a historical matrix of examples which can be used to generate a new digital piece.
• An artscape – derived from voting and linking on artworks which presents a visualisation of selected artworks which have been categorised by visitors along a number of different dimensions.
LeonardoNet members are invited to suggest other facilities they wish to see on the site. The central aim should be to promote the aims of the Network in an interactive and collaborative way.
Creating collaborative hyper-physical interfaces for communal engagement (POI)
Jennifer G. Sheridan Lancaster University
Alan Dix Lancaster University
Nicolas Villar Lancaster University
Nick Bryan-Kinns Queen Mary, University of London
Alice Bayliss University of Leeds
Matt Cooper The Mob Lancaster
Pete B Planet Angel London
We propose to develop hyper-physical interfaces for collaborative and communal engagement particularly for “playful arenas†and experimental, real-world environments such as nightclubs. Our work will build on our previous investigation of how to take a traditional set of poi that clubbers often dance with in trance and techno clubs and augment them with sensor technology in order to allow the shapes, patterns and movements made by the participant to become visualised elsewhere in the club space. We will create a collaborative performance piece using these interfaces and we will perform this piece “in the wildâ€. Our ultimate goal is to allow anyone with a willingness to interact to be able to create their own DIY performance using these interfaces.
This project requires expertise in the areas of HCI, performance studies, music technology, computer science, graphic design and CSCW as well as a pool of skilled poi users, VJs and clubgoers. We believe that we have found this network of people through Leonardo. Because of the nature of this work, we expect that our findings will extend beyond the use of poi to include collaborative physical music and visual interfaces in general.
Phones in the post
The opportunity to collaborate in a project that combines moments from the lives of every node in an aural (and possibly visual) artwork. It will involve a mobile phone, a lot of envelopes and an element of surprise. For this reason, not a lot is being said here, but it will be fun and despite this cryptic introduction (which was much the one given at the workshop in Liverpool), several node members have signed up to take part. Join us.
The Literary Fridge
Mark Blythe, Suresh Mandahar, University of York
Joe Cutting,
Marie Jefsioutine, University of Central England
The smart fridge in the Centre for Usable Home Technology’s Responsive Home is currently configured so that Alex, a talking gesturing character helps pick recipes appropriate for the fridge inside it. A touch screen allows the user to interact and access information. This kind of application has been criticized as a rationalization of the kitchen and work practices within it. This project would aim to make a smart fridge that would focus not on tasks but art and entertainment. We would like to make a literary fridge which can – read a book of the week and enable users to make digital fridge poetry.
There are many commercially available text to voice simulations with voices that are easy to listen to similar to the one that Alex uses. Licenses to use them are however expensive. It would be interesting to explore the possibility of setting a free text to voice system to read from one of the free online libraries. The Gutenberg project makes almost every classical literary work freely available. If the fridge could be set to read a particular novel then it could make it into a free audio books. Free text to voice systems are not as attractive to listen to as Alex but they may be tolerable and the investigating would be an interesting proof of concept. This might be particularly useful to the blind and older people with failing eyesight.
Fridge poetry kits, have been popular for the last couple of years. There are various sets containing differing vocabularies – romantic, Shakespearean, erotic etc…. There has been a lot of work done in artificial intelligence on machine poetry and these usually involve the development of algorithms to choose words or phrases from particular databases. This would not be necessary for the development of a fridge poetry kit. Again, free online libraries could be used to access out of copyright literature. In the nineteen sixties William Burroughs pioneered the “cut up†technique. He would take clippings from newspapers, novels, his own journals and manuscripts and assemble them at random in order to create new fragmented texts. If a number of different online sources could be searched for particular words the cut up technique could be used to generate new texts. If the user could determine which texts were to be sampled from this might form the basis for a compelling and fun interaction. The poem below resulted from word searches in online collections of the works of Shakespeare, Byron and Shelly for the word egg:
To pay him a fresh visit, with a dish
For breakfast, of eggs coffee and fish.
Exposing what is mortal and unsure
To all that fortune, death and danger dare,
Even for an egg shell.
But what am I about?
If my grandmother sucks eggs, was it I who taught her?
Shelley was placed last although, Byron was searched first, because it read better that way. If a user could edit the results of searches in this way they could produce fridge poetry that they might want to display on the fridge and dedicate to family members.
Proposal for a community audio-object installation
David Frohlich, University of Surrey
Mark Blythe & John Robinson, University of York
Stephen Barr, The Govan Initiative
The audio-photo desk recognises printed photographs that are placed on it and plays associated sound files of commentary, ambient sound or music (Frohlich, Clancy, Robinson & Costanzo 2004). By creating audioprints as different elements of a story, users can create different versions of the story in the performance of an interactive narrative. Sian Lindley and Andrew Monk have conducted a number of user studies with the desk and noted that there is fun to be had in doing this for sharing personal photographic material. They call this activity DJing with photos (Lindley & Monk 2004).
This work might be extended to community storytelling, by creating a public work of art based on oral history materials. If the material related to the history of some community location or activity, the desk might provide a novel and compelling way of reviewing and playing back oral history fragments, and engaging the local community through a public installation. With a small modification to the existing software, printed photographs could be supplemented with physical objects having associated sounds, so that historic artefacts could also be used in the playback or performance. This would create audio objects as well as audio-photos. Members of the public could approach the working installation to play and mix various images, objects and sounds as they pleased, creating new mixes in conversation with each other. The conversation itself might provoke new memories and stories to be told by the audience. These could themselves be captured by the system, resulting in new material for later exhibition.
A context for this exploration has been offered by the Govern Initiative in Glasgow. They commission re-generation projects and are interested in capturing stories of Glasgow life and stimulating community activities and conversation. In partnership with Hewlett Packard they are setting up community centres equipped with state of the art computing facilities, and these could be used to create oral history materials with local people. The project would focus on collecting artefacts, sounds, music and stories relating to the Fairfield Shipyard in Glasgow, to whom many people were connected in the past. A multimedia artist would be hired to gather these materials and assemble them in a form which is suitable for the audiophoto desk. Hewlett Packard have also agreed to donate a new PC and webcam for the audio object installation itself. This could be exhibited in a new Digital Media Centre about to open in the city.
Frohlich D.M., Clancy T., Robinson J. & Costanzo E. (2004) The audiophoto desk. Proceedings of 2AD: Second International Conference on Appliance Design (p139).
Lindley S.E. & Monk A. (2004). DJing with photos. Proceedings of HCI 2004 (2), 178-180. Leeds: Research Press.
“You’ve got a lovely smileâ€: a compliment kiosk
Joe Cutting
Basic Idea
Concepts behind the idea
• All media have constraints. How do we feel if a media constrains us to behave in certain ways?
• The emotional power of simple communications like text messages and email.
Can we artificially engineer these feelings?
So we could…
• build a kiosk
• which can be placed in bars, arts festivals and other public spaces
Rough Outline
1) The user is presented with a selection of pictures of previous users of the kiosk and asked to choose one person to send a compliment to.
2) They then choose what they would like to make a compliment about
3) And then choose an exact compliment to send
4) The kiosk then sends the compliment as a text message to the chosen person
NEW MESSAGE
Someone in the FACT
bar thinks you have
beautiful hair
(Note for safety reasons the message is not traceable back to an original sender)
5) The user then has their picture taken by a camera attached to the kiosk
6) They then enter their mobile number which for safety reasons is never revealed to other users. Their picture is then added to the initial choosing screen.
Possibilities and Options
The idea outlined above should really be seen as just a starting point. After discussion with other members of the network here are some directions that the project might move in.
Reactions
One of the biggest questions in a unconventional piece like this is “How do people react to it?†It would be very interesting to do some research on this. Although this could be done by observation the kiosk also stores the phone numbers of all users so a simple study could be done just by ringing people up.
Conversation
I’m told that there’s a large body of work on “compliment theory†which maintains that a real compliment is a two way process with the response being as important as the original act. We could look at whether it is possible for people to respond to their compliment. At the very least we could collect any text message response they make to it on the kiosk.
Random vs Predictable
In conversation with other group members there seemed to be two different schools of thought regarding whether we should try and make is kiosk more or less random and unpredictable. The first school favoured ideas such as searching the internet for random people’s details and sending them a compliment or sending a different message from the one that was selected. The other school felt that although we are surrounded by supposedly rational and predictable technology such as computers and email the actual experience of these things is random and unpredictable. They felt that it would be good to try and make something, like a telephone, where although users don’t understand the technology they do understand the complete functionality and there are few surprises.
Internet
The possibility of putting something like this on the internet was discussed. This is technically possible but would change the nature of the piece considerably. We could reach a greater number of people but the knowledge that someone in the immediate vicinity had sent the message would be lost. There may also be safety concerns and difficulty stopping people putting up “dodgy†pictures of themselves….
Opportunities for help and collaboration
At the moment this project is looking for collaborators who can help with the practical aspects of making it happen. Here are some of the areas where help would be appreciated
Graphics and 3D Design
The physical kiosk needs to be designed and built. Similarly the software needs to be designed both from a graphical and interaction perspective.
HCI
This is a novel system that users will have to operate unaided. There are a number of HCI challenges in looking at how users understand what the system will do and how to encourage them to use it.
Language and Poetry
Although the system will not let users enter free text compliments this does not mean that the compliments themselves have to be like the ones in the example screens. It would be great to have something more creative and poetical.
Social Studies
As has been mentioned there is a large body of work looking at compliments and responses. There is a lot of scope of studies and research into how people react to a system like this and how the system can be engineered to produce particular results.
Art and publicity
It is envisaged that this system will be installed in art festivals and other public spaces. If you know of an event or space which it could be installed in and could help with practicalities this would be very useful.
Technical
I am able to handle the technical issues of the kiosk myself but if further technical help were available we may be able to look at extending the functionality of the basic system or even producing several different versions. If anyone has technical equipment they can donate this would be very helpful and reduce the budget.
‘IDE Art’ (Integrated Development Environment Art)
Proposed by Chris Newell and Stuart Andrews, University of Hull
Supported by Janet Finlay, LMU
We propose to engineer a small series of 3 stand-alone audio-visual exhibits.
The work will be presented as looped soundtracks to accompany 3 original artworks in frames. These original works will be output onto canvas to be hung on a wall or on a display board at a conference. A sound system will be hidden behind the canvas and the sound will play as spectators approach the piece. The first impression will be the familiar image of the computer interface (Windows) in the curious context of an art gallery and on canvas. On closer examination it will be revealed that the interface design (produced in a Microsoft interface design tool, hence the title) is a non functional commentary on the process captured on the soundtrack. The soundtrack is based on conversations within the Leonardo workshop groups and nodes. The extent to which these conversations might be ordered, edited and captured will be explored during the project.
Of most significance is that the work should be eye and ear catching and should stimulate new conversations in a workshop or conference poster setting.
The concept is to freeze action-led, dynamic, interactive processes and to represent these as a static display in its own right. This is reflected in the captured conversations and the captured screen, both of which are effectively secured in the frame that surrounds them. It may be that in drawing directly on the experience of several nodal groups, this project may contribute to a ‘brand’ that marks aspirations and frustrations which are being addressed within Leonardo-net.
Crossing Boundaries
The work is designed to act as a waving flag to draw in users and inspire comment and perhaps unease. The premise of using an engineering tool to produce art and propaganda may lead to new negotiations of understanding. Specifically, the artwork is intended to unsettle any secure definitions of place and so will provide a marginal space in which to consider opportunities for collaboration. This will create an opportunity to explore the layers of collaboration that can exist within such a boundary space and the ways in which such layers can begin to combine together.
The work will raise a series of issues for reflection and these may include the following:
• By existing through both sounds and images, the artwork will sit between more conventional forms of display and this sense of juxtaposition will increase as visitors approach the work.
• Passers-by may begin to take up a position of being spectators and perhaps even members of an audience (Fiske), before moving away and perhaps returning to more secure territory.
• The piece will invite spectators to become aware of listening and observing space and, further, of their contribution to making and writing space.
• The piece will encourage spectators to recognise that writing space marks a commitment to and an interpretation of the work. This may lead spectators to reflect
• on the work and to begin to identify new spaces for connections and debate that contribute to the audio context for the work.
• As more spectators arrive, it may become apparent that earlier spectators have become part of the space of the work. Although the work may begin as stand alone elements, by the end of an installation, it may be far more difficult to draw any clear lines of separation between the artwork and conversations in the conference/workshop at which it is displayed.
• Early investigations will involve exploring the borders of the artwork itself and techniques of disrupting everyday life as part of the work itself. The project involves a suspension of space and elements of space. This will create opportunities to suspend apparent distinctions between science/art, object/art and sound/vision.
Collaboration
In the last two years, there have been an increasing number of conversations across subject boundaries in the Scarborough School of Arts (SSA). These have related both to co-teaching of modules and to research and practice, specifically through the School of Arts Seminar Series. These connections have begun to address the benefits and the difficulties of working on the borders of subject specialisms. The Leonardo Net Proposal offers an opportunity to develop this investigation of conversation with colleagues in other institutions. Indeed, the project itself will enable conversations from this collaboration to be replayed and reformed in the display of the final artwork.
We would hope to address multiple themes and conversations emergent within the Leonardo group.
1. Conversation and its relationship to the process of designing an engineered artefact.
2. Confronting the popular notion that art is about galleries and frames and that the computer interface is about ‘Windows’ and frames. The computer interface as organised rectangles/art as organised rectangles
3. The computer interface as art divorced from function
4. The computer interface as agitprop. We may need to be rude and rail against the rectangle.
5. Issues relating to capturing the design process (through the use of audio recordings)
The chair project
Sal Fiore, University of Wolverhampton
The project began with an exploration into blind experiences through reading of autobiographical narratives. This marked a first phase of appropriation where I sought to better understand something about blind experience. I then sought to interpret my new understanding in a short fictional narrative and it is around this second phase of appropriation that the idea of the interactive chair emerged. It emerged, not through the identification of a need for such a chair, but because the idea of it provoked many fascinating questions. For instance, I wondered about the ambiguity of such a ‘lively’ object for both blind and sighted people, what sort of meanings this otherwise mundane and everyday yet potentially highly symbolic object would have for people and how interacting with such an object could challenge our experiences of a place and affect our interactions. For example, I was intrigued by the idea that the chair could make ‘invisible’ people evident for a blind person by signalling their approach in a secret way and curious about the changing meanings such an object could embody if installed in different places and contexts.
From this point, a third phase of appropriation took place. This time, Mikey Ball, an amateur sketch artist based in Birmingham, took my story and appropriated it into a storyboard. He realised the characters, objects and interactions in imagery and gave his own interpretation to the experiences represented. When he presented this storyboard to me, it changed my own understanding of the story and the chair and enabled me to construct new meanings around the object. These phases have already raised interest with papers presented at HOIT 2005 and Include 2005.
The next step in this project is to begin addressing how the chair might look, feel and sound (the formal characteristics). There is no technological determinism here, but instead an emphasis on opening possibilities with respect to the contributions already brought by others, as in the previous phases of appropriation.
Relevance to Leonardo.Net
I hope with this project to progress through further stages of appropriation to realise the construction and installation of the chair. The object as yet has no fixed form or technical engineering. These two phases will need to take place through appropriation by other individuals who would like to bring their specialised interpretation to the making of the chair (e.g. graphic design, modelling, electronics). There is freedom in this endeavour: the only requirement is a respect towards the appropriations made before. Each phase should in this way (as with the phases before) represent a new opportunity to re examine our own understandings of the subject-matter, where each appropriation is unique and an opportunity for reflection by both the designer and others involved in the process, to find new meanings and possibilities within the emerging objects.
The process of designing the chair will represent the first half of the project. The second half will be realised when the chair is installed and interacted with. The purpose of this installation will not be to demonstrate the ‘artwork’ but to create conditions for people (blind and sighted) to experience and appropriate it.
The prospect of the installation invokes further routes of enquiry: How will the vibrating chair make people feel? How will the chair’s meaning change in different places for different people? What new stories will be created? How will people appropriate the chair? Will we better understand blind experience through making this object real? Installing the chair, however, would not be just about trying to answer some of these questions, as much as to open up new questions. We would hope to suggest a possibility for understanding the ambiguity of invisible things and ultimately better understand experiences of designing in a pragmatic way.
To sign up for any idea, please contact the first name on the list associated with the idea, our administrator. See below for more information on the artwork criteria and other related artwork postings.