This page contains some examples of the interaction design, HCI, and other kinds of design projects
I've been involved with over the years; 2004-2008 as Research Director at Umeå Institute of Design
and from 2009-2013 as Studio Director at Interactive Institute Swedish ICT Umeå. I struggle to
keep this page updated, and there are some projects that can't be shown publicly due to IPR.
Interaction Design | The ABB Eye-Catcher | 2013
The brief for this project was: what might human-machine-interaction in the processing industry look like in 5 to 10 years? How would an operator then interact with a multi-screen remote operations environment? Are there alternatives to the traditional screen, keyboard and mouse?
In this fully functional prototype, we combine the use of gesture recognition and eye tracking to allow the users to interact, using physical gestures, with whatever they are looking at. A Microsoft Kinect is used to sense the users' gestures. The technology uses a 3D camera that detects 3D gestures and makes it possible to steer objects on the screen. Tobii Technology's eye tracking technology enable users to interact with computers using their eye gaze.
The prototype consists of two flat screens, placed a couple of meters in front of the operator. On the right-hand screen, a 3D representation of an oil rig is shown. By swiping vertically with the arm, the operator can navigate through different levels of the oil rig model. Instead of using a mouse to highlight objects and make them clickable on the screen, the operator uses his or her eyes. When an object is selected, it is moved by a swipe gesture to the screen on the left. Here, in the process view, the operator can interact more in detail with a specific object, e.g. a compressor, and see how it performs. Via the eye tracking, menu items can also be expanded to reveal more information.
Interaction Design | The Voice Harvester | 2012-2013
The Voice Harvester is an exploratory interactive installation that embodies human voice in physical materials. Sound input is amplified and transmitted through audio drivers connected to a thin, flexible membrane that agitates the material on it. The title “Voice Harvester” was derived from the fact that the installation create was able to elicit nonlinguistic, expressive, and naturalistic human vocal sounds to explore the full range of capability of the human voice through use of a novel, playful, and embodied interaction.
This design exploration took place by a team at Interactive Institute Umeå under the guidance of Artist and Composer Anders Lind. The concepts of human engagement, involvement, and embodiment as well as the creation of a physical tangible thing were central to this design exploration.
The object elicits curiosity and subsequent interaction utilizing the unusual appearance and leveraging tacit knowledge within users of the purpose of microphones. When a user speaks (or makes any sound) into the microphone they will see the materials in the acrylic tubes animate with the physical embodiment of their voice. The intent was that once this happens users will continue to engage and interact with the Voice Harvester to see the different ways in which they are able to manipulate the materials into action.
The Voice Harvester is built as part of a series of interactive installations to be shown in Umeå, Sweden as part of the Umeå 2014 Capital of Culture and has already been exhibited at the MADE festival 2013.
Ballade of Women is an interactive installation that explores perspectives on women’s rights. Through the lens of the personal experience of three women, the exhibition offers an interactive narration on three fundamental themes: emancipation, self-determination and violence.
The starting points of the exhibition are three paintings of the private collection of the Fondazione Monte dei Paschi di Siena. They illustrate three historical characters: Cleopatra (by Marco Pino, XVI century) who supports the theme of emancipation, Maria Maddalena (by Rutilio Manetti, XVII century) who embodies the theme of self-determination, and Lucia from Siracusa (by Maestro dell’Osservanza, XV century) who recalls the them of violence.
The installation is a dynamic space constituted by floating and mobile fragments, whose behaviour is influenced by the physical presence of the observer as well as by the virtual presence of online discussion groups debating the themes of the exhibition. When the screens are positioned in a fragmented way, they display media coming from online groups, discussing issues addressed in the exhibition. Software continuously monitors news related to the treated themes and how these are addressed through posts and online discussions. The number of threads created from the internet is an input for the installation and influences the speed of movements and informative content. The soundscape allows the viewer to pick up fragments of the play of a selection of poems, related to the three themes. Verses are whispered and vanish, immediately after being heard. The dynamics of the fragments redefines the contours of the three paintings from specific viewpoints in the space.
The experience suggests that each of us can contribute to compose a harmonious picture of the complex and controversial world of women's rights, by approaching it, and by being confronted with points of view of other people, facing the same topics from different perspectives all over the world. Our presence and our ideas can change the world.
this project was carried out at Interactive Institute Swedish ICT with University of Siena, Eindhoven University of Technology, Fondazione Monte dei Paschi di Siena and Vernice Progetti Culturali.
(Team: me, Ambra Trotto, Nigel Papworth, Björn Yttergren, Jeroen Peeters)
Interaction Design | Sketching in Interaction Design Using Stop Motion Animation | 2011
Widely acknowledged as an archetypal design activity, sketching is typically carried out using little more than pen and paper. Today’s designed artifacts however, are often given qualities that are hard to capture with traditional means of sketching. While pen and paper sketching catches the character of a building, it may not equally well capture how that building changes with the seasons, how people pass through it, how the light moves in between its rooms from sunrise to dawn, and how its façade subtly decays over centuries. Yet, it is often exactly these dynamic and interactive aspects that are emphasized in contemporary design work. So is there a way for designers to be able to sketch also these dynamic processes?
Over several years and in different design disciplines, we have been exploring the potential of stop motion animation (SMA) to serve this purpose. SMA is a basic form of animation typically applied to make physical objects appear to be alive.
The animator moves objects in small increments between individually photographed frames. When the photographs are combined and played back in continuous sequence, the illusion of movement is created. Although SMA has a long history in filmmaking, the animation technique has received scarce attention in most design fields including product design, architecture, and interaction design.
In this project we applied stop motion animation to sketch new functionality for ABB's 800xA industrial control system process and automation overview and control. 800xA is an old, legacy system that has evolved over the years with neither user experience nor a strong notion of usability in mind. This has resulted in systems that are difficult to learn, especially for novel users. Often, there are many ways of achieving the same result, but users tend to get stuck in one way of using the system, even though there might be better, more efficient ways of doing what they do. It is also not uncommon that users in industrial environments might be afraid or feel ill-at-ease with exploring and testing new functionality, as even small mistakes may have huge effects on the factory. Users also experience automation and control systems as difficult to navigate in, to know where they are in the system, and to zoom in and out and return to a previous position. To deal with some of these problems, we collaborated with ABB Corporate Research to bring forth new concepts and tools to appear in future versions of 800xA.
Interaction Design | SCA Complex Process Industry Overview, 3D Multitouch | 2010-2011
In this project, we designed and implemented a fully functional large, multitouch screen interface that interface with several existing industrial computer systems (such as SAP, ProTAK, WinMOPS, and other) and incorporate it in a seamless, fluid 3D zoomable interface. The system provides real-time feedback about the factory's entire process on a single monitor using a slick, easy-to-use, and quick and responsive interface. The project is the result of a long series of creative workshops with employees at SCA Packaging and ABB Corporate Research.
The system provides two modes of operation: first, an 'ambient mode' in which a 3D representation of the factory provides a quick and easy-to-grasp overview of the status of the factory as a whole, i.e. a dynamic source of information about the status of the process throughout the day that subtly informs everyone related to the process—from operators and service technicians to administration and management—about the overall ‘health’ of their process industry in real time.
Second, the system also allows users to step up to it and interact with it, using multitouch interaction. Here, the user is able to zoom into the factory's 27 different departments to view more specific information about what is going on in there.
At the core of the design is a 3D model of the paper mill facility. By means of tapping into the company’s production management system, status information about the process is presented in the rectangular areas connected to the various departments of the facility. Also, at the lower end of the display, the key values mentioned above are presented.
The interactive 3D-model is primarily not a model of the facility, but rather an information display about the facility. It is designed to resemble the facility to such an extent that it allows for expressing the spatial logics of the process. We have not opted for a photorealistic representation, but rather aimed for a level of realism that speaks to the ability on part of the employees to recognize different parts of the facility and the role they play in the overall milling process. Apart from the overview of the factory, users are able to zoom into any of the 27 departments to get a more detailed view of that specific department.
Relax. Take a deep breath. Spread your wings and fly like a bird. How quickly you can reach the goal? You stand in the reception hall in Skellefteå Airport, in front of an 82 inch touch screen with an ultra-modern 3D camera that lets
you control characters called Wood Bots. A breathtaking game experience in which your body's movements are used to navigate the gaming environment, but its more than a pastime for those who are waiting for departure or arrival. Much more. The installation of Skellefteå Airport is the result of a research project that ties together a number of industries and businesses in the Skellefteå innovation-driven region.
The Woodbot Pilots Game was a project born out of collaboration between numerous companies in Northern Sweden (Skellefteå Kraft, North Kingdom, Interactive Institute, Optronic and Skellefteå Airport) that wanted create a public installation to showcase the capabilities of a new 3D camera made expressly for industrial environments.
The 3D Optronic camera (released prior to Microsoft's Kinect platform) interprets your movements and lets you control the Woodbot character you have chosen to fly with.
Interaction Design | MITE: Mobile Information Technology Environments | 2005-2007
Since 1999, we have worked together with ABB Corporate Research, the R&D division of one of the world’s largest industrial engineering companies, in the area of interaction design for large-scale multiuser displays and personal mobile devices. A series of prototypes have come out of this collaboration; all researched, designed, and implemented to be used by service technicians and operators in various kinds of highly automated industries. These systems have not been ideated, designed, and implemented as individual, one-off systems, but rather to become part of an existing ecosystem, what we call ‘Mobile Information Technology Environments’. The vision has been that mobile devices and applications should be primarily designed to work together with—rather than replace—stationary information technology.
This particular interaction design research project was carried out between 2005-2007, funded by ABB Corporate Research, the Swedish Research Grants organization Vinnova, and the mining company New Boliden AB.
The objective of the project was to design and implement a functional experience prototype of a future control room system, based on ABB’s 800xA, specifically targeted for one of ABB’s customers: New Boliden AB.
We designed and implemented this prototype as a large, multitouch screen which automatically connects and extends to the mobile device (a pda -- remember, this was 2005!)
of any users standing in front of the screen.
the brief was to provide ideas, construct prototypes, and lay down general principles for the design of train information terminals
accessible for the widest target group possible.
our primary objective throughout the project, from our pre-study to the finalized, fully functional prototype, has been to allow people
with various kinds of disabilities access to the service of public transport, following the ideals of inclusive design.
throughout our user-centered design process, we have worked in close collaboration with a number of disability organizations and real
users have been active in all phases of the projects. the resulting prototype has also been subject to thorough testing and evaluation
in situ at orebro train station.
we are quite excited that no later than 2010, the results of this project will be found in all train stations throughout sweden. this is a project
carried out in umea design research group.
(team: me, linda bogren, catharina henje, fredrik nilbrink)
Interaction Design, Service Design, Inclusive Design | Audio Index | 2007
photo: fredrik nilsson
the audioindex system gives disabled library visitors the possibility to independently browse for books, get guidance, and access
various kinds of information.
the system allows visually impaired users to browse for audio books by pointing directly with their index fingers on the spine
of the audio book as it stands in the shelf. the system then recognizes which audio book it is and quickly provides the user with
information about the author, the title, and a summary of the contents of that particular book in the form of synthesized speech.
users are also able to point at other objects in the library, such as bookshelf, to find out what kinds of books are to be found in that particular shelf.
the fully functinoal audioindex prototype system is partly the result of an european union-funded project that focused on accessibility to libraries for all visitors,
run by the public libraries in the umea region. the main target group has been visually impaired visitors and the entire project was
carried out in close collaboration with end users and disability organizations. this prototype has been subject to long-term public evaluation at umea city library,
and is now a commercial product
in this project, we designed a collaborative interface for highly automated, industrial environments.
the resulting system, the abb powerwall, consists of large, shared interactive displays and
several personal mobile information technology devices. on-site service technicians can seamlessly move information back and forth from their
mobile devices to the shared display. the system supports various kinds of collaborative work, including making annotations;
browsing for information; and visualizing blueprints and three-dimensional representations of objects and torrents.
the design vision has been to provide end users with an unobtrusive way of sharing information, discussing problems and
issues with others in front of a large collaborative screen, and the chance of socializing and learning from each other.
located strategically in the specific environment for which it has been designed, the abb powerwall is intended to become a
natural gathering point that increases interaction, afford gathering, discussions, collaboration, small talk,
socializing, and community-making.
Photography | Phenomenology of Mobile Interaction | 2003
for a project called phenomenology of mobile interaction, i needed some product shots of a couple of
mobile information technology devices to go with a lot of text. rather than simply paying someone with a reputation
in this area for the real stuff, i saw this as a good opportunity to do some amateur photography on my own and to get to know our
camera better. it's an olympus camedia e-10.
the lens it has cannot really cope with close-ups like this, i
guess, but it was sort of a fun experiment anyway. other than the camera, i used two very bright stage-style
lights and a poor man's studio background setup consisting of a couple of large sheets of white paper. it was all done on a regular
office desk. to get the blur-into-horizon effect, the aperture was set to as low as you can get with this camera: 2,2.
at the time, I thought the series of grainy images with their narrow focus were quite cool.
this is a project in which we needed to design a physical prototype to be able to implement and test
a new interaction style. information about the interaction style can be found elsewhere.
the physical design had some initial requirements.
first, it needed to be a casing to host various circuitry and batteries.
second, it needed to be shaped in such a way that when lying on for instance a table,
the pda's screen should be facing upwards while an optical mouse sensor should be
facing downwards. third, we wanted the prototype to be appealing and useful for its interactional purpose.
in this, we assumed that as the prototype would not incorporate any additional buttons,
knobs, or any other interactional means other than the downwards-facing sensor, the physical
design would benefit from being small and anonymous; not aiming to be the center of attention.
the actual physical design, using a minimal budget, was carried out by me during the winter break of 2002-2003.
a standard compaq expansion jacket,
originally intended to allow two pc-cards to be attached to the ipaq,
was taken to pieces. its plastic casing was then used as the basis for the physical design.
two custom made electronics circuits, including an infrared beamer, the circuitry containing the
optical mouse sensor, and four battery holders were then attached to piece of plastic panel.
the original flipside of the expansion jacket was cut off and replaced by this plastic panel.
To connect the infrared beamer with the pda, a small vacuum formed plastic hood was designed
and attached to the top of the prototype. this hood, other than aesthetically somwhat pleasant, holds
a small mirror which reflects the infrared beamer's signal into the pda's infrared eye.
finally, the whole of the physical prototype was painted.
(team: me, mikael wiberg, andreas lund, bjorn yttergren)
Graphical User Interface, Interaction Design | The ABB Mobile Service Technican | 2001-2003
here's a screenshot of the graphical user interface i did for
a research project i was conducting in collaboration with
swedish industrial company abb (see above). the user scrolls between these two
screens by tilt. all selection is done by finger-tapping the screen, hence the seemingly
oversized controls. the gui is intended to look good on a compaq ipaq screen, because of which the colors
look a bit odd on most other screens.
this gui was made in photoshop, and consists, other than the text, of a
number of bmp images (with one color representing transparency). the font is
myriad pro (regular, italics, bold italics).
the gui was implemented using ms embedded visual c++ 3.0
Interaction Design | The Reality Helmet | 2002
the reality helmet is a wearable device providing a novel form of interactive experience, in which the user's vision and
hearing is completely shielded off from the world.
video and sounds are sampled by the reality helmet from the
surrounding environment, but through computer processing sounds are presented to the wearer as vision and sights are
turned into a soundscape. the result is a radical transformation of the nature of being in the world (pretentiously put, i agree); an unusual,
first-person experience of artificial synaesthesia.
the reality helmet leads its wearer to question the relationship
between what is out there and what is sensed; it is a very unusual kind of experience.
(team: me, staffan eriksson, kalle jalkanen, john waterworth, jonas westling)
Poster Design | The Penguin: Spell by the Web | 2002
this is the poster i made for the chi 2002 conference, as a
presentation of the penguin application (below).
the poster featured cut-out penguins (see
of the poster in action at the conference for reference) but which are missing on the screenshot (i'll fix it soon). the penguins
produce a neat 3d effect in rl. the pdf files from the
indesign original are available
here (for the screen optimized version)
and here (for the 9 mb full-size version).
as noted, the poster was made in indesign 2,
while the pictures, which are screenshots, where converted to cmyk and edited in photoshop.
the poster text is based on myriad pro (regular, italics, bold, bold italics),
but there are some instances of myriad wild family as well: for the
headings and the preamble i used
myriad tilt, while the black outline of the large 'the penguin' heading is
the poster was printed at the local printshop nra. the 3d penguins where cut out
from the printout (see pdf layout for reference) and glued onto cardboard, which where then in turn glued onto yet another small piece
of cardboard before we attached the whole thing to the poster using two-sided sticky tape.
Graphical User Interface | The Penguin: Spell by the Web | 2001-2002
this is a ms visual c++ application that i designed and implemented during 2001. if you want to know more about this project,
it has its own homepage. the gui comes in three different flavors; the
the settings mode,
and the tray mode.
i feel somewhat content with the penguins and the iceberg in this design. the former move around on the iceberg according to query results.
Web Design | Dept. of Informatics, Umeå University | 1999
i redesigned a new look and feel for the department's homepage in 1999-2000. i could do pretty much whatever i wanted, but on the other hand,
it had to pass people that were still questioning the use of images on web pages, referring to all the 14.4-ers out there. so, in retrospect,
it is possible to see some 'restrained design choices'.
anyway, the whole site is still highly active -- and probably will for some time. the image on the front page is a random pick, which was
about as much fun i was allowed to have. i think that the color scheme is not too outdated yet, even by web standards.
the graphics for this site was almost entirely done in a previous version of
macromedia fireworks. the font is gill sans (roman, light, bold italics).
Web Design | Greenland Whale Fisheries (Band) | 1995-1996
although it is clear that this design, not to mention the whole structure and idea of the site, wouldn't quite pass by today's standards,
the frames (all over), the menu as an image map (left), the animated gifs (in blue, which the stills spare you from), and the guestbook (see picture 3)
were actually high fashion in 1995 and 1996 when i designed, implemented, and kept after this site. i still think the menu map is quite cool, though. the 'recycled' style paper was part of our branding; we used that kind of paper for all our posters.
for archaeological purposes, the site is still up; feel free to check it out [update: apparently it's gone now, too bad].