STEAM Imaging: A Pupils’ Workshop Experiment in Computer Science, Physics, and Sound Art
By Bianka Hofmann, Sabrina Haase, and David Black, Fraunhofer Institute for Medical Image Computing MEVIS
Alongside Taiwanese media artist Yen Tzu Chang, Fraunhofer MEVIS scientists Sabrina Haase and Bianka Hofmann designed a creative workshop for pupils in the field of computer-assisted medicine and sound art within the STEAM Imaging artist-in-residency program. The program takes place at the Fraunhofer MEVIS Institute at the Technology Park in Bremen, Germany and at Ars Electronica in Linz, Austria. Fraunhofer MEVIS is a software-oriented R&D Institute that develops assistance systems to help physicians with early detection, diagnosis, and therapy of cancer and diseases of the circulatory system, the brain, breast, liver, and lung. The goals are to detect diseases earlier and more reliably, customize treatments for each patient, and make therapeutic success measurable. We have developed several science communication formats that express the value of our R&D to the general public and enable and encourage people to engage with medical technology topics in more depth. We want to remove barriers that prevent people from interacting with difficult health topics such as cancer and heart disease, focusing on the clinical impact of new technologies and their underlying math, physics, and computer science concepts.
Yen introduces the pupils’ performances.
Copyright: Ars Electronica / Martin Hieslmair.
The STEAM Imaging workshop took place as part of our new artist-in-residency program. Fraunhofer MEVIS researchers, alongside the artist, explored medical imaging technologies and sound art in collaboration with the Fraunhofer Talent School Bremen and Ars Electronica in Linz. The participating scientists, foremost mathematician Sabrina Haase, who has six years of experience in designing and hosting STEM workshops with a strong focus on medical imaging, and project designer Bianka Hofmann, a technology-oriented science communicator, enhanced the workshop with artist Yen Tzu Chang to include theoretical and practical sound art components. Through this school workshop, we have brought the artist into contact with new professional outlets.
An R&D Institute designing and hosting STEM and STEAM workshops? The developments in medical technology are changing our everyday lives in increasingly shorter innovation cycles. We need adaptive, cross-disciplinary, intergenerational platforms to deal with the impact and design of these technologies at early stage - a new avant-garde that feeds on the fusion of natural and social sciences, technology, and art to develop societal utopias, not dystopias or promises of technological salvation. Our science communication projects are expressions of responsible research and software development in health care. Programming will become a fundamental school subject much like language and mathematics. The rise of digitalization, big data, and artificial intelligence demands an immediate discussion of ethical principles that need to be publicly understood. This includes generating a set of legal and ethical values for institutes and researchers as well as a transparent culture to deal with failures and the open exchange between researchers, the media, and policy makers.
Live stream from Fraunhofer MEVIS’ 3 Tesla MRI-scanner in Bremen, Germany to the workshop taking place at Ars Electronica in Linz, Austria.
Copyright: Ars Electronica / Martin Hieslmair.
Image out of one video artwork of the pupils. “Flying” through ribs.
Usage rights: Fraunhofer MEVIS and Ars Electronica.
With our STEM workshops, we want to raise awareness of how computerization supports and influences health care and the role of STEM in life sciences. One of our workshops encourages young girls to engage in STEM topics. We always focus on the curiosity of algorithms, procedures, and technologies - learning is a side effect. The participating scientists are inspired to gain clarity and composure concerning their research topics. Being involved in science communication fosters discussion among researchers about different views based on the spectrum of their expertise. Hands-on STEM projects for the next generation have a long tradition at Fraunhofer MEVIS. We place a lot of trust in young talents and provide them with mathematical and natural science knowledge so that they can take action themselves. Pupils can, for example, participate in acquiring MR images and evaluate them with special software. For us, expanding these proven workshops to an artistic level was new territory. We hoped to reach further groups of pupils. Even those who want to study psychology, for example, will need to learn math and statistics, whereas prospective artists have many new tools at their disposal that require basic computer science abilities. To create the STEAM workshop, we built on the experience and workshop material from Children’s University, Girls’ Day, Talent School, Summer Academy, Internships and Math Research Day.
An artwork from the pupils, made with their scanned own hand model.
Usage rights: Fraunhofer MEVIS and Ars Electronica.
Intensity or insanity? In the first phase of the residency, Yen worked closely for two weeks with researchers at Fraunhofer MEVIS. Experts from five fields took part: Sabrina as mathematician and workshop leader, Bianka as workshop designer and technology-oriented science communicator, sound designer David Black, scientists from the MRI physics group, and an expert for the exploration of new technology within the MeVisLab medical imaging development platform. The artist explored selected scientific topics and technologies in medical image processing and learned MeVisLab. She explored relevant ethical and societal issues and imparted an artistic perspective to the workshop. In the second phase, Yen will stay at Ars Electronica in Austria for several weeks to develop the artwork entitled “Whose Scalpel,” a fusion of an interactive model, movies based on medical data, and sound, which will be presented during a live performance at the Ars Electronica Festival in September 2017.
A girl is preparing her performance.
Copyright: Ars Electronica / Martin Hieslmair.
During Yen’s two weeks at Fraunhofer MEVIS, she was mentored by Bianka. The team prepared the two-day workshop for the pupils. Yen and experts held one-to-one conversations on topics such as artificial intelligence. An essential part was dedicated to learning MeVisLab. In addition, during fast-forward sessions, colleagues gave three to five minutes presentations of their work, as an overview and a teaser for intensified communication. Yen gave a plenary talk for all employees entitled “The Transformation of Media Art.” She shared her approach to art and her current artworks, most notably her sound installations. Finally, she presented a brief sketch of the anticipated artistic outcome within the artist residency. We invited interested colleagues to hold a session on ethical issues with Yen, including an open discussion. The aim was to name and include questions that concern our R&D, which proved to be a success. As a result, we will continue to address these issues on an ongoing basis.
Pupils’ prepare their STEAM Imaging performance.
Copyright: Ars Electronica / Martin Hieslmair.
Dealing with new technologies: The aims of the STEAM workshop We want to stimulate engagement with, critical dialogue about, and implementation of new technology, however, we want to help an artist complete her work with new technology and explore new fields. The aim of the STEAM workshop was to bring STEM topics to pupils interested in art and raise the awareness of the role of math, physics, and computer science in health care and life sciences and to show the value of current science and technology for art. The pupils used medical image processing technology in an artistic way, and learned about media art. Both artists and scientists have the chance to receive insight about the coming generation.
Yen Tzu Chang, Sabrina Haase and pupils prepare their performance.
Copyright: Ars Electronica / Martin Hieslmair.
Combining models, medical imaging, sound and performance The first two-day workshop with pupils from 7th to 9th grades took place at the Institute with 24 pupils at the end of March 2017 and the second at the beginning of June 2017 with 20 pupils at Ars Electronica in Linz.
Yen and Sabrina first split the pupils in Bremen into two groups. One group dipped their hands in small trays to create plaster casts. They then poured gelatin into the casts to create detailed models of their hands. The pupils placed their models in the Institute’s MRI scanner. 3D images of the hands were created without pupils having to insert their hands into the scanner. In an open discussion, we addressed ethical questions concerning new technologies in health care with the pupils. On the next day, the pupils loaded their image data onto a computer with MeVisLab to view the data in 3D and transform it artistically. They were able to color the images and create their own personal artworks with the help of an MRI scanner and scientific image processing software. The second group digitally processed 3D head and torso image data provided by MEVIS scientists. The pupils created short artistic video sequences. We combined those videos with audio software created by Yen and produced electronic sounds based on the images. Furthermore, we incorporated the sounds of the MRI scanner. To generate the sounds, the pupils used fruit to produce their own MR image acquisitions. These harsh mechanical sounds established the basis for two performances during which pupils played everyday objects such as tables and trash cans to unleash unusual sounds. Together with the self-made videos from the medical images, they developed sound-art performances, presented at the end of the two-day workshop.
The Linz Group benefitted from the experiences gained in Bremen. As in Bremen, students played a game to differentiate between medical imaging techniques such as MRT and CT. The students worked with medical images and videos using a custom web application based on MeVisLab. We also addressed the ethics of new health care technology with the pupils. However, we adapted the workshop in Linz to use the Deep Space 8K, a 3D theater at the Ars Electronica Science Center. Three short films by Fraunhofer MEVIS, including “Virtual Journey Through the Heart” are part of the permanent lineup. We explained the possibilities of medical imaging very vividly. Because we did not have access to an MRI scanner in Linz, we did not build phantoms. As a substitute, we streamed a live video to our Institute’s scanner, and, as in Bremen, we explained the scanner itself and scanned fruits. The parts of the workshop related to sound art were more structured and oriented towards pupils (Yen had been more familiar working with adult students). We gained insight into the history of sound art, including works by John Cage and Nam June Paik. The students learned to program an audio software tool. As in Bremen, groups of students recorded environmental sounds with self-made sounds and combined them with the sounds produced by an MRI scanner.
Paths to pursue with and for the next generation We hope that all participants – pupils, scientists, artists, and the project developer – depart the experiment with their own questions and paths to pursue. We provide the framework, including computer, software, and workshop materials. In the end, what matters is a mutual respect, devotion to the topics, and curiosity about what can emerge and the collective work and openness to become involved with and relate to one other. An external scientist specialized in the effects of arts-based initiatives will evaluate both workshops and the results will be published in August (see HERE). In autumn, we will discuss the future of this form of science communication at the Institute.
We have to admit that it is difficult to be creative in the office environment beyond working at a computer. A pure STEM workshop can be done in this environment, although we could have benefitted from a more ideal working space for phantom building and a music room. It was not easy to open up and engage in the process within the daily goal-oriented and direct routine of applied R&D to take a deeper look at what happens around us. We would have enjoyed having more time to physically share space to allow play, fogginess, and adaptation within the cross-disciplinary topics. This STEAM workshop is one way to inspire pupils to explore STEM topics. We want to create additional forms of communication to bring people closer to new technologies in health care. We will continue learning how to integrate dialogue and collaboration with coming generations and the public in the continuous R&D process and viewing society as a partner to help shape future R&D.
A zoom into our STEAM Imaging project image.
Copyright: Fraunhofer MEVIS.
However, the ethical idea of a common decency toward the society and the next generation when making important advances in science and technology seems to now have a chance. A fundamental understanding of new technology must be available to all pupils. We need projects that foster the talents and resources of every child instead of just mirroring the education of their parents’ generation. What ideas for the future, sustainable beyond our own lifetime, do we want to carry forward? How do we want to discuss new possibilities in science and technology and describe their development? How can we expand R&D teams, not only in terms of expertise in ethics, philosophy, and art but also the spectrum of participants to include women and people of color? The future belongs to the first companies and organizations to clearly prioritize such teams and address sustainable R&D and new ethical narratives in all their creativity to develop livable future scenarios.
We need to deeply involve the next generation and let them actively co-develop and promote science education and digital expertise, but we can no longer train a generation that is competent and educated in all topics. Often, young people can simply grasp the applications and possibilities of new technologies better than the adults teaching.
The artistic outcome of the residency "Whose Scalpel" will be presented at the 2017 Ars Electronica Festival in Linz. This artist-in-residency program involving pupils will be presented in a talk at the Art & Science conference, which poses the closing point of the European Digital Art and Science Network.
Bianka Hofmann realizes scicomm projects to involve people early in new scientific topics and future technology thus making the implications for society understandable. She develops content and formats such as an artist-in-residency program, 2D and stereoscopic 3D short movies, interactive exhibits, children’s University and teacher enhancement projects. She focuses on innovative communication concepts to encourage people to get involved and to create their own experiences. She works as Head of Corporate Communication at the Fraunhofer Institute for Medical Image Computing MEVIS, where she developed the institute’s science communication and strategic press- and media work. Previously she worked at the University of Bremen on transferring knowledge into schools. She has worked in various companies on the transfer of scientific research results into application-oriented concepts. She studied Comparative Religion and Biology and investigated the evolution of cultural behavior among Mammals and is a qualified communication coach.
Sabrina Haase is a mathematician and, currently, coordinator of international research & industry projects at the Fraunhofer Institute for Medical Image Computing MEVIS in Bremen, Germany. Her previous research at Fraunhofer MEVIS entailed bio-physical procedures for mathematical modeling and simulation of thermal therapies as means of improving the therapies’ success. Since many years she is expanding and leading the educational offerings the Institute provides to up-and-coming young researchers—for example, the STEAM Imaging project.
David Black is a researcher at the University of Bremen and the Fraunhofer Institute for Medical Image Computing MEVIS in Bremen, Germany. He has a music conservatory background in classical composition and electronic music as well as expertise in usability for medical applications. He applies sound synthesis techniques to medical applications, including navigated instrument guidance, novel gesture interaction, and dataset sonification. Mr. Black is interested the use of auditory feedback for medicine and how this fits alongside other innovative methods into the operating room of the future. In addition, he is a composer and sound designer for short films, mobile applications, and games.
