Sha Xin Wei & the Synthesis Center
By Joe Ferguson, contributor
Joe Ferguson: Tell us a bit about your background - how did you find your way to the intersection of art and science/technology?
Sha Xin Wei: Well, ever since I was in high school I was interested across the board in everything from creative fiction and writing music to mathematics with history and linguistics as hobbies inherited from my parents. My formal (both undergrad and graduate) education, however, was in theoretical mathematics (differential geometry, analysis on manifolds, geometric measure theory). But I revelled in the oceans of learning available in college for exploration, from classical Chinese philosophy, and folklore and mythology, to quantum mechanics, and general relativity. I learned so much from my friends over lunches and dinners in the common rooms, and from working in the radio station.
But there was always this sense that my one profession was mathematics, whereas my other interests were all secondary. During the 1990s this ran from social activism to American Sign Language, experimental theater, and computational media.
It took decades for me to weave these into a synthesis, a path that led out of graduate studies into the then novel phenomena of personal computers with Xerox PARC, and Steve Jobs (Apple and NeXT), and back again into a unique PhD under the benevolent if sometimes bemused protection of Rafe Mazzeo (Mathematics), Terry Winograd (CS), and Tim Lenoir (History and Philosophy of Science), together with a host of "friends-who-believed."
The more collective synthesis started with my founding of the Topological Media Lab in 2001 at Georgia Tech, dedicated to the study of gesture and semiosis from computational as well as phenomenological perspectives. What really allowed the TML to flourish, however, was moving it to Canada, thanks to a Canada Research Chair in media arts and sciences, the first Chair outside the sciences in that federal program. The CRC program is designed to bring academic talent to Canada, plus a diverse rich ecology of funding from Quebec and the 15 year-long Hexagram network for research-creation. Also, we won funding from the Canada Fund for Innovation, originally set up to fund technoscientific apparatus, like MRI scanners or radio telescopes, to equip the TML with state of the art equipment oriented around our unique blend of movement arts, time-based media, and realtime responsive systems, to support experimental, experiential investigations of questions inspired from continental philosophy. We created, exhibited, and published work that looked like media art, fine art or performing arts, or technoscience and engineering, but the inquiry was, at heart, philosophical. Over the years, as the techniques got stronger, we built and published experiments that were more explicitly philosophically inflected.
Palimpsest time lenses installation, realtime responsive video filtering different rates, pasts, and rhythms from video streams from live cameras, and re-blending those subsets of the video stream together in a painterly palimpsest, projected into the apses of the 800 year old church adjoining the Museum of Arts and Crafts in Paris. Part of Les corps dessinant exhibition. Photo credit Synthesis Center ASU.
JF: Art inspired by science is a familiar theme. You propose that a transdisciplinary approach yields a different product - one that is truly collaborative. Tell us about that.
SXW: First of all, I think arts and sciences are incommensurate activities, with incommensurate goals, practices, and values that change according to cultural context and history. It doesn’t do justice to the profound diversity of human imagination to think otherwise.
Artists don’t have to merely represent, permute, or remix, data (images) from the sciences. And scientists don’t have to make “tools” for artists to use in place of paint brushes. Though it may be cozy, that often makes boring work. That sort of art/science merely re-inscribes CP Snow’s divide, when in fact the world, is not divided in the first place. Life as well.
Disciplines really make sense with respect to institutions, such as the European academy (which in its modern form is about a thousand years old dating back to Padova and Bologna, for example, in the 12c and 13c). But the world, and life, is not disciplined. Modern science, since Newton’s day, draws its strength in great part from the divide-and-conquer approach to understanding the world: in order to understand the world you partition it, or our experience of it, into some narrower categories or portions, and try to understand those portions according to certain categories, ignoring the rest of the world. This modular approach in science has led to the great sciences: physics, chemistry, astronomy, neuroscience, etc.
Another source of scientific strength is its co-construction of instruments, observables, entities, and theories. One does not precede the other! A good scientist knows that at the same time that she or he is making some aspect of the world visible (via an instrument, a theory, or more fundamentally a decision on what is to be observed or measured), she is also rendering all other aspects invisible.
And how could it be otherwise? We are finite mortal beings.
Art (poetry, music, performance, visual and time-based media, etc.) is part of our human response as finite mortal beings to the world that infinitely exceeds every category. How can a finite squiggle or movement connote infinite things like feelings, values, principles?
Another aspect that art helps is enriching whole experience. Transdisciplinary work is a natural way of dealing with the world and experience as wholes.
500 years ago, natural philosophy included the study of the elements, of what we now call nature - living and non-living - as well as what we may now think of as more philosophical matters. Alchemy, for example, split into medicine, chemistry, and spiritual practices, which read each other as sacred and profane when those were later interpretations by institutions that needed to develop their own turf.
Group of students collectively creating a vortex in simulation of weather using realtime implementation of cm1 model in the Synthesis responsive environment. Collaborating with researchers from the Computational Information Systems Laboratory at the National Center for Atmospheric Research (NCAR), the EMA’s allows people to inhabit and become distributions of moisture, pressure, heat, of air using full-body interaction. In other words, inhabitants don’t just look at representations, they _become_ the weather. Students can experiment and _enact_ specific atmospheric processes for creating phenomena such as the convection, orographic lift, monsoons, storm fronts, and cyclones. Photo credit Synthesis Center ASU.
JF: Professional practice - in the arts and the sciences - usually requires familiar, highly-skilled methodology. How does transdisciplinarity fit into professional development and practice? What are the benefits of that approach? Are there any drawbacks?
SXW: Transdisciplinary is an adjective that’s useful to warrant what some people are doing as they work from their particular core competencies, or metiers. In our Synthesis Center, and the antecedent Topological Media Lab, transdisciplinary work pursues fundamental questions (such as “How does meaning form through movement and gesture?” or “What is rhythm? or "How do ensembles of people, and things, coordinate activity through rhythm?” or “How does sense emerge processually?”) which demand drawing transversally across whatever disciplines are relevant. This is one aspect of transdisciplinary work.
We work collectively - combining the best practices of lab, studio into a modern atelier - because we tackle fundamental questions, questions that cannot be resolved inside any one discipline. Over years of such work, the participants begin to transform their own discipline’s methods. This is the other aspect of transdisciplinary work.
The main challenge is there are no shortcuts. I say to people, if it took you ten years to read and absorb Wittgenstein or Deleuze, it will take another decade to learn photography. This takes time, often more time than is typically given in a typical undergrad program.
JF: What is experiential science/technology and how does it - or does it - fit into academia?
SXW: Experiential science means paying attention to the felt meaning (Eugene Gendlin) of any situation or process as it unfolds. This means recognizing that anything that is measured or expressed or articulated is measured or expressed by some entity that anticipates, reflects on, interprets those acts of measurement or expression, or articulation.
This demands that we both learn from those who study experience (e.g. the American pragmatists, phenomenologists, writers, historians), as well as those who skillfully modulate experience (artists, engineers).
JF: Your work bridges many domains not conventionally thought of as having connections. In "Serra," for instance, you use temporal manipulation to explore similarities between human and vegetal intelligences. Tell us about this work - why is it important for us to understand this connection/similarity and how may it affect our perspectives on science, art, and/or technology?
SXW: In Canada, research-creation (or recherche-création) is a very useful term describing the creation and dissemination of knowledge gained via artistic methods or modes of inquiry. This is more accurately describes the way Synthesis works (and the way that TML worked while I was directing it as Canada Research Chair 2005-2013).
All our research and creation works start from compact inquiries - Serra was born out of several inquiries: “What could art be that is not made by humans and not for humans?” “How can we come up with an ecological way of designing things that doesn’t assume that we humans are the only beings worth attention in the universe?” “How can we come up with more humane politics and ethics, but not starting from the human as the center of the world?” These led to four years of readings and seminars (Spinoza, Deleuze and Guattari, Michael Marder), paralleled by some physical installation works (including the ironically named Plant Life Support System(s)). Later, these processes jumped to a high level of ambition when Oana Suteu Khintirian brought a collaboration with the famed choreographer Ginette Laurin, founder of the O Vertigo! Dance company in Montreal. We secured Canada Council for the Arts funding that enabled Oana to come and work with Synthesis’ core team of media artists, Todd Ingalls and Julian Stein, to produce a second generation of work - a responsive environment in an installation built as a poetic reference to a greenhouse made of paper and timelapse films palimpsest together via realtime video processing techniques inspired by optical printing from analog film.
A basic challenge for Serra is to make it possible for people to sense the plants as dynamic beings, and have a rich ensemble experience with plants via rescaling of perspectives and felt time.
JF: You visit temporality again in Rhythmanalysis - this time to examine the experience of cities, organizations, bodies, and media. Tell us about this work. Why is it important for us today?
SXW: In his last book, Rhythmanalysis, Henri Lefebvre tried to get a sense of the boundlessly complex life of the city through the figure of rhythm. Inspired by that manifesto, I thought that we might pursue a multi-scale study of the complex dynamics of the world and our lived experience in the world, using rhythm as an entry. First of all, there’s no one definition of rhythm. But the beauty of this is that everyone has their own notions and forms of rhythm. By rhythm, I loosely mean the dynamical variation of matter-energy and feeling (affect).
Our particular areas of strength include experimental and computational methods for reproducibly varying time-based media (video, sound, actuators) at millisecond scale, and working with movement and gesture (tenths of a second). We complement that by working with, for example, organizational anthropologists and psychologists (scales of days, weeks, months), and with urban geographers, architects and planners (scales of years, generations, centuries). Rather than flatten all these diverse phenomena by applying a one-size-fits-all technique, we can turn this around.
A collective tapestry-like fabric sensitive to proximity and gesture of groups of people, who can perform everyday activity around it, or perform it as an ensemble-gestural instrument. Woven on Hexagram/ Concordia’s computer- controlled Jacquard loom, with conductive thread and custom electronics. Hardware and software behavior composed by computer musicians / sound artists. Photo credit Topological Media Lab.
Frankenstein’s Ghosts – 2009-2011. Creation-Research project making a monstrous ensemble of musicians, dancers, and responsive media instrumentalists, inspired by Shelley’s novel. Performance-lab and collaboration between the Blue Riders Chamber Ensemble, Topological Media Lab, and Concordia faculty, including Michael Montanaro (director and choreographer), Liselyn Adams, Sha Xin Wei (responsive media systems). Photo credit Topological Media Lab.
JF: Synthesis is a bold and exciting investment of resources - both intellectual and physical. Tell us how it came to be, and where it’s going.
SXW: It’s the generous support of President (Michael Crow) and Provost (Elizabeth Capaldi Phillips) of ASU that persuaded me to wager that we could transplant back to the United States the pursuit of fundamental questions using transdisciplinary fusions and inventions of whatever techniques give most purchase and generate the most fertile further effects. The wager is that backed by the resources of a major American research university in the U.S. scientific and cultural economies, and embedded in the dynamic communities that ASU serves and attracts, we can even amplify this free and fundamental inquiry and extend it significantly beyond the frame of art and computational media.
This depends a lot on the people we attract, support, and the communities we form around us. For example, in addition to art work like the Serra vegetal life installation, which is now being transplanted to one of the largest botanical gardens in the world in Montreal, we’re also supporting work ranging from experiential complex weather systems (being a hurricane, rather than looking at its photograph), to multidimensional signal analysis of continuous gesture, to enactive experiments of economies that are not based on discrete finite transactions as an alternative to blockchain logic.
JF: How do you facilitate or create collaboration between disparate professional and academic domains?
SXW: The basic principle is respect for each other’s wealth of expertise or experience, also known as the principle of charity which I’ve applied since I ran the Interaction and Media Group at Stanford 20 years ago: “I don’t know what you’re talking about, but I grant that you know what you’re talking about, so I’ll continue the conversation with you.”
Our more spectacular projects attract creative, intellectually, and technically gifted people who are not content to simply turn the crank on their expertise, and want to do something different and to explore something more ambitious than what any one of us can do on our own. Practically, I say to people, it takes a year of conversation with not just me but everyone in the center, and multiple encounters or works before we can figure out whether and how an affiliation is mutually beneficial.
Concretely, we start with internships - where the intern works on exercises that parallel the main research streams or projects, mainly to become adept in some techniques. Further along, some take on apprenticeships and shadow the more experienced affiliates (faculty, research staff, senior research students) in some roles that are part of some Center-level project. Learning professional responsibility is critical part of these apprenticeships. Of those who have or develop such professional-level skills, some also blend the theoretical, philosophical, and conceptual resources that I and colleagues continuously introduce in parallel seminars. Armed with technical know-how and intellectual knowledge, it takes experience to learn how to come up with fertile and significant questions, propositions, or poetic visions that can bear fruit via making experimental experiences or experiential experiments using our time-based media, responsive environments, movement or gestural arts, and so forth. There is no one path, and hundreds of people have made their way through our research-creation nexus. More adventurers are always welcome!