For me, the intersection of planetariums ans biomedical data began in 2010. I was working at the Chaffee Planetarium, in Grand Rapids, Michigan, and our parent institution, the Grand Rapids Public Museum, announced it would rent a plastinated cadaver exhibit. On the planetarium team we were charged with making a complementary planetarium show. I spent the next 10 months producing models and animations for Our Bodies in Space, exploring the anatomical effects of space on the human body.
A still from an animation in Our Bodies in Space depicting oxygen depletion to the brain, modeled from months of manual work by me with medical reference imagery.
The Chaffee wasn’t yet a fulldome video planetarium, everything we visualized was projected on slides, or played on inset projectors with DVDs, plus some Digistar II wireframe graphics as appropriate. But I felt like there was potential in the work we had done. And Grand Rapids was a booming center of biomedical research. Was there potential for local researchers to use the planetarium to explore their work using our planetarium?
Producer Patric McPike pilots through the cellular structure of the brain during the premiere of Neurodome at Morrison Planetarium.
I’d next end up revisiting these ideas at Morrison Planetarium. In 2014 we hosted the premier of Neurodome. This project, led by neuroscientist Dr. Jonathan Fisher and produced by Patrick McPike and developed with Sciss for their Uniview software, brought a wealth of high-quality, high-accuracy neuroscience assets into the planetarium. With the same ease we could traverse the night sky and digitally explore the universe, now we could take audiences through the inner universe of their brain.
For our support in the premiere, Sciss supported our continued use of these assets, eventually providing a visual backbone for the Chen Institute Brain and Mind Lecture series at the California Academy of Sciences. A personal highlight was working with UC Berkeley researcher Dr. Gül Dölen on what I nicknamed “Octopus neurodome”—the first ever visualized full 3D CT scan of a whole Octopus’ nervous system.
CT scans of the nervous system of an Octopus, including the retinas to either side, and the tentacles, which contain approximately half the neurons in an Octopus’ body.
It was sharing that work when I launched Tau Immersive that put me back in touch with Jonathan Fisher. He’d been continuing to work on bridging the worlds of biomedical research and immersive visualization—and even how this can support clinical treatment and communication with patients.
Jonathan shared with me his Snorkle visualization platform, developed by him and former Sciss Uniview developer Jeffrey SubbaRao. Designed as an independent web-based visualization platform, Snorkle supports native ingest and easy visualization of data formats produced by medical scanners, which are also used for biology research. A prompt-based engine allows for assembly of visual tours through the data and contextualization with other anatomical and molecular visual resources. They were curious: could we get Snorkle working in planetariums?
I immediately saw a unique possibility here with the OpenSpace project software. Over the next few months we prepared technical demonstrations of a basic pathway to bring Snorkle visualizations directly into OpenSpace.
Initial test piping Snorkle CT scan visualization into OpenSpace
The team was eager to see larger scale tests of this capability. In mid-July we visited our colleagues at the American Museum of Natural History, who supported us with technical demonstrations of the software in the immersive environments of Big Bang Theater and Learning with Data lab, powered by our friends at Elumenati.
Dr. Aaron Steiner tours us through volumetric scans of the sensory system of a zebrafish–research that may one day help humans recover from hearing loss. Photo courtesy Jonathan Fisher.
Working with Jonathan and Jeffrey we rapidly iterated various improvements to the software connection, and by the afternoon had researchers giving demonstration presentations of their work in these theaters.
I see the potential of this project in another successful biomedical collaboration. On my visits to the Ars Electronica Center in Linz, Austria, I’ve been privileged to see their Cinematic Biomedical Rendering tours in the stereoscopic 3D DeepSpace8K theater. This technology inspired the staff at Johannes Kepler University to work with Ars Electronica Solutions to built MedSpace, an on-campus immersive theater dedicated to exploring medical visualizations.
Cinematic rendering tours of real anatomical scan data at Ars Electronica’s DeepSpace8K
There’s still work to do on refining this project, but it was inspiring to see it rapidly evolve in our short time working together in person. In our previews there were stories of how visual exploration through these scans and models changed researchers’ perception and understanding of their own work.
A volumetric tour of the anatomy of a cockroach, data from Dr. Hollister Herhold
The potential to smooth out the conversion process for this data has enormous potential for making planetariums and dome theaters a resource to their communities. Planetariums affiliated with universities and schools can become a visualization asset across disciplines, and researchers may be able to prepare their own immersive talks, hopefully making the job of the planetarium team easier too.
A tremendous thanks to Dr. Jackie Faherty and Ryan Wyatt of the American Museum of Natural History for hosting us, OpenSpace developer Micah Acinapura and AMNH’s engineering team for technical help with AMNH systems, and Dr. Jonathan Fisher and Jeffrey SubbaRao for inviting me to support their ambitions with this project. I’m very excited to see its potential.
