Research in the AlloSphere

The AlloSphere Provides a Collaborative Research Environment

NanoMed

Photo by Valerie Liu

The AlloSphere is a next-generation, large-scale, audio and visual immersive instrument and laboratory that allows one to literally step inside a representation of experimental data. The AlloSphere can function as a very large digital microscope connected to a computer cluster or as many other types of virtual laboratories. Thus researchers, up to 30 at a time, can conduct experiments in simulation, yielding scenarios to attempt in their own facilities. Additionally, the AlloSphere instrument can incorporate diverse data—such as inorganic and organic materials⎯that typically cannot move from one laboratory to another.

In proximity to the AlloSphere are associated research laboratories also run by Media Arts and Technology professors: Four Eyes Lab, Vision Research Lab, Systemics Lab, Experimental Visualization Lab, Plurilabs, Translab, and the Center for Research in Electronic Art Technology (CREATE).

The AlloSphere Supports Diverse Research Applications

For convenience, we divide the AlloSphere’s research applications into two broad categories: activities that use the instrument as a research framework for immersive, multimodal environments, and activities that use the AlloSphere as a functional tool for scientific exploration. Research areas to pursue include: new media systems design including computation, interactive interfaces, projection and immersive displays; networking; 3D spatial audio; media signal processing and software; virtual reality; nanotechnology; physics; materials science; medical-related applications including genomics and omics in general; finance and economy; social networks; and perception and cognition.

Interactivity is essential so that the computer can be used as a simulated instrument much as scientific instruments are used in the laboratory. This shifts the computational platform from the traditional off-line super computer calculator to real-time multi-sensory computing, e.g., using our senses to analyze voluminous amounts of complex information. There are frequency relationships in complex quantum, nano-scaled, chemical and biological information, which we precisely map into the visual and audio domains. The AlloCore computational language focuses equally on visualization, sonification and interactivity. Our extensive use of the audio domain emphasizes unfolding data through the 4th dimension, time.

The AlloSphere Research Group

The AlloSphere Research Group is comprised of a number of sub-groups of faculty and graduate students based on their primary research interests and expertise. The 3D immersive visual design sub-group conducts research in spherical display, flat panel display, and stereographic techniques. The Audio sub-group specializes in 3D spatial audio systems design and data sonification. The Systems sub-group focuses on real-time distributed multimedia systems design. The Human/Computer Interaction subgroup focuses on multi-user interactive tracking systems, handheld devices, and interaction design.

AlloSphere Research Demonstrations - Proofs of Concept

1. New Atomic Bonding: Multi-Center Hydrogen Bond⎯An Interactive Visualization and Multi-modal Representation of Unique Atomic Bonds for Alternative Fuel Source.

Photo of MCHB

In this multi-center hydrogen bond demo, the source of conductivity is zinc. Hydrogen replaces oxygen and forms a highly unusual multi-center bond. Simulations will allow for calculations at a higher level of complexity, leading to the investigation of how bonding strength changes as hydrogen is gradually drawn out of a hydride compound. This is a technique for using hydrogen as an alternative energy source, functioning as it would in a real world hydrogen car. The research is focusing on substances that hold hydrogen like a sponge, with the hydrogen atoms bonded weakly to the crystal structure of the host material so that they can be released with a small amount of heat. Visualizations and interactive simulations are leading to new discoveries on how these materials bond and can be released. One flies through the 2000 atom lattice navigating by the sonification of the atomic emission spectra of oxygen and zinc. The unique hydrogen bond has its own "musical voice": all sonic information comes from precise mathematical calculations transposing the atomic emission spectra into the audio domain.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Professor Chris Van de Walle, Dr. Anderson Janotti, Professor JoAnn Kuchera-Morin, Dr. Lance Putnam, Dr. Basak Alper.

A video of this project is located on the Media Page.


2. AlloBrain

Photo of the AlloBrain demo

An experiment into multimodal data exploration, Allobrain presents a virtual world consisting of isosurfaces of brain blood density drawn from fMRI imaging data, attempting to provide the experience of being inside a brain as architectural space.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Professor Marcos Novak, Professor JoAnn Kuchera-Morin, Dr. Xavier Amatrain, Dr. Dan Overholt, Dr. Lance Putnam, Wesley Smith, John Thompson and Dr. Graham Wakefield.

A video of this project is located on the Media page.









3. Center for Nanomedicine Research

Nano Art

In the Center for Nanomedicine research collaboration we are building an interactive simulator that will facilitate virtual experiments in the delivery of chemotherapy to cancerous tumors in the pancreas and liver through nanoscale particles. In the past year we were able to re-construct an anatomically correct human body from MRI data including the vasculature that connects the pancreas and liver. We are currently working on the fluid dynamics simulations for blood flow through the vasculature. This will allow scientists to study blood flow through different sized arteries and veins, witnessing the blood in the bifurcations of the vasculature. We are also building a particle system that will allow the study of nanoscale particle flow within the fluid dynamics simulations. As we receive more data from the experiments of the materials scientists who are building the nanoscale particles, we can simulate the precise geometries and binding equations of the particles, trying various scenarios to discern which shapes will bind better to the sides of the vessels to leak through to the organs where the tumors occur. This research is uniting an interdisciplinary group of nanoscientists that cross physics, biochemistry, chemical engineering, mechanical engineering, and fluid dynamics.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Pablo Colapinto, John Delaney, Dr. Haru Ji, Qian Liu, Gustavo Rincon, Dr. Graham Wakefield, Dr. Matthew Wright, Professor JoAnn-Kuchera Morin, and Professor Jamey Marth.


4. Hydrogen Atom

Hydrogen Atom 1

Hydrogen Atom 2

Images by Lance Putnam

The Hydrogen Atom projects simulate and display the quantum mechanical wavefunction of a single electron in a superposition of different atomic orbitals using solutions of the time-dependent Schrödinger equation. Users can dynamically set all quantum parameters of the physics simulation and observe dynamic behaviors such as photon emission or absorption to develop their intuitions about unpredictable spatiotemporal phenomena.

Key faculty and graduate student researchers associated with the project: Professor JoAnn-Kuchera Morin, Professor Luca Peliti, Dr. Lance Putnam.



















5. Adrift

Adrift is an artistic audio/visual composition made specifically for the AlloSphere 3D immersive environment. Underlying the work is a recursive matrix multiplication that generates an infinite sequence of coordinates. Adjusting the matrix coefficients gives an endless variety of both regular and complex patterns. The work interpolates from one parameter set to another producing an evolving visual and sonic environment. The mathematical system used is an iterated function that produces a sequence of 4-dimensional coordinates called an orbit. This system can be dissected into both a linear and non-linear part. The linear part is simply a recursive application of a transformation matrix on a 4-vector (a coordinate of the orbit), representing a scaling, rotation, and translation in four dimensions. Recursively applied, this transformation matrix results in a variety of spiraling orbits.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Dr. Lance Putnam.


6. Copper/Tungsten Data Set

A series of visualizations of a 3D volumetric Materials Science dataset collected by UCSB’s Tri-Beam microscope. A per-slice viewer, a volumetric viewer, and a full-surround volumetric viewer provide insight into the spatial structure of the copper channels in a tungsten matrix.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Dr. Matthew Wright, Dr. Graham Wakefield, Charlie Roberts, Professor JoAnn Kuchera-Morin, Professor Tresa Pollock, Dr. McLean Echlin.


7. Knot Theory

This research presents the experience of standing inside of a knot, from the simplest trefoil knot (the kind of knot you tie your shoes with) to arbitrary numbers of windings (even fractional) in multiple dimensions. As the knot winds around us, we can also adjust the writhe, or cabling, around the knot itself. We visualize and sonify the knot’s “intrinsic energy” and can watch as it gradually relaxes into its lowest energy configuration for optimal embedding in space. We can also see and adjust the Hopf fibers around which the knot winds. We are just beginning to investigate some of the more complicated aspects of even the simplest knots.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Pablo Colapinto, Professor JoAnn Kuchera-Morin.


8. Graph Browser

Stereoscopic Highlighting

The Graph Browser is a class of techniques to display datasets combining structural data and text, such as labels on a molecular model, feature annotations on a geographic point cloud, or textual descriptions of nodes in graphs representing social networks, coauthorship, etc. The goal is to combine a shared large display alongside legible text for each individual. This includes 2D Graph Visualization on Stereographic Displays. Our motivation for this project is to investigate alternative ways of making use of stereo displays for graph visualization. We propose a technique called stereoscopic highlighting that utilizes the visual emphasis provided by virtual depth to highlight points of interest on a 2D node and link diagram. Our technique utilizes stereoscopic depth to highlight regions of interest in a 2D graph by projecting these parts onto a plane closer to the graph without resorting to other highlighting techniques like color or motion, which can then be reserved to encode other data attributes. This mechanism of stereoscopic highlighting also enables focus+context views by juxtaposing a detailed image of a region of interest with the overall graph, which is visualized at a further depth with correspondingly less detail.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Dr. Basak Alper, Professor JoAnn Kuchera-Morin, and Professor Tobias Höllerer.


9. 3D Panoramas

3D panoramas of the earth, moon and many earth locations including the plaza of the Ufizzi museum in Florence, the dining room of the Ennis Brown house (designed by Frank Lloyd Wright), and San Francisco’s Grace Cathedral.

Key faculty, postdoctoral and graduate student researchers associated with the project: Dr. Graham Wakefield, David Adams, Dr. Matthew Wright.


10. Utility Space

The economic value of any item can be measured in arbitrary units of utility. The utility space transcends valuation in terms of monetary units by mapping the value of multiple items to a single multi-dimensional space. This demonstration displays four currencies, the Euro, the Dollar, the Pound, and the Yen, fluctuating over the course of a week in an immersive 3 dimensional graph. The utility space may facilitate analysis of global transaction costs in a fluctuating currency environment by enabling multi-dimensional comparisons.

Key researcher associated with this project: Dennis Adderton.


11. Time of Doubles

This installation presents an interactive artificial life simulation using multiple depth cameras to reconstruct participants’ doubles as an integral component within an evolving virtual ecosystem. It aims to evoke an aesthetic experience of wonder through the careful observation of a natural system.

Key faculty, postdoctoral and graduate student researchers associated with the project: Dr. Haru Ji and Dr. Graham Wakefield.


12. Fluid Space

Nano Art

A fully immersive artificial ecosystem - Biogenetic algorithms unfold this immersive artificial world that exits in a Navier-Stokes fluid dynamics simulation.

Key faculty, postdoctoral and graduate student researchers associated with the project: Dr. Haru Ji and Dr. Graham Wakefield.














13. Ensemble Interaction In Virtual Reality Environments Using Mobile Devices

In typical multi-user Virtual Reality Environments, one user actively manipulates the environment via interactive controls while other users observe passively. In contrast to this, the AlloSphere Research Group pursues a model where users adopt roles and then perform associated tasks concurrently. Early results of our research enabled multiple scientists to probe changes in the probability current and gradient of an electron's wavefunction in the “Hydrogen Atom” project; in the “Allobrain” project, scientists can selectively call agents to report blood density levels from different parts of a human brain. In addition to providing interactive controls to multiple users, our current research also provides users individual viewports into data visualizations. In one example (pictured at right), users perform data mining tasks on graph visualizations. Using tablet devices, users can select graph nodes on the AlloSphere screen and then read the data associated with the nodes on their tablet. Since users have their own personal text displays there is no need for text to be presented on the AlloSphere; this allows the shared screen to be devoted solely to the visualization. In another recent example multiple users can concurrently filter and consume data from the Seattle Public Library system on their tablets while viewing an overview of the data on the surface of the AlloSphere. Mobile devices interact with AlloSphere applications using the app Control, available for free from both the Apple App Store and the Android Market. Control is an open-source application that allows users to define custom interfaces for controlling virtual reality, art and music software. It has been downloaded over 50,000 times since its introduction last year.

Key faculty, postdoctoral, and graduate student researchers associated with the project: Charlie Roberts, Professor JoAnn Kuchera-Morin, and Professor Tobias Höllerer.