NYU Meyers develops Holodeck

NYU Meyers Receives $2.9M Grant from the National Science Foundation to Develop a Holodeck Instrument

A powerful, single, well-integrated engine for distributed transdisciplinary research and innovation with the potential to create new insights into fundamental natural phenomena, offering unparalleled tools for intellectual and creative output across disciplines. 

An interdisciplinary team of New York University researchers, led by Winslow Burleson, PhD, MSE, BA, an associate professor at NYU’s Rory Meyers College of Nursing, received a $2.9M grant from the National Science Foundation (NSF) to develop a well-integrated software/hardware instrument incorporating visual, audio, and physical (haptics, objects, real-time fabrication) components, known as the NYU Holodeck.

“Our goal is to create an immersive, collaborative, virtual and physical research environment with unparalleled tools for intellectual and creative output,” said Professor Burleson.

The NYU Holodeck will be developed as a well-integrated software/hardware instrument incorporating visual, audio, and physical (haptics, objects, real-time fabrication) components.

A collaboration of researchers throughout NYU’s schools and colleges, the NYU Holodeck has a strong interdiscplinary backing, including a substantial additional $1.2M cost share from NYU. The team is composed of researchers from the NYU-X Lab, based at New York University Rory Meyers College of Nursing, NYU’s Courant Institute of Mathematics (both in downtown Manhattan), NYU’s Steinhardt School of Culture, Education, and Human Development, and NYU Media and Game Network (MAGNET) based at the NYU Tandon School of Engineering (in Brooklyn), amongst others.

Among the collaborators is renowned Professor of Computer Science, Ken Perlin, of the NYU Media Research Lab. In 1997 he received a Technical Achievement Award from the Academy of Motion Picture Arts and Sciences for the development of Perlin Noise, a technique used to produce natural appearing textures on computer generated surfaces for motion picture visual effects.

“The NYU Holodeck will provide a compelling opportunity to explore and advance new types of science, permitting researchers from diverse disciplines to interact with theoretical models, real objects, robots, and agents, engendering insights that may not be possible using current 2-D and 3-D representations and analytic techniques,” says Professor Perlin.

The holodeck project is an amalgamation of collaborators’ prior work in disparate areas, ranging from Burleson’s astronaut robot mission simulators, affective learning companions, and teachable robots designed to help teach school children geometry, to Ken Perlin’s Holojam—a virtual reality experience that entails drawing and collaborative interactions.

“One of our collaborators Agnieszka Roginska, PhD, Clinical Associate Professor of Music and Music Education has a high end Music and Audio Research Lab (MARL),” says Burleson. “We do a lot of motion capture work in the game development department,” adds Perlin. “So individual pieces of the Holodeck already exist. It’s our intention to fuse these components into an integrated instrument.”

The NYU Holodeck has drawn comparison to CERN’s particle accelerator in its potential to create new insights into fundamental natural phenomena, offering unparalleled tools for intellectual and creative output across disciplines.  Others have likened the Holodeck to the flexibility, versatility, and power of Iron Man.

“Development of the NYU Holodeck has strong support from the Vice Provost for Research and NYU administration,” says Eileen Sullivan-Marx, PhD, RN, FAAN Dean & Earline Perkins McGriff Professor at the NYU Meyers College of Nursing. “We see the project providing significant impact on NYU infrastructure and research capabilities that can extend to national and international collaborators. The NYU Holodeck has significant potential for broader impact including: research by women: underrepresented minorities and persons with disabilities; enhancing future research; and training the next generation of STEM innovators and instrumentalists.”

A comprehensive Data Management Plan accompanies the proposal. The goal is to give every researcher the ability to discover and interact with datasets across all research sites. Therefore, the Holodeck equipment and nodes will be connected via NYU high-speed internal data links (10Gbps or faster), and Internet 2 to permit seamless exchange of data and simulations. The Holodeck team will use both distributed representational state transfer data services to support internet access and allow service chaining with data processing and analytical services, as well as direct use for visualizations, sound and haptic interaction. This structure will allow collaborators to create data/analytical workflows that can be validated and reproduced.                                              

About the NYU Rory Meyers College of Nursing

NYU Rory Meyers College of Nursing is a global leader in nursing education, research, and practice. It offers a Bachelor of Science with a major in Nursing, a Master of Science and Post-Master’s Certificate Programs, a Doctor of Nursing Practice degree and a Doctor of Philosophy in nursing research and theory development.