Dr. Mi Zhang, Assistant Professor of Electrical and Computer Engineering of the College of Engineering's Department of Electrical and Computer Engineering, and Dr. Taiwoo Park of the College of Communication Arts and Sciences (ComArtSci) are bringing the power of technology to day-to-day life with JARVIS, a ubiquitous mixed reality system named after Tony Stark’s Iron Man suit from the Marvel film series The Avengers.
With JARVIS, the two assistant professors have brought together ubiquitous computing and mixed reality (MR) technology to collect research and enhance daily routines, like hitting the gym.
Park said the purpose of the research is to experience mixed reality anytime anywhere.
“We are creating an interactive system that automatically understands what is happening around you—ubiquitous computing—and draws a virtual world nicely blended with your physical world in front of your eyes— mixed reality,” Park said.
Zhang said the researchers studied how JARVIS’ ubiquitous mixed reality system operates in a fitness setting. Participants attached a sensory device to exercise machines at the gym, which collected their exercise data. JARVIS then analyzed and responded to a user’s movement during a workout, producing an MR scene for the user. Once a user begins working on an exercise machine, JARVIS will detect the type of exercise and create a personalized exercise coach. The research team’s sample app for seated abdominal training also showed that the app can help users better activate target muscle groups during a workout by showing the user’s virtual self in the MR world.
The Power of Collaboration
Zhang said the research is a natural combination between his research on intelligent sensor systems and Park’s research on virtual reality and augmented reality systems.
Mi Zhang's research focuses on intelligent sensor systems. Zhang and Park’s study has been published in the Association for Computing Machinery’s Interactive, Mobile, Wearable and Ubiquitous Technologies and GetMobile.
For the next step in their study, Park and Zhang will expand the context in which JARVIS can be used outside of the gym. They’re working with external collaborators to explore how to create more variety in ubiquitous mixed reality applications in both industry and academia.
“There will be a lot of opportunities for creative and immersive services using ubiquitous mixed reality,” Zhang added.
ComArtSci supported the development of the research through its interactive media studio. The research was conducted with support from Trifecta Initiative funding, as well as startup funding from ComArtSci and the College of Engineering. Research at MSU utilizes "JARVIS" as your personalized exercise coach.
ECE communications contact: Wen Li