Dr. Steve DiPaola | Meehae Song iVizLab, SFU
The main objective of this project, aim to create an application for patients with dementia by exploring 3D abstract forms, real-time feedback and an immersive 3D world.
The aim was to create either Augmented Reality (AR) or Virtual Reality (VR) application, which can easy be adopted to our target audience. Weighing out the pros and cons, we decided to implement an AR application because the VR headset can be hard to put on patients. Using Kinect, simple animations will be triggered based on users head and neck movement. We wanted to implement the head and neck movement because it’s natural and not forced. For example, when you go into a space, naturally you look around the space.
In terms of animations, we aimed implement simple gesture for gender neutral human figure such as gazing, meditation pose, dance motion, sitting, or breathing. The reason why it’s best to have different animations is because we wanted our users to be immersive and engaged in our space. Our application will keep patients with dementia attentive, and since it’s responsive to faceTracking it will help them connect to each of our entities, and help them immerse into our 3D world.
One of the questions that we explored is to what degree can an abstract form be recognized as a human figure?
Inspired by 2D abstract forms of human figures created by Steve DiPaola. My aim was to recreate this in 3D. I started by exploring softwares best suited for our goals. I looked into Rhino and Grasshopper. Then Maya, PopcornFX, and Mixamo. Before creating a 3D model, I started by making sketches.
Not knowing much about Rhino and Grasshopper, the main challenge was to learn the software and create an abstract figure that can be controlled in realtime. Here we wanted to use the parameters in conjunction with user movement. Thus, the shape changes form based on user movement, giving users instant feedback in 3D environment.
After creating the models in Rhino and GrassHoppper, I exported the model in different states changing the parameters. Then it was imported into maya to change state using the blend shape editor. Although I was able to create an abstract figure and access the parameter that changed its form; the blend shape editor was not as responsive as we intended it to be.
Since that was not working, we wanted to test PopcornFX as it allows forms to be encapsulated in particles. To test popcornFX, I made a simple model in maya.
Since we figured out how it works in popcornFX, with another model, I animated a simple dance sequence in Maya.
After completing the models in maya, the models were taken into popcornFX. The reason why we chose this particle effect is because we wanted something that creates a relaxing effect, and we wanted to control the particle based on user’s’ movement. However, once we took the dance animation into popcornFX, the particles wasn’t as responsive as we intended it to be. And with the dance sequence, we felt more distanced to our entities and it lack the connection. Thus, we decided to go with simple gestures and movements.
I took the model into Mixamo to create simple animations. This software was fairly simple to learn and use. Then it was exported into PopcornFx to create the particle effect. One of the most challenging aspect in this project was learning popcornFX because the software itself is fairly new, and lacking tutorials. However, with the help of my teaching assistant we were able to learn, and create a desired look. Below is the screen captures of the results in popcornFX.
The next step was to create a surreal 3D environment in Unity. We wanted to create a surreal environment to immerse our users. Here, I experimented with Unity assets implementing skybox, fog, texture, and lighting.
Going further, the final result is much more simpler, and more polished. The image below captures the final iteration. At the moment we are working with Kinect to control the animation in unity.
This project helped me explore different softwares, and challenged me to try new ideas. It was about finding ways to communicate abstract forms in a recognizable manner. Based on my research, we hope our application will keep patients with dementia stay attentive, help them connect to each of our entities, stay relaxed, and help them immerse into our 3D world.
Maya 3D Modeling | Animation | Scripting | Unity environment
Rhino | Grasshopper 3D | Maya | PopcorrnFX | Mixamo | Kinect | Unity