Virtual Reality CAS
VR for a couple years now has proven its usefulness in academics for giving students graphics to help guide their learning. The problem I see however is that all of these VR educational tools are simply intractable diagrams and models and instructor makes. I believe a truly useful application needs to have more than just 10 minutes of usability. Having this in mind and the need to come up with a senior project for my highschool I set out to design some sort of truly useful educational application in VR.
With only the subject of Mathematical educational tool there is almost too much freedom. First thing I did was plan a group to target the product towards and give a general purpose. Now given that most students only take a class for one year k-12 and one term for college i figured it is unreasonable to imagine students buying vr headsets for a class. Knowing this, I decided to make the application targeting educators and allowing them to make there own diagrams and materials for classes. Being the maker i am i also wanted this application to be open source and modular. Along the lines of modular billy i knew i couldn’t reasonably program my own math engine so i planned on integrating a common math engine. Originally my plan was to integrate geogebra and find someway to in real time export the graphics to unity. The more I looked into this i found it was too complicated to be worth it. The fact that i’m using unity means that graphics are not a concern it’s computations. Because of this I settled on using a computer algebra system(CAS) such as MATLab, maple, and maxima. Given that matlab seems to be a bit more popular and is written in c++ i decided to use it.
First things first, for programing a VR Application i would probably want a VR headset to test it out. This part of the project is the most specific for my casinario but the decisions made are important to the project. Although this application is going to be in VR it isn’t hard to imagine its usefulness as an AR application. With this in mind it was just my luck that a week before researching my options in VR headset windows had released a line of VR headsets in their mixed reality lineup. With the research i made it seemed these headsets were planned to help developers on a budget program for there hololens which is a $2000 AR headset they sell for developing applications. Aside from these headsets there are two main VR headsets. The htc vive and the oculus rift. Both of which are nice headsets but are ment for gaming and having a space to set up external tracking towers. The windows mixed reality uses inside out tracking witch means there are two cameras on the front of the headset and they use these cameras to find the headsets and controllers positions in the room. For this project i bought the HMD Odyssey version of the windows mixed reality headsets. Although this headset is less targeted for development i have many more vr projects planed and i would like to have the most immersive headset for these. Now to address the computer. I had a powerful enough computer but there was a problem with the connections. The computer did not have any usb 3.0 witch was necessary for the headset to work or bluetooth which is needed for the controllers. To fix this i bought a bluetooth dongle and a usb 3.0 pci chip and pci riser.
Given that i’m going to use a CAS for my computations i knew i needed to first focus on getting the graphics figured out. I used a asset i would recommend to everyone programing for vr called VRTK (has a flying pig for a logo). This asset gave me all the scripts for basic interactions such as grabbing, scaling, clicking buttons, and walking. Why reinvent the wheel. I started by drawing up some ideas for VR work spaces and prototyped them in vr. My first idea was to have a vr table and have the menu built into the table but after some testing i found that design troubled sum because it restrict the users movement and made interactions less intuitive. My next redesign was to make a floating menu that is retractable so it won’t get in the way of diagrams.
Reference design examples
Graphing using unity was an interesting problem to tackle. I had found tutorials and have seen applications that use two main ways to graph using unity. There is the use of objects, and the use of particles to make graphs. These practically work the same but are programed differently. A gameobject is the simplest. All that needs to happen is a parametric equation is used and an object is spawned for every parametric variable u and v. the position vector of the game objects are then changed according to the parametric equation. Particle effects use different commands for changing positions and particles aren’t particularly treated as separate entities but instead the system of particles is treated as the same. A graph made from a particle system seems to be the best option of these too as it takes less processing power and therefore can have a higher resolution before it starts to lag a computer. However, there is another way that i wish to experiment with before committing to a type of graph. Unity has a set of functions that can allow you to procedurally generate a mesh. I want to try using this to make a graph by setting a vertisy resolution the same way as the previous methods then connect the vertices and create a solid mesh. My concerns with this method is that an animated graph will be drastically more resource heavy than the other methods. If this is the case i may settle on using particle graphs in the vr program but allow for 3d printable models of graphs to be made by procedurally generating a mesh.
Using a CAS fixes the largest limitation of the program so far, you can’t do intuitive math!! Using just unity without some other program for mathematics only allow for parametric equations typed in c to be used. Integrating a CASE allows for various function types and has many different processes that already contain standard names. MATLab has systems in place for integrating it with external programs. When i do integrate MATLab with the program i hope to allow for two types of inputs, a pure MATLab input on the computer screen, and then a gui input in vr and on the desktop screen.
The admin menu is going to be the specific popup menus someone creating an environment gets. It will have all the settings and functions for modifying and creating an environment, below are the settings to be added. This admin page will also have the ability to set who can see the environment and how to share it. I plan on adding videotaping and live streaming functionality along with control over what medium can see such as desktop vr, phone vr, 2d computer view, and phone AR.
|Create plot||Generates a graphic using CAS inputs|
|Set phone vr||Places the position were phone vr will remain stationary|
|Export plot||Can export plot to 3D object, data, or AR video (for AR video an image can be selected as the spawn)|
|Create slider||Create interactive slider for students|
The student menu will have tools for students such as zoom, give value, find value, and a simple calculator. The student manure can be restricted by the admin and functions can be added to the student menu.
|Zoom in/ out||Allows for plot to be scaled|
|Read value||Shows values next to controller|
|calculator||Opens calculator menu|
|Find value||Places highlight on value in plot|
Instances are the individual rooms are saved files for a room. I plan on making a menu in vr for opening instances and posable an online sharing for instances. The instances are what’s shared with students and are videotaped. I plan on adding the following sharing types.
|multiplayer||Allows for multiple pepapel in vr to share an instance|
|3D recording||Motion track the admin and will allow vr users to walk around the recording|
|2D recording||Can record from vr camera or from desktop screen with desktop functions|
|2D live stream||Similar to 2D recording but is shared through a live stream service|
|AR live stream||Allows for recording of admins voice and admins manipulation of A 3D object|
|Interactive problem||Exported instinct that can only be opened in student mode|
|Editors file||Exported instance that can be opened in both student and admin mode|
Virtual Reality CAS
This VR Graphing program was made for my high school senior project. I also entered it into the Intell science fair and won an honorable mention