Mathemagenic were grant winners of the VocTech Activate fund run by the Ufi. The proposed product is CritiCal, a Collaborative Learning Platform. Immersive learning with Virtual Reality is tricky, because it is designed for an individual experience. Sure, you can design multi-user VR experiences, but when you need groups of learners to collaborate, you need them to be more engaged with each other. Mathemagenic wanted to design an Immersive Learning Experience that allows groups of users to work together on problems, and learn through play.
Weald Creative partnered with Mathemagenic to help them create the initial prototype.
The proposed product was an Escape Room activity, with groups of learners taking on the roles of a crack team of investigators, attempting to outsmart a nefarious and omnipotent AI technology. They are sent to the home of a family who's robot assistant has gone crazy and attempted to kill the family. You must disable the robot and work out how and why the robot has behaved this way. By solving puzzles and learning about AI ethics, the team can finally discern the robots true incentives.
We knew that we wanted the groups of learners to work as a team, and be in an immersive space, we we collaborated with Lewes College who had an Immersive room.
The room was built with three blank walls and projection technology that brought the space to life. Users were surrounded by the content, but could also use the walls like a touchscreen to interact with the digital content.
The Immersive Room software supported the use of Game Engines like Unity, so we were excited to be able to push the boundaries of what could be achieved with this technology.
We also wanted the learners to use natural language to have conversations with the main character in the story, who would help with the puzzles and provide learning material. There wasn't an easy way to do this through the Immersive Room software, so we decided to build a second-screen mobile app that would act as an input device for voice.
There were three aspects to the technology that we had to explore:
The Immersive Room ran software and provided templates for coordinators to create interactive learning material really easily. These templates were limited, however, so we wanted to create a much slicker, dynamic experience in a game engine. Thankfully, the software supported Unity through an SDK, and we began building our prototype using its examples. The SDK was very robust, and we were able to build the escape room locally using its Room Simulator.
We built a web app that the learners could use as an input device for the Voice Assistant. They would tap a button to start talking and tap again to stop. The voice recording would then be uploaded to the AI and be interpreted. The AI would then send back the voice response with information based on the context of the activity. The Web Socket Server was ran in Google Cloud Services. The web app was a React Next JS app, also running on Google Cloud.
The Cloud AI was built with Google DialogFlow CX API, which is the main software that Google runs its Voice Assistant devices. It is an early agentic AI technology that can be tightly controlled to handle voice intents, and is perfect for activities like games, where you can't afford the AI to hallucinate. We developed a conversational model with branching narratives, and we could also provide contextual information to the model from the game engine via input data.
Together with Mathemagenic we devised and produced a series of Escape Room activities and puzzles that were built as part of the prototype. These included mini-quiz panels, word puzzles, and fast-paced action activities that were unique to the Immersive Room experience.
Initial tests of the Escape Room Prototype at Lewes College were promising, and the Immersive Room technology has lived up to expectations. The websocket server proved to be a great way to provide natural language input.
The team at Mathemagenic are now building the Alpha version of the CritiCAL product without Weald Creative. We wish them all the best of luck with the project.
Creating Immersive Learning experiences doesn't have to involve headsets. Although Virtual Reality experiences are incredible, and the sense of immersion cannot be beaten, they are a personal experience that can be a little uncomfortable. The fact remains that a lot of people just don't get on with wearing headsets. They find them claustrophobic, and can induce motion sickness. The best Immersive Experience is one that you can share with a group, that is grounded in the real world.
Immersive Rooms or popup Dome Projection environments are perfect. They can be installed quickly, and are a cost-effective way to host Immersive Learning material without the technical overhead of VR headsets, and servers.
Get in touch if you'd like to discuss Immersive Learning with your own personal Immersive Room. You may want to convert a spare meeting room into a Learning Hub, or host a Projection Dome in your lobby as part of a Careers Fair.
We can help you plan the perfect solution.
