Tools Used
- Unreal Engine
- Blueprints
- Blender
- Python
- Substance Designer
- Git
- YouTrack Issue Tracker
Project Description
The goal of the master thesis was the development of a HoloLens 2 application, where medieval castle life is simulated in augmented reality. The player can choose between different virtual buildings, transform, view and place these buildings in the real world. In order to convey historical information to the player, a presentation mode was created to shows a description and a picture of the historical building. Tooltips indicate important properties of the building.
When developing the application, special attention was paid to UX design. For this reason, different interaction options have been implemented to make operation easier. For example, some functions of the hand menu can be called up by voice commands, while the use of spatial audio should better embed the holograms in the real world. Eye and hand interactions encourage the player to engage with the application.
To avoid to many buttons of all ten different buildings in the hand menu, the main UI interface, with buttons for the 10 available buildings, the building bar was developed. On the building bars all different buildings are presented, separated by category. This allows the user to get an idea of the appearance of the building before placing it, the user doesn’t have to guess the appearance from 2D images.
A simple behavior tree was created to define the behavior of the AI. Spawned residents look for a job at other buildings that may already have been placed or roam around the level if no workplace has been found yet.
My Role
Using Blueprints allowed quick development and iteration of different gameplay features. Buildings are based on historical reference and modelled in Blender with performance in mind. Trim Sheets, which were created from scratch in Substance Designer, allowed for further optimization and modular building pieces. A Blender python addon was developed to speed up repetitive tasks and quickly export the modular buildings into Unreal Engine. YouTrack was used for keeping track of different issues, git with GitLab was used as version control system.
Notable Features
- Building Bars for easy selection and display of the 10 different buildings.
- Presentation Mode, which provides the user with historical info and a reference picture, also allows the inspection of the selected building.
- Different interaction methods:
- The hand can be used to interact with the buildings (e.g. transformation or reveal the top floors to allow a look inside)
- Use speech input to access functionality from the Hand Menu
- Spatial audio and visual effects help integrate the holograms into the real world
- Hand Menu with 3 different categories for gameplay, game and general settings. Can be transformed into a Near Menu and anchored into the world.
- Simple Behavior Tree with custom Decorators/ Services to control the spawned AI.
- Custom Blender Addon for workflow/ pipeline improvements.
- Modular building pieces to create different buildings fast.
Project Difficulties & Solutions
- No NavMesh could be created based on the spatial map. The reason is that the spatial map generated without collision. Possible solutions were searched for in the C++ source code of the HoloLens plugin, but no solution has been found yet. To allow AI movement nevertheless, a large collision box was attached beneath the main building. This way, Unreal Engine can generate a NavMesh.
