Training effective Artificial Intelligence (AI) algorithms today often requires large amounts of ground truth data. Typically, this is a laborious, costly and time-consuming process often requiring manual adjustment. These problems can be overcome by combining AI with Virtual Reality (VR): an emerging technology with various applications in medicine, training, business and simulation. VR creates artificial environments that often resemble our real world.This project aims at exploring the potential of VR as 3D ground truth generator for state-of-the-art Machine Learning (ML) and Deep Learning (DL) Algorithms.
Goals
In this project we aim to achieve the following goals:
- Using VR engines to generate ground truth data of city scenes including buildings, roads, sidewalks, cars and pedestrians
- Using the newly, virtually generated ground truth data to train state-of-the-art machine (ML) and deep learning (DL) algorithms to detect and classify objects of the city model
- Apply and benchmark the trained ML and DL algorithms to predict city objects in real-world city images
Tasks
- Explore current VR engines and their potential to build up VR city models
- Build up large and detailed VR city models as ML/DL ground truth using VR engines
- Investigate ML and DL methods and algorithms to detect and classify city objects in images
- Use the VR-engine generated ground truth data to train state-of-the-art ML and DL algorithms to detect and classify city objects
- Apply the trained ML and DL models to predict city objects in real-world city images
Requirements
- Fluency in some high-level, interpreted programming language (e.g. Python or Matlab)
- Previous knowledge with VR engines advantageous (e.g. Unity or Unreal)
- Some previous knowledge with ML and DL concepts
- Some previous knowledge of DL and ML frameworks and libraries (e.g. tensorflow, PyTorch, caffe, mxNet, scipy)
- Familiarity with Windows and Linux development environments
Kind of Work
50% Research, 20% Software Development, 30% Benchmarking
Requirements
Machine Learning + Deep Learning Basics, Python, Matlab
Time & Effort
Master’s Thesis, 1-2 Persons
Contact
fabian.schenkel@scs.ch