Virtual Reality as 3D Ground Truth Generator for AI, Machine Learning and Deep Learning

Training effective Artificial Intelligence (AI) algorithms today often requires large amounts of ground truth data. Typically, this is a laborious, costly and time-consuming process often requiring manual adjustment. These problems can be overcome by combining AI with Virtual Reality (VR): an emerging technology with various applications in medicine, training, business and simulation. VR creates artificial environments that often resemble our real world.This project aims at exploring the potential of VR as 3D ground truth generator for state-of-the-art Machine Learning (ML) and Deep Learning (DL) Algorithms.

Goals
In this project we aim to achieve the following goals:

  • Using VR engines to generate ground truth data of city scenes including buildings, roads, sidewalks, cars and pedestrians
  • Using the newly, virtually generated ground truth data to train state-of-the-art machine (ML) and deep learning (DL) algorithms to detect and classify objects of the city model
  • Apply and benchmark the trained ML and DL algorithms to predict city objects in real-world city images

Tasks

  • Explore current VR engines and their potential to build up VR city models
  • Build up large and detailed VR city models as ML/DL ground truth using VR engines
  • Investigate ML and DL methods and algorithms to detect and classify city objects in images
  • Use the VR-engine generated ground truth data to train state-of-the-art ML and DL algorithms to detect and classify city objects
  • Apply the trained ML and DL models to predict city objects in real-world city images

Requirements

  • Fluency in some high-level, interpreted programming language (e.g. Python or Matlab)
  • Previous knowledge with VR engines advantageous (e.g. Unity or Unreal)
  • Some previous knowledge with ML and DL concepts
  • Some previous knowledge of DL and ML frameworks and libraries (e.g. tensorflow, PyTorch, caffe, mxNet, scipy)
  • Familiarity with Windows and Linux development environments

Kind of Work
50% Research, 20% Software Development, 30% Benchmarking

Requirements
Machine Learning + Deep Learning Basics, Python, Matlab

Time & Effort
Master’s Thesis, 1-2 Persons

Contact
fabian.schenkel@scs.ch

SCS - Studienarbeiten