Advancing Automated Annotation of OCT B-Scans using State-of-the-Art Natural Language Processing and Deep Learning Techniques

The goal of this master thesis is to explore and implement advanced deep learning models to improve the automatic generation of accurate and detailed annotations for OCT B-scans. Leveraging recent advances in NLP and multimodal learning, the student will aim to create a model that better understands and describes the medical structures and irregularities present in the B-scans.

  • Background

    Optical Coherence Tomography (OCT) is a critical imaging modality used in ophthalmology to capture detailed cross-sectional images of the retina. These images, or B-scans, are essential for diagnosing various retinal conditions. Annotations by eye doctors, which describe key structures and abnormalities in these scans, are crucial for accurate diagnosis. However, the process of manual annotation is time-consuming and subjective.

    Scope of Work:

    1. Literature Review: Review recent advancements in NLP and vision-language models, with a focus on transformer architectures, multimodal learning, and their applications in medical imaging.

    2. Data Preparation: Work with the existing dataset of OCT B-scans and annotations. Preprocess the images and text to ensure compatibility with modern deep learning models.

    3. Model Development: Implement and fine-tune state-of-the-art transformer-based models (e.g., GPT-4, Vision-Language Transformers) to predict annotations from the B-scans. Explore multimodal architectures that effectively combine image features with text generation.

    4. Model Training and Evaluation: Train the models on the existing dataset and evaluate their performance using metrics such as BLEU score, ROUGE, and medical relevance metrics specific to the domain of ophthalmology.

    5. Optimization: Experiment with different model architectures, loss functions, and training strategies to optimize the performance.

    6. Validation: Validate the model's performance on unseen data and obtain feedback from medical professionals to assess the clinical relevance of the generated annotations.

    7. Documentation and Presentation: Document the entire process, including challenges, solutions, and results. Prepare a final thesis report and thesis presentation.

  • Expected outcome

    The project aims to develop an accurate and useful AI model for the automatic annotation of OCT B-scans. The student would ideally demonstrate improvements over previous models, particularly in generating detailed and medically relevant annotations.

  • Further Information

    • 30% theory, 70% implementation
    • Master's thesis, 1 student
    • Prior knowledge recommended in
      • Machine learning,
      • Deep learning,
      • and/or natural language processing

     

Have we aroused your interest?

I am interested in the student research project Advancing Automated Annotation of OCT B-Scans using State-of-the-Art Natural Language Processing and Deep Learning Techniques and would like to find out more. learn more.