Nach positiver Absolvierung der Lehrveranstaltung sind Studierende in der Lage unter Verwendung des PyTorch-Frameworks eigene neuronale Netzwerkmodelle für die Verarbeitung natürlicher Sprache zu entwerfen, zu implementieren und zu verstehen.

Welcome to this course on deep learning for natural language processing (NLP)! This course is designed to unravel the complexities of NLP through the lens of deep learning techniques. From fundamental concepts to advanced neural network architectures, we will explore the intricacies of how deep learning models can comprehend, generate, and manipulate human language. Get ready to embark on a transformative learning journey, where theory meets hands-on applications.

The topics covered in the course include the following: 

• word vectors, word window classification, language models
• backpropagation and neural networks, dependency parsing
• PyTorch
• recurrent neural networks and language models
• seq2seq, machine translation, subword models
• self-attention and transformers
• pretraining, natural language generation
• Hugging Face transformers
• prompting, reinforcement learning from human feedback
• question answering 
• convolutional neural networks, tree recursive neural networks and constituency parsing
• insights between NLP and linguistics
• code generation
• training large language models
• multimodal deep learning
• co-reference resolution 
• interpretability and explainability

Lecture part with frontal lectures in May and June (8 weeks, 2 hours per week).

Practicals with solving exercises are in May and June (8 weeks, 1 hour per week).

It has been approved that this course will be replaced by the 4 VU course "192.039 Deep Learning for Natural Language Processinge" of the same contents, assigned to the modules "Knowledge Representation and Artificial Intelligence" in the master programme "Logic and Computation" and the module "Algorithmic" in the master programme "Software Engineering & Internet Computing".    

Exercises have to be solved by each student individually.

ECTS Breakdown:

16h Lectures in the lecture hall
 8h Exercises in the lecture hall
16h Preparation at home
16h Solving exercise problems
18h Preparation for final oral exam
  1h Poster presentation and final oral exam
-------------------------------------------------------------------------------
75h = 3 ECTS Total workload 

Solving exercise problems and oral exam at the end of the course. 

Your final grade results from your solutions to the given exercise problems and your performance at the final oral exam.

Basic Python knowledge will be an advantage.