Nach positiver Absolvierung der Lehrveranstaltung sind Studierende in der Lage unter Verwendung des PyTorch-Frameworks eigene neuronale Netzwerkmodelle für die Verarbeitung natürlicher Sprache zu entwerfen, zu implementieren und zu verstehen.

Welcome to this course on deep learning for natural language processing (NLP)! This course is designed to unravel the complexities of NLP through the lens of deep learning techniques. From fundamental concepts to advanced neural network architectures, we will explore the intricacies of how deep learning models can comprehend, generate, and manipulate human language. Get ready to embark on a transformative learning journey, where theory meets hands-on applications.

The topics covered in the course include the following: 

• word vectors, word window classification, language models
• backpropagation and neural networks, dependency parsing
• recurrent neural networks and language models
  
• seq2seq, machine translation, subword models
• self-attention and transformers
• pretraining, natural language generation
• prompting, reinforcement learning from human feedback
• question answering 
• convolutional neural networks, tree recursive neural networks and constituency parsing
  
• insights between NLP and linguistics
• code generation
• multimodal deep learning
• co-reference resolution 
• interpretability and explainability

Lecture part with frontal lectures in weeks 1 to 8 (4 hours per week) from mid November to end January.

Practicals are in weeks 1 to 8 (4 hours per week).

• In week 1 to 2, you will review TensorFlow and PyTorch tutorials during the practicals with the help of the demonstrators.
• In weeks 3 to 4, you are expected to organize in groups (of 2-3 people), and each group is expected to select a paper from the list of papers in the assessment document (which will be available on the course resources section), and prepare a work plan with a timeline, to be submitted by 6pm Friday of week 6 by email to Thomas Lukasiewicz. Each group should submit an initial plan stating your assigned group number and team members, the person in charge of submitting the poster, the title of the paper selected, and a brief plan of action (equivalent to roughly half a side of A4).
• In weeks 5 to 7, each group is expected to reproduce the results in the selected paper, extending on the methodology and experiments.
• In week 8, each group presents their work in a poster, and the members of each group are orally examined.

Your final grade results from your submitted work plan, the type of extensions made on the selected paper, the submitted poster, the poster presentation, and the final oral exam.

The lectures will be kept as a block of 8 weeks (4 hours per week) from mid November to end January. 

The course is currently being assigned to the modules "Knowledge Representation and Artificial Intelligence" in the master programme "Logic and Computation" and the module "Algorithmic" in the master programme "Software Engineering & Internet Computing", pending approval. 

The course will be extended to 4 VU, pending approval.  

Oral examination at the end of the course. 

Basic Python knowledge will be an advantage.