057.021 VSC-School II Courses in High Performace Computing
This course is in all assigned curricula part of the STEOP.
This course is in at least 1 assigned curriculum part of the STEOP.

2020S, VU, 2.0h, 1.5EC, to be held in blocked form

Properties

  • Semester hours: 2.0
  • Credits: 1.5
  • Type: VU Lecture and Exercise

Learning outcomes

After successful completion of the course, students are able to

4) Shared memory parallelization with OpenMP (Open Multi-Processing)

  • explain that OpenMP has limited possibilities on high-performance computing clusters because its use is restricted to shared-memory architectures, 
  • describe the essential concepts of the OpenMP standard,
  • parallelize a program using OpenMP on shared-memory architectures,
  • ensure correct synchronization within the OpenMP program,
  • recognize the effects of data locality on the runtime of the parallel OpenMP program and, therefore, map the data at a suitable location in the memory,
  • avoid race conditions and especially data-races in OpenMP programs,
  • use suitable tools for identifying race conditions and data races,
  • identify any errors in OpenMP programs without and with help of the tools mentioned above, and
  • fix errors identified in OpenMP programs.

5) Introduction to Hybrid Programming in HPC (High-Performance Computing)

  • designate the key limitations of pure distributed-memory parallelization using MPI on modern high-performance computing clusters consisting of a combination of distributed-memory and shared-memory architectures,
  • apply several concepts of hybrid programming, i.e. different ways of combining distributed-memory (e.g., via MPI) with shared-memory parallelization (e.g., using OpenMP or MPI-3.0 shared-memory),
  • explain the main advantages and disadvantages of the concepts of hybrid programming,
  • select a suitable hybrid programming method depending on the specific situation,
  • optimize the runtime and memory requirements of a program through hybrid parallelization,
  • identify any errors in parallel programs, and
  • fix errors identified in parallel programs.

 

Subject of course

4) Shared memory parallelization with OpenMP (Open Multi-Processing)

  • The most important methods of parallelizing programs on high-performance computing clusters (distributed-memory vs. shared-memory architectures), their main advantages and disadvantages, and the selection of the most appropriate method, adapted to the architecture and specificity of the program in question,
  • the essential concepts of the current standard for OpenMP,
  • the program implementation.  

5) Introduction to Hybrid Programming in HPC (High-Performance Computing)

  • The main limitations of pure distributed-memory parallelization using MPI on modern high-performance computing clusters, which consist of a combination of distributed-memory and shared-memory architectures,
  • the various concepts of hybrid programming, i.e. different possibilities of combining distributed-memory (for example by means of MPI) and shared-memory (for example by means of OpenMP or by means of MPI-3.0 shared-memory) parallelization, and their most important advantages and disadvantages,
  • the program implementation.

 

Teaching methods

4) Shared memory parallelization with OpenMP (Open Multi-Processing)

Lecture about:

  • the most important methods of parallelizing programs on high-performance computing clusters (distributed-memory vs. shared-memory architectures), their main advantages and disadvantages, and the selection of the most appropriate method adapted to the specificity of the program in question und the system architecture,
  • the essential concepts of the current standard for OpenMP,
  • the program implementation.

Lecture and practical exercises about:

  • at the end of each new topic, the students solve tasks that are specifically formulated to train the in the last section learnt concepts by practically implementing OpenMP programs,
  • it is up to the students if they want to program independently or in teams of two,
  • discussion of the participants among themselves as well as in a one-to-one interview with the course instructors.

5) Introduction to Hybrid Programming in HPC (High-Performance Computing)

Lecture about:

  • The main limitations of pure distributed-memory parallelization using MPI on modern high-performance computing clusters, which consist of a combination of distributed-memory and shared-memory architectures,
  • the various concepts of hybrid programming, i.e. different possibilities of combining distributed-memory (for example by means of MPI) and shared-memory (for example by means of OpenMP or by means of MPI-3.0 shared-memory) parallelization, and their most important advantages and disadvantages,
  • the program implementation.

Lecture and practical exercises about:

  • at the end of each new topic, the students solve tasks that are specifically formulated to train the in the last section learnt concepts by practical creation and optimization of parallel hybrid (MPI + X) programs,
  • it is up to the students if they want to program independently or in teams of two,
  • discussion of the participants among themselves as well as in a one-to-one interview with the course instructors.

 

Mode of examination

Immanent

Additional information

The course is devided into individual blocks. Individual registration for each block should be done via the homepage of the course.

Lecturers

Institute

Course dates

DayTimeDateLocationDescription
09:00 - 16:4004.05.2020 - 05.05.2020 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Shared memory parallelization with OpenMP (Open Multi-Processing)
09:00 - 17:0017.06.2020 - 18.06.2020 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Introduction to Hybrid Programming in HPC (High-Performance Computing)
VSC-School II Courses in High Performace Computing - Single appointments
DayDateTimeLocationDescription
Mon04.05.202009:00 - 16:40 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Shared memory parallelization with OpenMP (Open Multi-Processing)
Tue05.05.202009:00 - 16:40 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Shared memory parallelization with OpenMP (Open Multi-Processing)
Wed17.06.202009:00 - 17:00 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Introduction to Hybrid Programming in HPC (High-Performance Computing)
Thu18.06.202009:00 - 17:00 FH Internet-Raum FH1 (TU Wien, Wiedner Hauptstraße 8-10, ground floor, red area)Introduction to Hybrid Programming in HPC (High-Performance Computing)
Course is held blocked

Examination modalities

The performance review takes place by participation in the courses and by reviewing the submitted program examples.

Course registration

Registration modalities:

This course will be held in blocks, please see http://vsc.ac.at/training.

Registration for each of the blocks should be done within the corresponding course at http://vsc.ac.at/training.

Contact for this coursevsc-seminar@list.tuwien.ac.at

Curricula

Study CodeSemesterPrecon.Info
ALG For all Students

Literature

No lecture notes are available.

Previous knowledge

Previous knowledge for the individual blocks: 

4) Shared memory parallelization with OpenMP (Open Multi-Processing)

5) Introduction to Hybrid Programming in HPC (High-Performance Computing)

Preceding courses

Miscellaneous

Language

English