This course analyses the essence of DataFlow SuperComputing, defines its advantages and sheds light on the related programming model. DataFlow computers, compared to ControlFlow computers, offer speedups of 20 to 200 (even 2000 for some applications), power reductions of about 20, and size reductions of also about 20. However, the programming paradigm is different, and has to be mastered. The course explains the paradigm, using Maxeler as an example, and sheds light on the ongoing research, which, in the case of the speaker, was higlhy influenced by four different Nobel Laureates: (a) from Richard Feynman it was learned that future computing paradigms will be successful only if the ammount of communications is minimized; (b) from Ilya Prigogine it was learned that the entropy of a computing system would be minimized if spatial and temporal data get decoupled; (c) from Daniel Kahneman it was learned that the system software should offer options realted to approximate computing; and (d) from Andre Geim it was learned that the system software should be able to trade between latency and precision. The course teaches the programming model details of the Maxeler MultiScale Data Flow Element (DFE) approach, with lots of hands-on examples. It also includes the advanced programming techniques for higher speedup and precision, and for lower power and complexity. The course also examines the Google TPU approach, comparatively with the Maxeler DFE approach. To shed more light on the issues related to speed, precision, power, and complexity, the course also shows relevant aspects of the desing of Control Flow machines like ManyCore or MultiCore.
About the Speaker:
Prof. Veljko Milutinovic (1951) received his PhD from the University of Belgrade, spent about a decade on various faculty positions in the USA (mostly at Purdue University), and was a co-designer of the DARPAs first GaAs RISC microprocessor. Later, for almost 3 decades, he taught and conducted research at the University of Belgrade, in EE, BA, MATH, and PHYS/CHEM. Now he serves as the Chairman of the Board for the Maxeler operation in Belgrade, Serbia. His research is mostly in datamining algorithms and dataflow computing, with the emphasis on mapping of data analytics algorithms onto fast energy efficient architectures. For 7 of his books, forewords were written by 7 different Nobel Laureates with whom he cooperated on his past industry sponsored projects. He has over 40 IEEE journal papers, over 40 papers in other SCI journals (4 in ACM journals), over 400 Thomson-Reuters citations, and about 4000 Google Scholar citations. Short courses on the subject he delivered so far in a number of universities worldwide: MIT, Harvard, Boston, NEU, Columbia, NYU, Princeton, Temple, Purdue, IU, UIUC, Michigan, EPFL, ETH, Karlsruhe, Heidelberg, University of Vienna, Vienna Politechnical University, Napoli, Salerno, Siena, Pisa, etc. Also at the World Bank in Washington DC, BNL, IBM TJ Watson, Yahoo NY, ABB Zurich, Oracle Zurich, etc.
About the TA of the course:
Milos Kotlar (1993) is a PHD student of Computing at the University of Belgrade, with years of professional experiance at ABB in Zurich, Switzerland. He is the author of a number of dataflow implementations in Machine Learning and Tensor Calculus.
Accompanying Textbooks and Journal Papers:
Milutinovic, V., et al, Guide to DataFlow SuperComputing, Springer, 2015 (one textbook) and 2017 (two textbooks).
Hurson, A., Milutinovic, V., editors, Advances in Computers: DataFlow, Elsevier, 2015 (one SCI textbook) and 2017 (two SCI textbooks).
Trifunovic, N., Milutinovic, V. et al, "The AppGallery.Maxeler.com for BigData SuperComputing,"
Journal of Big Data, Springer, 2016.
Trifunovic, N., Milutinovic, V. et al, "Paradigm Shift in SuperComputing: DataFlow vs ControlFlow," Journal of Big Data, 2015.
Milutinovic, V., "The HoneyComb Architecture," Proceedings of the IEEE, 1989.
Jovanovic, Z., Milutinovic, V., "FPGA Accelerator for Floating-Point Matrix Multiplication,"
The IET Computers and Digital Techniques Premium Award for 2014, Volume 6, Issue 4, 2012 (pp. 249-256).
Flynn, M., Mencer, O., Milutinovic, V., at al, Moving from PetaFlops to PetaData, Communications of the ACM, May 2013.
Trobec, R. Vasiljevic, R., Tomasevic, M., Milutinovic, V., et al, "Interconnection Networks for PetaComputing," ACM Computing Surveys, November 2016.
