Postdoc in High Performance Computing (Development of Parallel Numerical Algorithms for Nano-Device Simulation)

The modeling of nano-scale devices such as field-effect transistors or memory cells consists of predicting and optimizing the performance of not-yet fabricated components to guide on-going experimental activities. While really useful to the semiconductor industry, this type of research is computationally very intensive. It relies on a massive parallelization of the underlying work load. Typically, thousands of linear systems of equations, eigenvalue problems, and singular value decompositions must be solved for matrices with size comprised between 500 and 100,000. Most of the calculations are executed on GPUs, with the storage of intermediate data on CPUs.

The Computational Nanoelectronics Group of ETH Zurich recently started implementing a novel device simulator called QuaTrEx that exploits the latest developments in high performance computing to run on today's and future exascale computers. QuaTrEx is the successor of OMEN, which was awarded the ACM Gordon Bell Prize in 2019. It is written in Python based on the NumPy, SciPy, CuPy, and mpi4py modules. It makes use of optimized BLAS, LAPACK, and FFT libraries for both CPU and GPU architectures. Through CuPy, custom GPU (CUDA/HIP) kernels are seamlessly integrated to bypass bottlenecks that are not amenable to the aforementioned optimized libraries.

The Platform for Advanced Scientif Computing (PASC) in Switzerland intends to position Swiss computational sciences in the emerging exascale-era. For that purpose, it supports projects whose goal is to port existing or novel scientific applications to cutting-edge hardware architectures, in particular the novel Alps supercomputer located at CSCS.

The Computational Nanoelectronic Group will be supported by PASC from 2025 till 2027 to further develop the QuaTrEx code and make it a state-of-the-art device simulator. To accomplish the required work, we are looking for a post-doctoral fellow who will collaborate with the existing development team (two post-docs and four PhD students).

QuaTrEx relies on the Non-equilibrium Green’s Function (NEGF) formalism and on scattering self-energies (SSE). Dedicated parallel numerical algorithms have been developed to solve the NEGF equations on multiple CPUs and GPUs, taking advantage of the specific block-tri-diagonal sparsity pattern of the Hamiltonian (H) and Overlap (S) matrices typically encountered in device simulations. The parallelization scheme of QuaTrEx has been optimized to minimize the communication time and volume of data exchanged. Finally, the SSE calculation has been accelerated by grouping small matrix-matrix multiplications together through batching. Significant speed-ups have been obtained by combining these different techniques, with still ample room for improvement.

In particular, the NEGF and SSE methods build upon several computational kernels that are very similar to each other and whose performance can be further enhanced. They involve matrix-matrix multiplications of either “sparse times sparse,” “sparse times dense,” or “dense times sparse” type, with filling densities going from less than 5% up to more than 90%. These computations may be performed on blocks of the H, S, or SSE matrices with sizes in the order of hundreds to thousands or on full-device matrices, whose sizes range from tens of thousands to even millions. In case of small sizes, a single GPU is sufficient, whereas multiple GPUs are needed in large configurations. Preliminary tests revealed that libraries provided by vendors, e.g., cuSPARSE, or by the community, e.g., DBCSR, do not lend themselves optimally to our matrix-matrix multiplications, thus limiting the overall computational efficiency of QuaTrEx.

As part of this PASC project:

• An open-source matrix-matrix multiplication library that is specifically tailored to our problems and that works on single and multiple GPUs from different vendors should be developed
• A suitable strategy should be devised to store our sparse matrices and support rapid block extractions, dense matrix multiplications, or Fast Fourier Transforms (FFT), depending on the task at hand
• The algorithms to solve the NEGF equations should be generalized to make them capable of handling matrices with structured sparsity patterns going beyond block-tri-diagonal configurations, e.g., arrow-head matrices, that can be found, for example, in climate modeling applications

All codes will be made freely available to the scientific community through the GitHub platform.

Applicants should have a PhD in Computer Science, HPC, Electrical Engineering, or in a similar field. They should:

• possess a demonstrated expertise in parallel numerical algorithms, high performance computing, and scientific programming (C/C++, CUDA, Python, · · ·)
• enjoy collaborating with other researchers developing the same application as them and publishing their results in journals and conference proceedings
• be ready to supervise junior students

Note that pre-existing experience with ab initio methods, quantum transport, and/or density functional theory is an advantage, but not mandatory.

We offer an exciting and challenging activity in a team of highly motivated physicists, electrical engineers, and computer scientists and a salary according to the standard of ETH Zurich for post-docs. The duration of the post-doc can be up to three years. The participation in international conferences and the collaboration with industry and academia is strongly encouraged and supported.