Introduction
Exascale computing represents a significant advancement in High-Performance Computing (HPC), unlocking unprecedented opportunities to transform computational modelling. In this context, ab initio-based materials modelling codes are in the ideal situation to take advantage of this revolution, which will provide all the resources to enable more accurate calculations, larger size scales and high-throughput exploration of data sets for the discovery of novel materials. However, the distinguishing features of these new supercomputers include increased heterogeneity in their architectures, often incorporating specialised accelerators designed for specific applications1. To fully realise the potential of the exascale era, developers of ab initio electronic structure codes are investing efforts in exploiting the advanced capabilities of the new supercomputers2. Some efforts include the development of low-scaling algorithms for existing methods as well as developing efficient shared libraries for the most critical and computationally expensive parts of the codes. Some examples of popular libraries are; ELPA3, an efficient eigensolver for petaflop (and exascale) systems, libxc4, a library of exchange and correlation functionals, GreenX5, an open-source library that supports exascale implementations of Green’s-function-based methodologies, SIRIUS6, a domain-specific library for electronic structure calculations, ELSI7, an open infrastructure for electronic structure solvers, and PEXSI8,9, a fast method for electronic structure calculation based on Kohn-Sham density functional theory (DFT). Many efforts are also focused on the development of workflow managers and job schedulers for high-throughput computations (HTC)10, some examples are AiiDA11, the Atomic Simulation Environment (ASE)12 and Atomic Simulation Recipes (ASR)13, or FireWorks14. With all these active developments in software and hardware, co-design is also an important effort that is bound to play a crucial role in an efficient transition to the (post)exascale era.
Co-design is intended to facilitate the paradigm shift by concurrently and cooperatively developing both, hardware and software15. There are some tangible examples of co-design efforts focused on atomic-scale simulations. For example, while the Gromacs molecular dynamics (MD) code was ported to GPUs, NVIDIA also introduced stream priority bits in their hardware, which eventually benefited the communications and led to better code performance16. Another interesting example is the Anton supercomputer, which was originally designed to efficiently run classical MD simulations17. For this purpose, and being one of the most relevant applications and family of applications with more users in the HPC community, the performance of ab initio electronic structure codes has to be systematically profiled on new infrastructures. Therefore, effective communication between hardware engineers, software developers and users is especially critical during the systems’ design process. However, the extreme complexity of these codes, being many of them collaboratively developed among large research teams with rotating staff and for many decades, makes these tasks quite challenging. Therefore, it is always important to set a visible milestone in the current software status and try to achieve the optimum performance of novel HPC features at the earliest possible stage, when low-level (i.e. bit-level or compiler) adaptations are still possible. Otherwise, benchmarking would only be possible once the development of the system approaches production capabilities. This problem can primarily be addressed using simplified models, such as mini-apps, which are proxies of full code executions18. An overview of the co-design workflow and the mini-app suite presented in this paper is shown in Figure 1.
Figure 1. Overview of the co-design workflow for NOMAD CoE flagship applications.
The kernel’s names are listed in order in relation to the application’s logos.
Mini-apps strive for a simple compilation and small computing times, enabling a rapid iteration of code and leaving room for low-level improvements in hardware at experimental platforms. A mini-app comprises a small fraction of the code length and complexity with respect to its parent application while it retains the primary performance-intensive aspects. Therefore, mini-app benchmarking can inform the implementation of new HPC systems more effectively. Ideally, this would be a continuous process throughout the implementation and in the initial design phase of the systems. Mini-apps have the potential to support our efforts in co-design as the performance metrics will be easily transferable between selected HPC systems and pre/post-exascale prototypes more effectively. Moreover, mini-apps can also serve to inform developers of the parent applications of the feasibility of potential directions for future developments. An illustrative example of a mini-app suite is the one developed by the arch project, which unified several physical simulations such as heat transfer, gravity or hydrodynamics problems within a consistent coding practice under a common infrastructural layer19. Also, contributors to the Mantevo project have focused on developing tools to accelerate and improve the HPC by providing application and library proxies. This suite includes some applications that could be useful for materials research, such as classical MD or finite elements mini-apps20. Also, a methodology paper for showing the link between full application codes and their proxies was published21. In addition to all these, and following a similar spirit, the Sustained System Performance (SSP) (and its simplified (SSSP) metric) enable performance projection that correlates with full applications from a suite of mini-apps, providing a more direct estimation of the application performance in contrast to for example the popular Linpack benchmark22.
On this basis, our article presents a suite of mini-apps developed from a set of representative ab initio electronic structure codes that are part of the Novel Materials Discovery (NOMAD) Centre of Excellence (CoE)23, all using different basis sets and aiming to be a seed of collaboration in the co-design endeavour. The objective of the project is to facilitate systematic studies and predictions of novel materials enabled by upcoming exascale computing. In the following sections, first we introduce the importance of eigensolvers in electronic structure calculations and sketches the GW approximation to serve as context and reference to present the kernels identified for each mini-app. The next section describes the methods used to profile the codes, identify the kernels and develop the mini-apps suite. Then we proceed to describe the suite and how to operate the mini-apps within the suite. In the end, we give some conclusions and motivate further research that could be carried out using the NOMAD mini-apps.
Theoretical background
The mini-apps currently present in the suite correspond to isolated kernels both from the ELPA library (version 2022.05.001), an efficient eigenvalue solver for HPC applications3,24, and from the GW implementation of the ab initio codes exciting (oxygen version)25,26, which uses all-electrons with linearised augmented plane-wave plus local orbitals (LAPW+lo), Abinit (version 9.6.2)27–29, which uses plane waves and pseudo-potentials (PW+PP), and FHI-aims (version 210716_1)30,31, which uses all-electrons with numeric atom-centered orbitals (NAOs).
Eigenvalue problems
Eigenvalue problems are common tasks in ab initio electronic structure calculations when solving the Schrödinger equation or approximations of it, such as those found in Density Functional Theory (DFT) and many-body perturbation theories (MBPT) like the GW approximation. Eigensolvers are often the main computational bottleneck in Density Functional calculations, expending in large cases the vast majority of the total computational cost and, in practice, also limiting the systems’ sizes. For researchers in the field of computational materials science, an efficient and scalable solution to the eigenvalue problem is thus of major importance. The ELPA library (Eigenvalue SoLvers for Petaflop-Applications) is designed for exascale HPC. The ELPA scalability and parallelisation capabilities make it a key tool for handling computationally intensive tasks in material discovery and design3. ELPA is a well-established solver library, and today, it has interoperability with the majority of the most widely used ab initio packages, making it an indispensable component in the computational toolkit for advancing material science research.
The GW approximation
The GW approximation of Hedin’s equations represents a significant advancement in accuracy for electronic structure calculations, for example, enhancing the prediction of band gaps in semiconducting materials32. This improvement is notable when compared to traditional Density Functional Theory (DFT) with Local Density Approximation (LDA) or General Gradient Approximation (GGA) functionals. A key element of this advancement is the introduction of the self-energy, Σ, in Hedin’s equations, which encapsulates all electron-electron interactions extending beyond the Hartree energy. This incorporation addresses the limitations in conventional DFT methods, which often underestimate electron correlation effects crucial for accurate band gap predictions.
In practical implementations, especially where computational efficiency is paramount, the self-consistent GW formalism often gives way to a simplified approach termed single-shot GW or G0W0. This approximation involves using the non-interacting Green’s function, G0, in place of the fully interacting G. The equation for G0 is given by:
Here, ψnk (r) and εnk represent the DFT eigenfunctions and eigenvalues, respectively, with n and k denoting band and k-point indices in the Brillouin zone and ω is the frequency of the Green’s function. The term η is a small, positive (negative) number for occupied (unoccupied) states, ensuring numerical stability.
Similarly, the screened Coulomb interaction W in the GW formalism is approximated by W0, described as:
In this context, ν(r, r′) denotes the bare Coulomb interaction, and ϵ−1(r, r1; ω) is the inverse dielectric function, reflecting the material’s response to electronic perturbations.
More detailed descriptions of the theory and applications of the GW method can be found in more specific publications33,34
Methods
The identification of the relevant kernels on the exciting, Abinit, FHI-aims codes and the ELPA library was done by executing and profiling benchmarks in the MareNostrum-4 supercomputer. For this we used a set of profiling tools:
Extrae35 is a tracing tool. It collects information such as execution times, MPI and OpenMP calls, and performance counter information with a Performance Application Programming Interface (PAPI)36, supplying a consistent interface and methodology for collecting performance counter information from various hardware and software components.
Paraver37,38 takes traces generated with Extrae and provides a visual interface to analyse them. Trace figures in the manuscript were produced with this software.
These tools identified relevant kernels by comparing execution times and performance metrics related to their parallel performance (load imbalance, parallel efficiency, etc.). The compute-intensive regions of the code were found to be associated with the relevant computations in the implementations above. Once kernels in the original source code are identified, the mini-apps are developed through the migration of these to a stand-alone implementation. Data dependencies are addressed through the insertion of a checkpoint right before the kernel in the original execution, which captures all the relevant variables and parameters and serves as the input file for the mini-apps. To avoid extra dependencies, such as HDF5 or other common checkpointing modules, the checkpoints are implemented through a binary I/O wrapper included in the mini-apps distribution.
The mini-apps are integrated into a CMake build system, ensuring ease of compilation across a variety of architectures in HPC environments. All mini-apps are written in FORTRAN, the language that is mostly used in their parent applications. Our testing has covered a range of machines, including those with Intel, POWER9, and AMD processors. Given the diversity of these architectures, it is certainly worthwhile to maintain this broad testing approach. The full suite of mini-apps is compiled simultaneously, streamlining the setup process as the environment remains consistent for all mini-apps during testing. This integrated build system not only simplifies the initial setup but also enhances the suite’s extensibility for incorporating new mini-apps or updating existing ones.
Mini-apps suite
The Mini-apps Suite consists of four mini-apps. Each mini-app is described below by naming the source file and subroutines containing the selected kernel and a justification for its selection. The kernels might have dependencies to other code regions; those are assumed to be included in the mini-app as well, but special care was taken not to include dependencies that are not strictly required and to reduce the mini-app source code to the minimum.
ELPA Mini-App
The ELPA mini-app isolates the trans_ev_tridi_to_band subroutine, located in the source code elpa2_trans_ev_tridi_to_band_template.F90. The ELPA library implements different methods to solve the eigenvalue problem; among those methods, we selected the two-stage tridiagonalisation, which includes an intermediate reduction to bidiagonal form before doing a reduction to tridiagonal form, as opposed to the one-stage method, which reduces the original matrix directly to tridiagonal form24. The two-stage method is normally preferred to the one-stage one in large problems and when most of the eigenvectors need to be computed. Other choices made in the method selection are the type and precision of the numbers. Our mini-app uses real numbers with double precision and employs a generated random matrix as input.
The kernel selection was done after profiling two different executions. The first was a medium-sized problem with a matrix size of 8000×8000 executed with a block size of 16 on 16 processors (see Figure 2), the second was a large matrix of size 100000×100000 executed with a block size of 24 on 192 processors. Both traces showed similar behaviour in terms of task distribution and relative weight of the routines in the trace.
Figure 2. Trace of ELPA run on a matrix size of 8000×8000 with the AVX512 kernel.
The runtime was 8.3 s in 16 one-threaded MPI processes.
After analysing the several steps to compute the two-stage method, we conclude that the most suitable function for the mini-app is trans_ev_tridi_to_band. There are some good reasons for this selection. First, this is the most computationally expensive step. Moreover, its relative weight progressively increases its predominance proportionally to the system size. Also, it does not depend on external functions, while other functions heavily rely on external libraries such as BLAS, ESSL or KML. The function is not dominated by DGEMM or communications. It is also relevant to note that the developers have put substantial efforts into improving this function on multiple architectures, making it a good target to test on experimental hardware. Today, it supports SSE, AVX(2/512), SPARC64 SSE, ARM SVE(128/256/512), BlueGene/(P/Q), NVIDIA, AMD and Intel GPUs.
exciting mini-app
The exciting mini-app isolates the subroutines expand_products and calcmwm called from the source code calcselfc.f90. This source code is called in the last step for the calculation of the self-energy in the main loop of the Brillouin zone’s integration25. The selection of the kernel was done after profiling different executions of the benchmark ZrO2 primitive cell (3 atoms) with a 2×2×2 q/k-point grid, one with all (800) bands and another with a reduced number (12) of bands, see Figure 3 and Figure 4 for the respective traces.
Figure 3. Trace of exciting full execution.
ZrO2 primitive cell (3 atoms) with 2×2×2 q/k-point grid and 800 bands. The runtime was 3214 s with 4 MPI processes and 48 threads each.
Figure 4. Trace of exciting full execution. ZrO2 primitive cell (3 atoms) with 2×2×2 q/k-point grid and 12 bands.
The runtime was 302 s with 4 MPI processes and 48 threads each.
The benchmarks were executed using 4 MPI processes with 48 OpenMP threads each. It was observed that the main loop for the Brillouin Zone was distributed in 8 tasks (1 task per k-point) among the available MPI processes. The execution time of the G0W0 implementation of exciting is therefore explained by the duration of one of these tasks. Each of these tasks is composed of the computations of the necessary quantities in the G0W0 formalism using an auxiliary mixed product basis. In all these computations, a product expansion is needed to perform the calculations, and therefore, the expand_products subroutine is a common call in all the calculations. The amount of calls to expand_products and its execution time depends on the subroutine and the system’s size. For the benchmark, it was decided to focus only on the duration and performance of the GW implementation, not taking into account the ground state section. When all the bands are used, the subroutine calcminm2 (magenta sections in Figure 3 and Figure 4) called within expand_products, used for the calculation of the self-energy, is the most time-consuming section, increasing the total runtime to 3214 s. This subroutine alone takes 85% of the G0W0 task. Moreover, considering that calcmwm takes another 5% of the G0W0 task, the exciting mini-app presented here represents 90% of the G0W0 implementation for this specific benchmark. The other 10% is devoted to the calculation of other G0W0 quantities, which also execute the subroutine expand_products and are expected to become proportionally more relevant depending on the number of bands used in the calculation.
Abinit Mini-App
The Abinit mini-app isolates one iteration of the loop over q-points for the polarizability χ0 calculation in the screening step. This is a triple loop that iterates over k-points, conduction, and valence bands, these quantities are nested forming the triple loop28. This kernel is located in the m_chi0.F90 source code inside the cchi0 subroutine which is called for every q-point different from the Γ -point. The benchmark was executed in 48 one-threaded MPI processes and the runtime was 924 s for the screening step, from which 88% executes the subroutine cchi0 over 35 q-point iterations. Figure 5 shows the trace of this execution.
Figure 5. Trace of Abinit execution of the screening calculations for 35 q-points in a ZrO2 system.
The runtime was 924 s in 48 one-threaded MPI processes.
The mini-app includes the initialisation of the bbp_ks_distrb matrix, which reduces the size of the checkpoint by around two orders of magnitude, from GB to a few MB. Then, it executes a loop over k-points and bands to calculate the chi0 matrix. Within this loop, two subroutines were identified as the main computational kernels, namely rho_tw_g and assemblychi0_sym, which together use 70% of the execution time for cchi0, while the remaining 30% are memory accesses and variable updates for each iteration in the nested loops. The selection of the kernel was done after profiling the different steps involved in the GW calculations for a ZrO2 primitive cell system, from which it was observed that the calculation of the screening W0 is the most time-consuming for the G0W0 implementation.
In this case, the extraction of only one of these q-point iterations and the portion executed in a single thread is enough to represent the full code execution, as there are no MPI communications within the selected kernel. With this selection of kernel, the mini-app reduces an execution of 15 minutes running in 48 MPI processes to 14 seconds in a single process, while it still represents the 88% of the code execution time.
FHI-aims Mini-App
The FHI-aims mini-app isolates the routine that computes the LVL triple expansion coefficients in real space, and Fourier-transforms the LVL triple coefficients, which is called gw_init_lvl_tricoeff_recip. These coefficients expand the product of wavefunctions to the so-called Auxiliarity Basis Functions (ABF). These ABF are atomically-centred, and they are constructed similarly to the mixed product basis in the LAPW framework but without the plane-wave component in the interstitial region. This basis set expansion is used to achieve efficient implementations of Hartree-Fock, second-order Moller-Plesset perturbation theory (MP2), the Random Phase Approximation (RPA), and GW within the numerically tabulated atom-centered orbitals (NAO) basis set framework31,39. This renders the mini-app as a relevant kernel for all these methods. The FHI-aims mini-app tested a code version that is no longer current and has since been optimised, particularly in the LVL part. While this older version of the code is not distributed anymore, the mini-app can still be useful to benchmark different hardware platforms in a practical computational case. The selection of the kernel was done after profiling a Si system in 48 MPI processes and a ZrO2 system in up to 384 MPI processes. Checkpoint inputs for both cases have been included in the repository. For simplicity of visualisation, the trace of the Si benchmark is shown in Figure 6.
Figure 6. Trace of FHI-aims execution of a Si system.
The runtime was 375 s in 48 one-threaded MPI processes.
From the trace, two initial candidates were identified as potential kernels for the mini-app, with comparable execution times: gw_init_lvl_tricoeff_recip, which includes the subroutine my_get_pairwise_coeff_3fn (shown in magenta Figure 6) and evaluate_periodic_gw_selfenergy, which includes the subroutines evaluate_polarisability_kspace and evaluate_gw_selfenergy_band_kpoint (shown in orange and cyan respectively). Among these, the selection of gw_init_lvl_tricoeff_recip was made due to the limited influence of MPI communications (compared to evaluate_periodic_gw_selfenergy) and its shared applicability with several beyond-DFT methods. The behaviour and relative weight is consistent in both test cases. The selected kernel in the Si system has an execution time of 127.57 s, which is a 34% of the total execution, while in the ZrO2 system, the execution time of the kernel was 978.53 s, accounting for a 45% of the total execution.
Operation
The Mini-App Suite is intended to be easily deployed and executed on different machines. To facilitate this, the suite has an integrated compilation and execution scheme for all the mini-apps using a CMake build system. Instructions to build, compile and execute the suite are provided in more detail within the README file included in root of the repository.
Once the suite has been properly built, the checkpoint files (stored in Zenodo40) must be downloaded and copied into the benchmark folder found in the repository. This can be directly done by executing the bash script get-ckpts.sh inside the benchmarks folder. However, in any case, downloading of the checkpoints should happen automatically when attempting to execute any of the run scripts provided, which are designed to check for the presence of the checkpoints in the expected locations and download them in case they are not found. In case the machine has no internet access, the file can be downloaded following the link provided in the README file. If a user would like to use alternative material benchmark systems, these can be generated by copying the checkpoint generator files placed inside the repository folder utils onto the original code, recompiling it and executing the modified version of the code using the desired test case and input parameters. This will generate a new checkpoint file that can be loaded within the mini-apps.
Examples of execution scripts are provided with the benchmark of each mini-app. The corresponding binaries for the GW cases must be executed, providing a checkpoint file. The mini-app should be submitted to a queuing system; for that, an example of the slurm script used in MareNostrum-4 is provided. After their execution, the mini-apps will produce a summary report with information on performance metrics such as timings and numerical checks. The information produced from the reports can be readily used to get insights into the performance. More detailed performance analyses of the mini-apps for specific systems are left to be done by experienced users.
Conclusions
The NOMAD mini-app suite, presented in this article, includes four mini-apps extracted from a set of representative flagship codes within the ab initio electronic structure community. These mini-apps focus on critical computational kernels involved in the GW implementations of exciting, Abinit and FHI-aims, and the ELPA eigenvalue solver. This mini-app suite represents a pragmatic approach to facilitate co-design efforts, employing ab initio electronic structure applications as use cases. By isolating and targeting specific computational kernels, this suite not only offers a pathway for focused optimisation but also marks a significant stride in fine-tuning both software and hardware capabilities in tandem. This aspect is particularly relevant as we venture into the era of exascale computing.
The practical benefits of these mini-apps are further enhanced by their adaptability across various HPC platforms. Their user-friendly deployment, facilitated by a streamlined CMake build system, broadens their accessibility to a diverse group of researchers and developers. This accessibility is pivotal in fostering a collaborative environment, essential for the co-evolution of computational tools and hardware technologies.
While these mini-apps serve as a valuable asset in electronic structure calculations, it is essential to recognise that their role is one piece of a giant puzzle. As we continue to explore the vast potential of exascale computing, it is crucial to maintain an ongoing dialogue within the community, ensuring that these tools evolve in response to emerging challenges and opportunities.
The mini-apps suite, the codes that make part of it, and, of course, the whole HPC ecosystem are quite lively research fields and in constant development. Therefore, performance metrics should be properly documented and shared among the community to facilitate the co-design efforts. Our repository is open to incorporating new performance metrics when the mini-apps are executed on new machines. Contributions of new versions that attempt to optimise the existing kernels or new mini-apps addressing other kernels are also expected as the co-design activity develops.
Acknowledgements
We acknowledge the support from all the developing teams from the codes included in the mini-apps suite. We thank Andreas Marek (ELPA), Claudia Draxl, Andris Gulans and Alexander Buccheri (exciting), Xavier Gonze, Maryam Azizi and Matteo Giantomassi (Abinit), Patrick Rinke and Antonio Delesma (FHI-aims); all for providing the test cases and participating in fruitful discussions on the performance analyses of their respective codes. We also thank the Performance Tools team at the BSC, Judit Giménez and German Llort, and the EU Performance Optimisation and Productivity Centre of Excellence in HPC (PoP-CoE), for their assistance with the performance analyses of the mini-apps. We also thank Albert Farrés for initiating this task within the BSC Materials Science team.
Faculty Opinions recommendedReferences
- 1.
Chang C, Deringer VL, Katti KS, et al.:
Simulations in the era of exascale computing.
Nat Rev Mater.
2023; 8(5): 309–313. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 2.
Gavini V, Baroni S, Blum V, et al.:
Roadmap on electronic structure codes in the exascale era.
Modelling Simul Mater Sci Eng.
2023; 31(6): 063301. Publisher Full Text
- 3.
Kůs P, Marek A, Köcher SS, et al.:
Optimizations of the eigensolvers in the elpa library.
Parallel Computing.
2019; 85: 167–177. Publisher Full Text
- 4.
Lehtola S, Steigemann C, Oliveira MJT, et al.:
Recent developments in libxc—a comprehensive library of functionals for density functional theory.
SoftwareX.
2018; 7: 1–5. Publisher Full Text
- 5.
Azizi M, Wilhelm J, Golze D, et al.:
Time-frequency component of the greenx library: minimax grids for efficient rpa and gw calculations.
J Open Source Softw.
2023; 8(90): 5570. Publisher Full Text
- 6.
Zhang L, Kozhevnikov A, Schulthess T, et al.:
Performance enhancement of apw+lo calculations by simplest separation of concerns.
Computation.
2022; 10(3): 43. Publisher Full Text
- 7.
Yu VWZ, Campos C, Dawson W, et al.:
Elsi — an open infrastructure for electronic structure solvers.
Computer Physics Communications.
2020; 256: 107459. Publisher Full Text
- 8.
Lin L, Lu J, Ying L, et al.:
Fast algorithm for extracting the diagonal of the inverse matrix with application to the electronic structure analysis of metallic systems.
Journal = Comm Math Sci.
2009; 7(3): 755–777. Publisher Full Text
- 9.
Lin L, Chen M, Yang C, et al.:
Accelerating atomic orbital-based electronic structure calculation via pole expansion and selected inversion.
J Phys Condens Matter.
2013; 25(29): 295501. PubMed Abstract
| Publisher Full Text
- 10.
Himanen L, Geurts A, Foster AS, et al.:
Data-driven materials science: status, challenges, and perspectives.
Adv Sci (Weinh).
2019; 6(21): 1900808. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 11.
Uhrin M, Huber SP, Yu J, et al.:
Workflows in aiida: Engineering a high-throughput, event-based engine for robust and modular computational workflows.
Computational Materials Science.
2021; 187: 110086. Publisher Full Text
- 12.
Larsen AH, Mortensen JJ, Blomqvist J, et al.:
The atomic simulation environment—a python library for working with atoms.
J Phys Condens Matter.
2017; 29(27): 273002. PubMed Abstract
| Publisher Full Text
- 13.
Gjerding M, Skovhus T, Rasmussen A, et al.:
Atomic simulation recipes: A python framework and library for automated workflows.
Computational Materials Science.
2021; 199: 110731. Publisher Full Text
- 14.
Jain A, Ong SP, Chen W, et al.:
Fireworks: a dynamic workflow system designed for high-throughput applications.
Concurr Comput.
2015; 27(17): 5037–5059. Publisher Full Text
- 15.
Cardwell SG, Vineyard C, Severa W, et al.:
Truly heterogeneous hpc: Co-design to achieve what science needs from hpc. In: Jeffrey Nichols, Becky Verastegui, Arthur ‘Barney’ Maccabe, Oscar Hernandez, Suzanne Parete-Koon, and Theresa Ahearn, editors, Driving Scientific and Engineering Discoveries Through the Con- vergence of HPC Big Data and AI. Cham, Springer International Publishing, 2020; 349–365. Publisher Full Text
- 16.
Páll S, Zhmurov A, Bauer P, et al.:
Heterogeneous parallelization and acceleration of molecular dynamics simulations in GROMACS.
J Chem Phys.
2020; 153(13): 134110. PubMed Abstract
| Publisher Full Text
- 17.
Shaw DE, Adams PJ, Azaria A, et al.:
Anton 3: Twenty microseconds of molecular dynamics simulation before lunch. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. SC ’ 21, New York, NY USA, Association for Computing Machinery, 2021. Reference Source
- 18.
Heroux MA, Doerfler DW, Crozier PS, et al.:
Sandia report improving performance via mini-applications.
- 19.
Martineau M, McIntosh-Smith S:
The arch project: Physics mini-apps for algorithmic exploration and evaluating programming environments on hpc architectures.
2017 IEEE International Conference on Cluster Computing (CLUSTER).
Institute of Electrical and Electronics Engineers Inc. 2017; 2017: 850–857. Publisher Full Text
- 20.
Crozier PS, Thornquist HK, Numrich RW, et al.:
Improving performance via mini-applications. Publisher Full Text
- 21.
Barrett RF, Crozier PS, Doerfler DW, et al.:
Assessing the role of mini- applications in predicting key performance characteristics of scientific and engineering applications.
J Parallel Distrib Comput.
2015; 75: 107–122. Publisher Full Text
- 22.
Tsuji M, Kramer WTC, Sato M:
A performance projection of mini-applications onto benchmarks toward the performance projection of real- applications. In: 2017 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 2017; 826–833. Publisher Full Text
- 23.
NoMaD: the novel materials discovery laboratory. Publisher Full Text
- 24.
Marek A, Blum V, Johanni R, et al.:
The elpa library: scalable parallel eigenvalue solutions for electronic structure theory and computational science.
J Phys Condens Matter.
2014; 26(21): 213201. PubMed Abstract
| Publisher Full Text
- 25.
Nabok D, Gulans A, Draxl C:
Accurate all-electron G0W0 quasiparticle energies employing the full-potential augmented plane-wave method.
Phys Rev B.
2016; 94: 035118. Publisher Full Text
- 26.
Gulans A, Kontur S, Meisenbichler C, et al.:
Exciting: a full-potential all-electron package implementing density-functional theory and many-body perturbation theory.
J Phys Condens Matter.
2014; 26(36): 363202. PubMed Abstract
| Publisher Full Text
- 27.
Gonze X, Beuken JM, Caracas R, et al.:
First-principles computation of material properties: the abinit software project.
Comput Mater Sci.
2002; 25(3): 478–492. Publisher Full Text
- 28.
Gonze X, Jollet F, Araujo FA, et al.:
Recent developments in the abinit software package.
Comput Phys Commun.
2016; 205: 106–131. Publisher Full Text
- 29.
Gonze X, Amadon B, Antonius G, et al.:
The Abinit project: Impact, environment and recent developments.
Comput Phys Commun.
2020; 248: 107042. Publisher Full Text
- 30.
Blum V, Gehrke R, Hanke F, et al.:
Ab initio molecular simulations with numeric atom-centered orbitals.
Comput Phys Commun.
2009; 180(11): 2175–2196. Publisher Full Text
- 31.
Ren X, Merz F, Jiang H, et al.:
All-electron periodic G0W0 implementation with numerical atomic orbital basis functions: Algorithm and benchmarks.
Phys Rev Mater.
2021; 5: 013807. Publisher Full Text
- 32.
Hedin L:
New Method for Calculating the One-Particle Green's Function with Application to the Electron-Gas Problem.
Phys Rev.
1965; 139: A796–A823. Publisher Full Text
- 33.
Golze D, Dvorak M, Rinke P:
The GW compendium: A practical guide to theoretical photoemission spectroscopy.
Front Chem.
2019; 7: 377. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 34.
Aryasetiawan F, Gunnarsson O:
The GW method.
Rep Prog Phys.
1998; 61(3): 237. Publisher Full Text
- 35.
Extrae: BSC Performance Tools. 2023. Reference Source
- 36.
Mucci PJ, Browne S, Deane C, et al.:
Papi: A portable interface to hardware performance counters. In: Proceedings of the Department of Defense HPCMP Users Group Conference. 1999; 710. .
- 37.
Pillet V, Labarta J, Cortes T, et al.:
Paraver: A tool to visualize and analyze parallel code. In: Proceedings of WoTUG-18: Transputer and Occam Developments. 1995; 44. : 17–31.
- 38.
Paraver: BSC Performance Tools. 2023. Reference Source
- 39.
Ihrig AC, Wieferink J, Zhang IY, et al.:
Accurate localized resolution of identity approach for linear-scaling hybrid density functionals and for many-body perturbation theory.
New J Phys.
2015; 17(9): 093020. Publisher Full Text
- 40.
Mas Magre I, Grima Torres R, Gutiérrez Moreno J:
NOMAD mini-apps suite checkpoints. 2023. Publisher Full Text
- 41.
Mas Magre I, Grima Torres R, Gutiérrez Moreno J:
NOMAD mini-apps suite.
Zenodo.
[Software]. 2023. http://www.doi.org/10.5281/zenodo.10412697
- 42.
Mas Magre I, Grima Torres R, Gutiérrez Moreno J:
NOMAD mini-app suite checkpoints [Dataset].
Zenodo.
2023. http://www.doi.org/10.5281/zenodo.10142416
