HEAppE Middleware 
HPC-as-a-Service is a well known term in the area of high performance computing. It enables users to access an HPC infrastructure without a need to buy and manage their own physical servers or data centre infrastructure. Through this service small and medium enterprises (SMEs) can take advantage of the technology without an upfront investment in the hardware. This approach further lowers the entry barrier for users and SMEs who are interested in utilizing massive parallel computers but often do not have the necessary level of expertise in this area.
To provide this simple and intuitive access to the supercomputing infrastructure an in-house application framework called HEAppE (High-End Application Execution Middleware) has been developed. This framework is utilizing a mid-layer principle, in software terminology also known as middleware. Middleware manages and provides information about submitted and running jobs and their data between the client application and the HPC infrastructure. HEAppE is able to submit required computation or simulation on HPC infrastructure, monitor the progress and notify the user should the need arise. It provides necessary functions for job management, monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms.
Key HEAppE features
Multi-platform .NET Core version
Dockerized deployment and management
PBS and Slurm workload manager adapters
Support for local computing (HPC cluster simulation)
OpenAPI REST API
OpenID support
OpenStack authentication support
SSH Agent support
Multiple tasks within single computational jobs support
Job arrays support and job dependency support
Extremely long running job support
Database logging support
HEAppE’s security & user mapping mechanism
Probably the most important feature of HEAppE is its mapping mechanism. In terms of security HEAppE recognizes two types of credentials:
external user accounts
internal service/cluster accounts
External user accounts do not have direct access to the HPC infrastructure itself. They are used only to authenticate the external user via the HEAppE middleware to access only HEAppE’s provided functions. External user accounts can be managed directly via HEAppE, i.e., stored within HEAppE’s internal database, or handled externally by another identity and access management service such as KeyCloak, etc.
Additionally, for HEAppE to be able to submit jobs to the actual HPC cluster queue, a set/pool of so-called cluster service accounts is required. These service accounts are usually non-personalised standard cluster accounts bound to a specific computational project that has been allocated with a specific computational resources. The service accounts are generated specifically to be used within the HEAppE middleware at the request of the Primary Investigator (PI) of a given computational project; the PI of the project agrees with the creation of service accounts and gives permission for these accounts to be used within HEAppE. HEAppE provides the mapping between external user accounts and internal service accounts. It keeps track of which service account was used for the submission of which compute job and which user made the submission.
Using this mechanism HEAppE can be used in two submission modes: shared service accounts, where one service account can be used in parallel to submit multiple compute jobs even from different users; and the exclusive mode, in which the specific service account can be used only once at the time, and can be used again only when the current computational job using this account has finished and the account has returned back to the pool of available service accounts. The exclusive mode delivers extra security resilience, but limits the number of parallel job submissions to the number of available service accounts.
Technical documentation and code repository
For more detail-oriented information, please visit our HEAppE documentation . The web address contains comprehensive technical information about the HEAppE Middleware. This documentation will help you to ‘get started’, explains the prerequisites and requirements, deployment procedure and the basic HEAppE interactions.
HEAppE source code repositories: https://github.com/It4innovations/HEAppE
HEAppE command templates
For security purposes, HEAppE enables the users to run only a pre-prepared set of so-called command templates. Each template defines a script or executable binary, that will be executed using a specific HPC cluster queue to which the template is assigned. Command template also defines any dependencies or third-party software the executable might require.
The command template contains the set of input parameters: parameters that are exposed in the API and will be passed to the executable script during the run-time. Thus, the users can execute pre-prepared command templates with the pre-defined set of input parameters. The actual value of each parameter (input from the user) can be different for each job submission.
The following blocks present the example of command template setup that executes ‘test.sh’ script in a cluster environment. This script consumes one input parametr called ‘inputParam’.
#!/bin/bash
inputParam="$1"
echo "Input param: ${inputParam}"
for i in {01..10}
do
echo "Iteration: ${i}"
sleep 30s
done
echo "some result" >> resultFile.txt
exit 0
"CommandTemplates":[
{
"Id": 1,
"Name": "TestTemplate",
"Description": "TestTemplate",
"Code": "TestTemplate",
"ExecutableFile": "~/.key_scripts/test.sh",
"CommandParameters": "%%{inputParam}",
"PreparationScript": null,
"ClusterNodeTypeId": 2,
"IsGeneric": false,
"IsEnabled": true
}
],
"CommandTemplateParameters": [
{
"Id": 1,
"Identifier": "inputParam",
"Query": "",
"Description": "inputParam",
"CommandTemplateId": 1,
"IsVisible": true
}
]
$ ./test.sh exampleParameter
Input param: exampleParameter
Iteration: 01
Iteration: 02
Iteration: 03
Iteration: 04
Iteration: 05
Iteration: 06
Iteration: 07
Iteration: 08
Iteration: 09
Iteration: 10
For more detail-oriented information, about how to prepare HEAppE command templates or how to use generic command template feature please visit HEAppE documentation.
Connection between Puhuri and HEAppE
Puhuri is a service for sharing project and allocation information across HPC sites. It supports propagation of user identity information to HPC sites along with user credentials as well as generation of “group” or “robot” accounts representing a specific project.
The latter can be used for automatic integration with HEAppE - a Puhuri agent running at HPC site is able to generate new SSH keys and a) add the private keys to HEAppE; b) publish public keys to the corresponding robot account in Puhuri Core. Account information from the Puhuri Core is propagated into HPC cluster, e.g. based on SLURM, which allows HEAppE to get access to HPC cluster. Puhuri Core also contains information about user allocations, which can be used by LEXIS for connecting HEAppE tasks in a multi-tenant setup.
