HEAppE Middleware |heappeLogo| ============================== .. |heappeLogo| image:: Images/HEAppE_seda.png :target: ../../_images/HEAppECloudMapping.png :width: 10% HPC-as-a-Service is a well known term in the area of high performance computing. It enables users to access an HPC infrastructure without a need to buy and manage their own physical servers or data centre infrastructure. Through this service small and medium enterprises (SMEs) can take advantage of the technology without an upfront investment in the hardware. This approach further lowers the entry barrier for users and SMEs who are interested in utilizing massive parallel computers but often do not have the necessary level of expertise in this area. To provide this simple and intuitive access to the supercomputing infrastructure an in-house application framework called `HEAppE `_ (High-End Application Execution Middleware) has been developed. This framework is utilizing a mid-layer principle, in software terminology also known as middleware. Middleware manages and provides information about submitted and running jobs and their data between the client application and the HPC infrastructure. HEAppE is able to submit required computation or simulation on HPC infrastructure, monitor the progress and notify the user should the need arise. It provides necessary functions for job management, monitoring and reporting, user authentication and authorization, file transfer, encryption, and various notification mechanisms. Key HEAppE features ------------------- * Multi-platform .NET Core version * Dockerized deployment and management * PBS and Slurm workload manager adapters * Support for local computing (HPC cluster simulation) * OpenAPI REST API * OpenID support * OpenStack authentication support * SSH Agent support * Multiple tasks within single computational jobs support * Job arrays support and job dependency support * Extremely long running job support * Database logging support HEAppE's security & user mapping mechanism ------------------------------------------ Probably the most important feature of HEAppE is its mapping mechanism. In terms of security HEAppE recognizes two types of credentials: * external user accounts * internal service/cluster accounts **External user accounts** do not have direct access to the HPC infrastructure itself. They are used only to authenticate the external user via the HEAppE middleware to access only HEAppE’s provided functions. External user accounts can be managed directly via HEAppE, i.e., stored within HEAppE’s internal database, or handled externally by another identity and access management service such as KeyCloak, etc. Additionally, for HEAppE to be able to submit jobs to the actual HPC cluster queue, a set/pool of so-called **cluster service accounts** is required. These service accounts are usually non-personalised standard cluster accounts bound to a specific computational project that has been allocated with a specific computational resources. The service accounts are generated specifically to be used within the HEAppE middleware at the request of the Primary Investigator (PI) of a given computational project; the PI of the project agrees with the creation of service accounts and gives permission for these accounts to be used within HEAppE. HEAppE provides the mapping between external user accounts and internal service accounts. It keeps track of which service account was used for the submission of which compute job and which user made the submission. Using this mechanism HEAppE can be used in two submission modes: **shared service accounts**, where one service account can be used in parallel to submit multiple compute jobs even from different users; and the **exclusive mode**, in which the specific service account can be used only once at the time, and can be used again only when the current computational job using this account has finished and the account has returned back to the pool of available service accounts. The exclusive mode delivers extra security resilience, but limits the number of parallel job submissions to the number of available service accounts. .. The following Figure presents placement of the middleware solution within the LEXIS infrastructure. .. .. image:: Images/Haas.png :target: ../../_images/Haas.png :align: right :width: 45% Technical documentation and code repository ------------------------------------------- * For more detail-oriented information, please visit our `HEAppE documentation `__ . The web address contains comprehensive technical information about the HEAppE Middleware. This documentation will help you to 'get started', explains the prerequisites and requirements, deployment procedure and the basic HEAppE interactions. * HEAppE source code repositories: ``__ HEAppE command templates ------------------------ For security purposes, HEAppE enables the users to run only a pre-prepared set of so-called **command templates**. Each template defines a script or executable binary, that will be executed using a specific HPC cluster queue to which the template is assigned. Command template also defines any **dependencies** or **third-party software** the executable might require. The command template contains the set of **input parameters**: parameters that are exposed in the API and will be passed to the executable script during the run-time. Thus, the users can execute pre-prepared command templates with the pre-defined set of input parameters. The actual value of each parameter (input from the user) can be different for each job submission. The following blocks present the example of command template setup that executes *'test.sh'* script in a cluster environment. This script consumes one input parametr called *'inputParam'*. .. code-block:: :caption: **test.sh** script content #!/bin/bash inputParam="$1" echo "Input param: ${inputParam}" for i in {01..10} do echo "Iteration: ${i}" sleep 30s done echo "some result" >> resultFile.txt exit 0 .. code-block:: :caption: Command template definition for **test.sh** script and **inputParam** parameter "CommandTemplates":[ { "Id": 1, "Name": "TestTemplate", "Description": "TestTemplate", "Code": "TestTemplate", "ExecutableFile": "~/.key_scripts/test.sh", "CommandParameters": "%%{inputParam}", "PreparationScript": null, "ClusterNodeTypeId": 2, "IsGeneric": false, "IsEnabled": true } ], "CommandTemplateParameters": [ { "Id": 1, "Identifier": "inputParam", "Query": "", "Description": "inputParam", "CommandTemplateId": 1, "IsVisible": true } ] .. code-block:: :caption: **test.sh** script output after a successful execution $ ./test.sh exampleParameter Input param: exampleParameter Iteration: 01 Iteration: 02 Iteration: 03 Iteration: 04 Iteration: 05 Iteration: 06 Iteration: 07 Iteration: 08 Iteration: 09 Iteration: 10 For more detail-oriented information, about how to prepare HEAppE command templates or how to use **generic command template** feature please visit `HEAppE documentation `__. Connection between Puhuri and HEAppE ------------------------------------ `Puhuri `_ is a service for sharing project and allocation information across HPC sites. It supports propagation of user identity information to HPC sites along with user credentials as well as generation of “group” or “robot” accounts representing a specific project. The latter can be used for automatic integration with HEAppE - a Puhuri agent running at HPC site is able to generate new SSH keys and a) add the private keys to HEAppE; b) publish public keys to the corresponding robot account in Puhuri Core. Account information from the Puhuri Core is propagated into HPC cluster, e.g. based on SLURM, which allows HEAppE to get access to HPC cluster. Puhuri Core also contains information about user allocations, which can be used by LEXIS for connecting HEAppE tasks in a multi-tenant setup. .. image:: Images/Puhuri_HEAppE.png :target: ../../_images/Puhuri_HEAPPE.png