User’s view of LEXIS Platform

The LEXIS portal - a one-stop-shop for using the platform - is described in detail here: LEXIS Portal. It helps the user to easily organise computing workloads and data according to the platform’s concepts, which are shown in the figure and quickly detailed on below.

../../_images/LEXIS_Platform_Entities.png

Experienced users can access the platform and its features via Py4Lexis. In any case, the control of projects, computations and datasets follows a few basic notions and principles we discuss in the following sections.

LEXIS Computational Project

A LEXIS Computational Project is an abstraction of the computing and storage resources allocated for running LEXIS workflows. As such, each LEXIS Computational Project may comprise of multiple associated HPC Computational Projects which may be provided by different HPC centres following some dedicated approval process.

HPC Computational Project

An HPC Computational Project is an abstraction of the computing resources allocated to the HPC users by the HPC centres for running their applications. HPC centres usually allocate a given amount of resources represented by core hours (or different unit, depending on the centre). HPC Computational Projects have the following characteristics:

Requests are submitted by users that want to utilize the resources. HPC centres then grant the resources to the selected requests. Part of the request for the resources is also a selection of a computing resource of the centre (GPU nodes, Cloud, etc.).
After the resources are granted, the HPC Computational Project is created. The project itself normally has an ID which is unique within a given centre and has a given timeframe in which the resources have to be consumed.
Such a project typically has one Principal Investigator (see below) and possibly further project members with administrator-like roles (privileged users), depending on the HPC centre and the project. The privileged and normal users within an HPC Computational Project have respective accounts in the HPC centres’ AAI systems.

Principal Investigator

Each HPC Computational Project must have a Principal Investigator (PI) who is responsible for ensuring appropriate use of the Project resources. The PI can perform standard administrative operations on the project including adding, removing users, view usage, requesting more resources etc. Note that LEXIS can support HPC Computational Projects in which the PI is not involved in any LEXIS Project (i.e., he/she is not a LEXIS user).

LEXIS Task

A LEXIS Task represents an element of the LEXIS Workflow, and it defines a set of operations to be performed, along with attributes such as the required input data, output data, monitoring information etc. The lifecycle of a LEXIS Task depends on the specific operations that it has to perform; however, it is (in any case) restricted to the lifetime of the LEXIS Workflow.

LEXIS Workflow Template

The LEXIS Workflow Template is a description of the connections of LEXIS Tasks that must be performed to complete the processing of the input LEXIS Datasets and producing output LEXIS Datasets. As such, LEXIS Tasks inside a LEXIS Workflow Template can be organized to form a DAGs (Direct Acyclic Graph), with possibility to access to databases, etc. A LEXIS Workflow Template provides a description of its LEXIS Tasks and the correct sequence of their execution.

LEXIS Workflow

Starting from a LEXIS workflow template, the corresponding LEXIS Workflow associates the inputs to the LEXIS Tasks and the application of configurations (i.e., setting up of specific input parameters) to each LEXIS Task.

LEXIS Workflow Execution

A LEXIS Workflow Execution represents the actual deployment of the LEXIS Workflow on the resources selected by the orchestrator. As such, each LEXIS Task is deployed on the set of computational resources dynamically selected by the orchestrator (according to monitoring data, internal placement policies, users’ specified criteria), along with their required inputs (e.g., LEXIS User Dataset). The operations defining the LEXIS Tasks are thus executed. As such, the execution of LEXIS Tasks generates outputs (e.g., output LEXIS User Dataset).

LEXIS Task Execution

A LEXIS Task Execution represents the actual execution of a task within a LEXIS Workflow Execution. As such, the tasks are deployed on the resources dynamically selected by the orchestrator.

LEXIS Workflow Catalogue

The LEXIS Workflow Catalogue provides the repository of LEXIS Workflow Templates managed by the LEXIS Platform. It contains the set of workflow templates (catalogue) that users can instantiate and execute on the associated compute resources in particular centres (HPC systems, Cloud, etc.) Note that access restrictions apply to the Workflow Catalogue: users can only create LEXIS Workflows based on LEXIS Workflow Templates available to them.

LEXIS User Dataset

A LEXIS Dataset represents the input or output set of data that a LEXIS Workflow respectively consumes or produces during its execution. The datasets can be uploaded/downloaded by the LEXIS Portal user interface.

LEXIS Dataset Listing

The LEXIS Dataset Listing produces a list of the datasets available via the LEXIS Distributed Data Infrastructure (DDI) and via the WCDA (Weather and Climate Data API). In general, this comprises of public datasets, project specific data sets and datasets owned/managed by the user, all hosted on the DDI.