How-to Guide
We would like to briefly introduce you to the LEXIS platform by following paragraphs. The LEXIS platform offers various actions for the users interested to usage of HPC and cloud via graphical interface or HTTP API. Within the other possibilities, following actions are available on the platform:
create LEXIS projects
request access to existing LEXIS projects
manage members of LEXIS projects as project manager
create and manage users datasets
manage LEXIS project computation resources
create LEXIS workflows with containerized applications
create LEXIS workflows with HPC command applications
manage LEXIS custom workflows
execute LEXIS workflows
see LEXIS workflow execution progress
browse LEXIS workflow execution results
Registration
Registration is so easy. Upon clicking the login button located in the upper right corner, you will encounter two sign-up options. The B2Access and MyAccessID - Puhuri are trustful authentication authorities supporting eduGAIN, ORCID and Social Identities.
If you don’t have any projects in LEXIS yet, you can create a new one or request access to an existing project. Both of these options will be described in the following chapters.
Creation of LEXIS Project
LEXIS Computational Project is the basic organization unit. If you don’t have access to any project yet, you will see the option to create a new one immediately after logging into LEXIS.
If you already have access to projects but need to create a new one, this can be done in the Projects in the navigation menu using the button Create Project.
Next, it is necessary to add in the details about the new project.
The newly created project can be viewed in the project list in the Projects menu.
Requesting access to existing LEXIS Project
If you don’t have access to any project yet, you can request access to an existing project immediately after logging in.
If you already have access to Lexis Projects but need to request access to another existing project, this can be done in the Projects tab using the button Request Project Access.
To submit a request, it is sufficient to know the short name of the project.
The request will be sent to the Project Owner, who must approve access to the project and assign you the appropriate role in it (Manage members of LEXIS projects as project manager).
Next you will receive an email in response with information on whether you have been granted access to the LEXIS Project or not.
Requesting to LEXIS project computation resources
Execution of workflows and storing files requires a LEXIS Computation Resource. To assign one with computation time and one with data storage allocation, click on Resources next to the created project on the Projects.
You can request the allocation of resources in the project list using the Resources button.
This will display a list of resources already allocated to the LEXIS Project, and there you will find the Request Resources Access button. The right to request Resources within the LEXIS Project is only granted to users in the role of Project Manager or Project Owner.
Click on Request Resource Access and fill out the form. Please provide the Resources name and the email of the PI for the selected project.
After your request has been successfully submitted, you will be informed about the progress via your email. Now wait until the PI of the resource accepts your request. If you are the PI, an email will be sent to you with a one-time link to accept the request. To assign a storage resource, fill in the resource name as “iRODS LEXIS V2”, choose the provider “IT4Innovations”, and fill in the email field with “support@lexis-project.eu”. After the request has been processed by the PI, the applicant will be notified by email whether access to the requested resources has been accepted or rejected.
Creation of Users Dataset
Before executing some computation on LEXIS Platform, you would like probably to upload some input data - LEXIS Users Dataset. To do that, navigate to Data Sets and Uploads section via navigation menu on the left side. You will find yourself on page with all accessible uploaded data sets list.
Clicking on green button Upload Dataset, following form will appear.
For uploading data an existing assigned LEXIS Computation Resource to a LEXIS Computational Project. Access defines visibility scope of dataset to one of user (dataset owner), project (all members in the project), public. Every dataset can have metadata.
Modify of Users Data Set
The list of already uploaded Users Data Sets can be viewed in Data Sets/Uploads. The allowed actions for a specific dataset can be displayed using the Actions button.
Commonly used actions for the User Data Set include: Copy ID, Copy PATH, File List, Delete and Downland.
You can use or get inspired by publicly available Users Data Sets. Here they are available for preview and possible download.
Container Applications
One of the supported methods for delivering instructions for a Workflow in the LEXIS Project is recording them in an HPC Container Application.
You can add a Container in the Applications/Containers menu. Then, in the tabs, select the LEXIS project for which you want to add the container and click on Create Container button.
Now, you need to upload the file with the container’s contents, an icon, enter the name, and gradually fill in additional information about the new container.
At the end, a summary will be displayed for review and to complete the saving of the new container.
Existing containers are displayed in blocks, including basic information. By clicking on each block, you can create a new Workflow with selected container.
HPC Command applications
Another option for defining an input template for a Workflow is preparation (compiled) by specialist an HPC Command Application. If you have your own HPC application ready, please contact support@lexis-project.eu, who will ensure the addition of your HPC application to the LEXIS Project. After the new HPC application is uploaded, it will appear in the list of already uploaded applications in the Applications/HPC Application. module. By clicking on a specific application, you can continue configuring the Workflow and then run it with the specified parameters.
Custom HPC Jobs
To create user HPC Jobscript, navigate to Applications and Custom HPC Jobs. Click on green button Create Jobscript.
Use following code as example job script and continue.
# Example Job Script for LEXIS Workflow
source /cvmfs/software.eessi.io/versions/2023.06/init/bash
ls ./input # in this directory should appear staged input dataset
cat ./input/vinice-geojson.json # reads content of uploaded file to dataset
echo "I am running!!"
Now the job script for workflow is prepared and workflow can be created by clicking Create Workflow next to the prepared job script in Custom HPC Jobs page.
In second step, one of assigned resources to the project is selected together with execution cluster and partition.
The data input is enabled and default dataset is selected. The dataset will be staged to /input directory relative to HPC job execution contex.
After creation of workflow, it will finally appear after a few seconds in Custom HPC Script Worfklows section in Custom HPC Jobs page.
Now, it is ready to create first execution. Click on green Execute button and fill the form.
In the execution detail, there is visualisation. Particular steps produce logs, which are accessible by moving the cursor over the step and clicking on the Logs button.
Logs from HPC job can usually be found in …heappe_job_waiting task’s log.
# Job log - partial GeoJSON content and “I am running” message
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - [
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - 16.64590904709604,
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - 48.810181792975015
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - ],
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - [
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - 16.64727894500959,
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - 48.81075254030276
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - ]
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - ],
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - "type": "LineString"
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - }
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - }
[2024-07-18T11:58:00.642+0000] {logging_mixin.py:188} INFO - ],
[2024-07-18T11:58:00.643+0000] {logging_mixin.py:188} INFO - "bbox": null
[2024-07-18T11:58:00.643+0000] {logging_mixin.py:188} INFO - }
[2024-07-18T11:58:00.643+0000] {logging_mixin.py:188} INFO - I am running!!
[2024-07-18T11:58:00.650+0000] {base.py:293} INFO - Success criteria met. Exiting.
[2024-07-18T11:58:00.672+0000] {taskinstance.py:1138} INFO - Marking task as SUCCESS.
Custom LEXIS Workflows
Another option for specifying a template with tasks and dependencies is defining a User Workflow using a YAML script.
To add a new User Workflow, select Application/User Workflow on the left. Then, the Create Workflow button will appear on the right.
The YAML script for the new Workflow can either be uploaded from your local machine or created using the form in the provided editor.
Now you need to enter the name of the new workflow and choose a short name for the LEXIS Project.
After successfully creating the Workflow, you can immediately execut and test the new Workflow. In the execution detail, there is visualisation. Particular steps produce logs, which are accessible by moving the cursor over the step and clicking on the Logs button.
Executions of LEXIS Workflows
LEXIS Workflows can always be executed immediately after entering the input application and dataset within the Applications module. Subsequently, all executed LEXIS Workflows can be viewed in the Workflows module. Here, there is also the option to execute the workflow again with modified input parameters.
So, if you need to view the details of an already executed workflow, you can do it using the magnifying Glass button in the row of the specific workflow or the Executions button.
In the Details tab using the blue button Workflow Executions, you can view the list of all executions of this Workflow, including the details of each run. With the green button Create Workflow Execution, you can set the parameters for a new execution and start it.
In the Workflow Graph tab, you can view the graph of individual tasks in the selected LEXIS Workflow.
Below, we provide a legend for the colors used in the Workflow to indicate different statuses of Workflows Tasks - LEXIS Task.
Users Dashboard
The portal features a Dashboard module that provides a convenient overview of running workflows, their progress, and results. In the Dashboard/Workflow, this information can be viewed in clear diagrams, including some details about the workflows.
Users Permissions Management
In the top right corner, you can view the details of your user account.
By clicking on View Profile, a table of roles for each of your projects will be displayed on the right.
Within the LEXIS Project, we define 4 user roles. Below is a list and description of each:
Project Owner - This role is automatically assigned to the user who creates a new LEXIS Project. There can be only one Owner per project. The user in this role has the highest level of permissions on the project and is responsible for approving new users and assigning them appropriate roles.
Project Manager - The Owner can assign this role to other users on the project to delegate project and user management to additional users. Multiple users can hold the Manager role within a project.
Project Member - This role is for standard users on the project. A user in this role can work with Workflows and DataSets but cannot publish any DataSets.
Project DataPublisher - A user in this role can work with Workflows and DataSets, and also has the permission to publish Data Sets.
Manage members of LEXIS projects as project manager
If someone Requesting access to existing LEXIS Project where you are in the role of Project Owner or Project Manager, a new request will appear here for your approval. If you approve the request, the new user will automatically be assigned the role of Project Member.
If you want to set a higher permission level for the user (role Project Manager) or deny access to the project, this can be done in the Project module, under the Users button.
