************ LEXIS Portal ************ Source code ----------- The portal source code is available at: `https://github.com/lexis-project/lexis-frontend-general `_. Project management ================== The LEXIS Portal provides an user friendly interface to manage organization projects, theirs approved resources and its usage. User could create new project, with proper permissions, view its detail, edit and the project manager could assign other organization users to the project. Project listing --------------- The organizations project list is accessible on page Projects, it should be found in main menu on the left side of the Portal UI. Projects are listed with following information in columns: * **Project name** * **NORM. CORE HOURS** * **PROJECT STATUS** * **LINKED ORGANIZATION** .. image:: Images/projects-listing.png Project creation ---------------- User, with proper permissions, can create a project by clicking on button Create new LEXIS project in the right upper corner in the projects list and fill the following form. .. image:: Images/projects-create.png Project detail -------------- Project detail could be opened by clicking on the name of the project in projects list. .. image:: Images/project-details.png Project editing --------------- For editing project user should go to the project detail and click on Edit project info button in right upper corner of the page. The edit project form is the same as the project creation form. Project users listing --------------------- List of assigned users to the project is accessible through the ``Manage users`` button at the top of the page of the page. In table are listed following information: * **First name** * **Last Name** * **Email Address** * **Organization** * **Dat. Access** * **Dat. Create** * **Dat. Pub. Create** * **Prj. Mngr.** * **Wfl. Mngr.** User may go back to project detail by clicking on the name of the project on the top of the page under the logo LEXIS. .. image:: Images/projects-users-management.png Project users assignment ------------------------ User could be assigned to the project by clicking on the ``Assign User to Project`` button on the upper right corner of the page with user management. User may search and select users to assign from organization users in field, which appears right next to the subtitle ``Add user to project``. After selecting one from the list below the search field, its required to confirm the action to ``Assign User`` button right next to the search field. .. image:: Images/projects-assign-user.png Project users unassignment -------------------------- User could be unassigned by clicking on the ``Unassign User from Project`` button on the upper right corner of the page with user management. User may search and select users to unassign from assigned users in field, which appears right next to the subtitle ``Unassign User from Project``. After selecting one from the list below the search field, its required to confirm the action to ``Unassign User`` button right next to the search field. Dataset management ================== The LEXIS portal provides a user interface to create, view and modify datasets, including metadata handling (e.g. creator or publication year), or file upload, download and deletion within the directory structure of the dataset. Integration of the datasets with EUDAT services for data discoverability and replication is also provided. Staging of the data to the different supercomputing centres available in LEXIS is also presented here. Finally, the visualization of images within datasets (used for tracking workflow execution status) is also provided in this module. Dataset listing --------------- When opening the LEXIS Portal UI, the user will first need to navigate to the Data Sets page. At that page there are two tables "Workflows" (datasets created by workflow executions) and "Uploads" (uploaded datasets) . The tables are shown, containing all the datasets filtered by global filter the user has access to. The "Workflows" table shows the following information for "Workflows" datasets: * **Data set title** * **Access** #. **user (only that user has access)** #. **project (all members in the project have access)** #. **public (everybody has access)** * **Publication Year** * **Creator** * **Workflow Id** * **Workflow Execution Id** The "Uploads" table shows the following information for Uploaded datasets: * **Data set title** * **Access** #. **user (only that user has access)** #. **project (all members in the project have access)** #. **public (everybody has access)** * **Publication Year** * **Creator** The information may be filtered by typing into "Search" field at the top right of each table. The user may create new datsets here, or click on a dataset "Details" button to retrieve more information about it. Created by workflow executions: .. image:: Images/datasets-filter-listing-workflows.png Uploaded: .. image:: Images/datasets-filter-listing-uploads.png Dataset metadata ---------------- After selecting a dataset, a view with additional information appears, including the owner of the dataset, and a buttons: "Copy data set iRODS path", "Download the Dataset as archive", "Modify dataset metadata", "Stage deletion of the Dataset". The user can retrieve more information by clicking **Advanced**. .. image:: Images/datasets-detail.png The **Advanced** button shows additional metadata such as identifiers (DOI, PID), resource types and rights management (i.e. license), and EUDAT interaction (B2Stage GridFTP access location, EUDAT replicas). .. image:: Images/datasets-detail-advanced.png Update dataset metadata ----------------------- The user can here modify metadata fields such as creator, title, publisher etc. .. image:: Images/datasets-metadata-update.png Dataset upload -------------- Upload of datasets to the portal is implemented using TUS protocol, which uploads the file in chunks over standard HTTPS connection. To upload datasets go to Datasets menu and click on “Create new dataset” button in top right corner. The upload process is two-step. First the file gets uploaded to the server using TUS protocol from the user browser, then the service uploads the file to the iRODS backend. .. image:: Images/upload-button.png Individual files can be uploaded as well as whole directories packed in the ``.tar.gz`` format. Upload inputs ------------- Dataset to upload - browse for a file to be uploaded Uncompress at destination - upload a file in ``.tar.gz`` format to be decompressed in the dataset root. Encryption - encrypt the uploaded file in the iRODS dataset collection Title - arbitrary dataset title Project - which LEXIS Project to upload the dataset to Access - select level of access (User,Project,Public) Zone - select iRODS zone to upload to .. image:: Images/upload-wiz-1.png Users can fill in the dataset metadata in the DataCite format to enable proper FAIRness. The metadata are then propagated to iRODS metadata and can be further used when publishing the datasets to external services. .. image:: Images/upload-wiz-2.png Check the filled out information and click on the button to start the upload. .. image:: Images/upload-wiz-3.png Watch the progress of the upload. The list of uploads in progress for a particular user is available in Datasets page by clicking on the Upload/download statuses button. .. image:: Images/upload-status-button.png Viewing dataset filelist ------------------------ Dataset filelist is accessible throught page with datasets detail, where is the button below the list of information. A tree-like view of the directory structure of a dataset is provided. For each directory, a button exists to enable the user to upload files there. For each file or directory, buttons exist to enable the user to download or delete. Finally, image files within a dataset have an extra button which allows the user to view them. .. image:: Images/datasets-filelist.png Viewing images within a dataset ------------------------------- The portal provides the possibility to view an image within a dataset, with automatic refresh. When a workflow is being executed, this enables the workflow to provide information about the current state of the workflow, by overwriting a specific image in a dataset. The user is then shown an up-to-date view of the workflow state. .. image:: Images/datasets-filelist-view.png Workflow execution management ============================= The purpose of this page if to describe Workflow Execution management. The LEXIS Portal provides a user interface for user to create, manage and monitor ``AIRFLOW`` Workflow Executions this document will provide details on how this can be done through the LEXIS Portal User interface. Airflow workflow execution creation ----------------------------------- To run a ``Airflow`` Workflow Execution the user must navigate to the Workflow detail page within ``Airflow`` section of the Workflow they wish to run a Workflow Execution for, from here they click the ‘Create Workflow Execution’ button. .. image:: Images/AirflowWorkflowDetails.png The user now is redirected to the creation form. To Run the Workflow Execution, fill the inputs appropriatley and click ``CREATE``. .. image:: Images/AirflowWorkflowExecutionCreationForm.png User is redirected to the Execution details page. .. image:: Images/AirflowWorkflowExecutionDetails.png Airflow workflow execution monitoring ------------------------------------- To monitor and manage a Workflow Execution the user simply selects the Workflow Execution they wish to inspect from the Workflow Executions list, like above, by clicking on the ``Details`` button. .. image:: Images/AirflowWorkflowExecutionsList.png This will redirect the user to the Workflow Execution Detail page. .. image:: Images/AirflowWorkflowExecutionDetails.png From here the user is presented with three tabs, as seen in the above screen shot, these are; * **Detail** * **Progress** * **Logs** The first of these is the Detail tab, as seen above this tab is the landing page for a Workflow Executions monitoring and management. Here we can see the high level details for a Workflow Execution, including the following: * **Workflow Information: Details on the workflow, including name and project.** * **Workflow Execution Information: Details on Workflow Executions, including name, id, creation detail and overall Workflow status.** * **Workflow Execution Inputs: The inputs provided to the Workflow Execution** * **Output Properties: Button which redirects user to result Data-set** Users can check the progress of a Workflow Execution by simply looking at the overall status seen in the Details tab, or by looking at the Progress tab. .. image:: Images/AirflowWorkflowExecutionProgress.png This tab gives a more detailed view of the Workflow Execution progress. A graph is presented showing each step of the Workflow Execution, as well as the steps status shown by the colour of the node in the graph. More details on each step can be seen by hovering on the node. .. image:: Images/AirflowWorkflowExecutionProgressStepDetails.png The user can view the logs of a Workflow Execution by choosing the Logs tab. .. image:: Images/AirflowWorkflowExecutionLogs.png