Find and download the output data

This documentation outlines how to retrieve the output data from a successful scan.

Note

All data products are identified by a unique execution block ID.

How you retrieve the data product depends on where and how you installed your instance of the SDP. You can use the Data Product Dashboard if you deployed on an environment such the ITF. If you deployed locally or in an environment where the Data Product Dashboard is not available, you should have created a local PVC in which the data is stored. More information on the Data Product Dashboard can be found here.

Data Product Dashboard

Note: The Data Product Dashboard is only available if you installed the SDP in a SKA environment which has been connected to the shared dashboard.

You can access the data product dashboard at Data Product Dashboard. This is on the STFC SDHP cluster.

How to use Data Product Dashboard

Once connected to the dashboard, a list of data products will appear. Click on the relevant data product and a side bar should appear with information about the data product. There is a download button if you wish to download the data product. For more information click here.

Kubernetes Commands (if deployed Locally or no Data Product Dashboard available)

Connect to Kubernetes pod

In the processing namespace (<processing-namespace>), you need to look for a pod called ska-data-product-access-pod-xxxxx-xxxxx.

Now you need to exec into this container. If you are in k9s, select the container and press the s key. Otherwise, if you are using kubectl, use the following command:

kubectl exec -it deploy/ska-data-access -n <processing-namespace> -- sh

Commands needed to download the data product

Once in the container you need to go to the following folder and list the data products in the folder:

cd /mnt/data/product
ls

You are looking for a folder with the same execution block id that you used to define the AssignResources config.

Now you can exit the container, and run the following command to copy the file to your local

kubectl get pods -n <processing-namespace> | grep ska-data-access
kubectl cp -n <processing-namespace> <ska-data-access-xxxx-xxxx>:/mnt/data/product/<folder_name> <local_path>