Find and download the output data

This documentation outlines how to retrieve the output data from a successful scan.

Note

All data products are identified by a unique execution block ID.

How you retrieve the data product depends on where and where you installed your instance of the SDP. You can use the Data Product Dashboard (DPD) if you deployed on an environment such as the ITF. The DPD is a separate sub-system of the SKA software and needs to be deployed on its own. If you deployed SDP locally or in an environment where the Data Product Dashboard is not available, you should have created a local PVC in which the data is stored.

Data Product Dashboard

You can access the data product dashboard at Data Product Dashboard. This is on the STFC SDHP cluster.

More information on the Data Product Dashboard can be found here.

Kubernetes Commands (if deployed Locally or no Data Product Dashboard available)

Connect to Kubernetes pod

In the processing namespace (<processing-namespace>), you need to look for a pod called ska-data-product-access-pod-xxxxx-xxxxx.

Now you need to exec into this container. If you are in k9s, select the container and press the s key. Otherwise, if you are using kubectl, use the following command:

$ kubectl exec -it deploy/ska-data-access -n <processing-namespace> -- sh

Commands needed to download the data product

Once in the container you need to go to the following folder and list the data products in the folder:

$ cd /mnt/data/product
$ ls

You are looking for a folder with the same execution block id that you used to define the AssignResources config.

Now you can exit the container, and run the following command to copy the file to your local

$ kubectl get pods -n <processing-namespace> | grep ska-data-access
$ kubectl cp -n <processing-namespace> <ska-data-access-xxxx-xxxx>:/mnt/data/product/<folder_name> <local_path>