Installing the SDP in a remote cluster

SKA provides various environments for testing its software:

Data Processing (DP) Cluster

Accessing the platform

Normally, Product Owners of DP teams are able to provide k8s access to their team members. If your PO does not have these rights, please point them to this page:

https://confluence.skatelescope.org/display/SWSI/DP+testing+platform+-+Kubernetes+Access

The access to the platform happens via a Kubernetes configuration file (KUBECONFIG file). Once you obtain the file, you can use it by exporting as follows:

$ export KUBECONFIG=<my-config-file>

Replace <my-config-file> with the absolute path to your config file. You will need to run the export command from every terminal window that you want to use for access.

Upon inspection, you will see that you have access to various namespaces, including:

  • dp-shared

  • dp-<team>

  • dp-<team>-<user>

Everyone with Kubernetes access has access to dp-shared; only team members of the given <team> have access to dp-<team>, and only you, the user, have access to the dp-<team>-<user> namespace. Each of these namespaces have a processing equivalent with the same privileges. These are denoted with -p, so e.g. the processing equivalent of dp-shared is dp-shared-p. The processing namespaces normally have more underlying resources.

dp-shared contains a persistent SDP deployment described in the “Testing persistent deployments” section of Automated integration tests.

Installing SDP and interacting with the deployment

We recommend that for testing purposes always use your personal (user) namespaces. With the KUBECONFIG file, you are able to run the same kubectl, k9s, and make commands as you would for a local deployment.

Instructions to install SDP can be found at Installing the SDP. For <control-namespace> use your personal namespace, i.e. dp-<team>-<user>, e.g. dp-dolphin-bob, and for <processing-namespace> use the processing equivalent of it, i.e. dp-<team>-<user>-p, e.g. dp-dolphin-bob-p.

Alternatively, use the SDP Integration Makefile as described here: Using the SDP Makefile. In this case export the following environment variables to use your namespaces, before installing SDP:

$ export KUBE_NAMESPACE=dp-<team>-<user>
$ export KUBE_NAMESPACE_SDP=dp-<team>-<user>-p

Once installed, you will be able to interact with SDP the same way you would from a local deployment. See Interacting with the SDP.

Cleaning up

It is important that you leave any remote namespace in a clean state, where you installed SDP. Do not forget removing any lingering processing blocks either (see instructions in the install guides).

Running the ska-sdp-integration tests

The “Running the tests” section of Automated integration tests gives instructions on what set up you will need in order to run the integration tests in your DP Cluster deployment.