General guidelines

The ska-sdp-integration repository integrates the components of the SKA Science Data Processor (SDP).

It is an open GitLab project accessible from:

https://gitlab.com/ska-telescope/sdp/ska-sdp-integration

Cloning the repository

You may clone it via SSH or HTTPS. For SSH:

$ git clone git@gitlab.com:ska-telescope/sdp/ska-sdp-integration.git --recurse-submodules

And for HTTPS:

$ git clone https://gitlab.com/ska-telescope/sdp/ska-sdp-integration.git --recurse-submodules

The --recurse-submodules option makes sure that you clone the ska-cicd-makefile submodule together with the repository, which is an important part for development.

Note

Make sure you include --recurse-submodules when you run git pull as well, to keep up-to-date with the submodule too.

Branch-based development

We follow a branch-based development process. New code is always added to a branch created from master. Then a (draft) Merge Request is created, which needs to be reviewed and approved by CODEOWNERS before it can be merged. Only maintainers of the repository can merge code to master. Only code on the master branch is released.

Using poetry

Although the repository does not contain a Python package (its main artefact is the Helm chart), its tests are written in Python. The project uses Poetry for package management. Make sure you have it installed and configured to your needs. This is needed if you are developing and running tests.

Python dependencies of the project are collected in pyproject.toml. A poetry.lock file is also included, which pins the versions of dependencies the code has been tested with. If you change the requirements in pyproject.toml make sure you also update the poetry.lock file.

The dependencies are organised into four groups in the pyproject.toml file:

test for running the tests

test-tango for running the tests with the Tango device proxy client (optional)

test-dev for developing the tests (formatting and linting utilities)

docs for producing the documentation

Repository structure

The main layout of the repository is as follows.

The most important top level files include:

.gitlab-ci.yml

.readthedocs.yaml

CHANGELOG.md,

CODEOWNERS

CONTRIBUTORS

LICENCE

Makefile

README.md

poetry.lock

pyproject.toml

When developing the code, update these as needed.

Directory structure:

Folder	Description
`.gitlab/ci`	Contains yaml files for GitLab CI job definitions
`.make`	Contains the ska-cicd-makefile submodule (not directly changeable from the SDP repository)
`charts/ska-sdp`	The SDP Helm chart that integrates all the components. The main product of the repository.
`docs`	Documentation and related content
`resources/dashboards`	Taranta dashboard files
`resources/values`	Values files that can be used to customise the SDP installation
`tests`	All of the tests and test-related infrastructure

Documentation

We document using Sphinx and Read The Docs. Every time you change the code in the repository, make sure you review the documentation, and update it as needed.