Managing requirements
RASCIL uses Poetry for dependency management and packaging.
The poetry.lock file is provided with the repository to indicate what versions of
dependencies the code has been tested with.
Manually updating the requirements
The pyproject.toml file needs to be regularly updated to test the
code with newer major versions of dependencies, which is not done automatically.
Once can also run poetry lock to update the lock file with all latest
minor and patch versions of dependencies. Please make sure you use Poetry 1.8.
Process automation
The compile-requirements scheduled job updates the lock file
once a week using skart and poetry lock.
For this, we use the standard make target deps-update-release.
The target and its usage is also explained here in the
SDP Integration documentation
The following steps are executed in the job:
run
make deps-update-releasecheck if there are changes compared to the existing remote files
if there, create and check out a new branch
commit and push the changes to the new branch
create a Merge Request (MR) of the new branch into the source branch
assign the MR
if there aren’t any changes, do nothing
The tests are not run as part of this pipeline, because the MR created at the end of will have the tests run as part of its own pipeline.
The assignee now has the responsibility of keeping track how the pipeline of this new MR does. If it succeeds, then it should be merged to master. If it fails, then the failing tests should be checked and the reasons for failure should be fixed. Packages should not be pinned, just because tests are failing, unless there is a very good reason for it. Packages pinned should be regularly revisited and if possible, unpinned.