Test Data Manager
To simplify and organize handling of data to be used by BDD and system tests a test data manager module has been added to allow the tester to:
upload test data
download test data
list the existing test data under a directory
These files will be hosted and managed on the artefact repository (using nexus at the moment) in a directory tree. Each listing of files under a sub-directory will be a mapping to a component.
This is captured under the named argument artefact_directory
in all three functions. For e.g. if the test data file will be used for subarray tests, then the destination may be like test/subarray/
.
Note the positions of the slashes in the artefact directory in each usage case.
Install
Install the ska-ser-skallop
Python package.
pip3 install -U ska-ser-skallop --extra-index-url https://artefact.skao.int/repository/pypi-internal/simple
Once installed the data-manager script will be available in your Python bin directory.
Usage
Upload Test Data
To upload test data:
from skallop.bdd_test_data_manager.data_manager import upload_test_data
def main():
outcome = upload_test_data("/path/to/test_data", "destination/directory/")
if outcome:
# upload was successful
The command-line utility works as follows:
$ data-manager upload --help
usage: data-manager upload [-h] [-f FILE_PATH] [-a ARTEFACT_DIRECTORY]
optional arguments:
-h, --help show this help message and exit
-f FILE_PATH, --file-path FILE_PATH
The absolute path of the file to be uploaded
-a ARTEFACT_DIRECTORY, --artefact-directory ARTEFACT_DIRECTORY
e.g. path/to/files/
Download Test Data
To download test data:
from skallop.bdd_test_data_manager.data_manager import download_test_data
def main():
path_to_downloaded_file = download_test_data("test_data", "destination/directory/")
with open(path_to_downloaded_file) as test_data:
# do something
The command-line utility works as follows:
$ data-manager download --help
usage: data-manager download [-h] [-f FILE_NAME] [-a ARTEFACT_DIRECTORY]
optional arguments:
-h, --help show this help message and exit
-f FILE_NAME, --file-name FILE_NAME
The name of the file on the repository
-a ARTEFACT_DIRECTORY, --artefact-directory ARTEFACT_DIRECTORY
e.g. path/to/files/
Search for Test Data
To list test data in a directory:
from skallop.bdd_test_data_manager.data_manager import search_for_file
def main():
data_files = search_for_file("*", "/destination/directory")
if "destination/directory/some_specific_file" in data_files:
# do something
$ data-manager download --help
usage: data-manager search [-h] [-f FILTER] [-a ARTEFACT_DIRECTORY]
optional arguments:
-h, --help show this help message and exit
-f FILTER, --filter FILTER
The name of the file on the repository. It can also be
a wildcard which will have all files listed in that
directory
-a ARTEFACT_DIRECTORY, --artefact-directory ARTEFACT_DIRECTORY
e.g. /path/to/files