performance_analysis
performance_analysis is a command line app written using RASCIL. It helps in analysis of performance files written by rascil_imager.
The performance files can be obtained using a script to iterate over some parameter. For example:
#!/usr/bin/env bash
#
results_dir=${HOME}/results/5km_resource_modelling
for int_time in 2880 1440 720 360
do
mshome=${HOME}/data/int_time${int_time}
for npixel in 512 1024 2048 4096 8192
do
results_dir=${HOME}/data/int_time${int_time}_npixel${npixel}
mkdir -p ${results_dir}
python3 ${RASCIL}/rascil/apps/rascil_imager.py --mode cip \
--clean_nmoment 3 --clean_facets 4 --clean_nmajor 10 \
--clean_threshold 3e-5 --clean_restore_facets 4 --clean_restore_overlap 32 \
--use_dask True --imaging_context ng --imaging_npixel ${npixel} --imaging_pol stokesI --clean_restored_output list \
--imaging_cellsize 5e-6 --imaging_weighting uniform --imaging_nchan 1 \
--ingest_vis_nchan 100 --ingest_chan_per_vis 16 \
--ingest_msname ${mshome}/SKA_MID_SIM.ms \
--performance_file ${results_dir}/performance_rascil_imager_${int_time}_${npixel}.json
done
done
In addition, the memory usage can be tracked using a dask plugin. Currently this requires setting up the dask scheduler with the plugin:
ssh $scheduler dask-scheduler --port=8786 --preload dask_memusage --memusage-csv \
./performance_rascil_imager_${1}_${2}.csv &
Command line arguments
RASCIL performance analysis
usage: performance_analysis [-h] [--mode MODE]
[--performance_files [PERFORMANCE_FILES ...]]
[--memory_file MEMORY_FILE] [--tag TAG]
[--parameters [PARAMETERS ...]]
[--functions [FUNCTIONS ...]]
[--vis_nvis VIS_NVIS] [--verbose VERBOSE]
[--results RESULTS]
Named Arguments
- --mode
Processing mode: line | bar | contour | summary | fit
Default: “summary”
- --performance_files
Names of json performance files to analyse: default is all json files in working directory
- --memory_file
Name of memusage csv file
- --tag
Informational tag used in plot titles and file names
Default: “”
- --parameters
Name of parameters from cli_args e.g. imaging_npixel_sq, used for line (1 parameter) and contour plots (2 parameters)
Default: [‘imaging_npixel_sq’, ‘vis_nvis’]
- --functions
Names of values from dask_profile to plot e.g. skymodel_predict_calibrate
Default: [‘skymodel_predict_calibrate’, ‘skymodel_calibrate_invert’, ‘invert_ng’, ‘restore_cube’, ‘image_scatter_facets’, ‘image_gather_facets’]
- --vis_nvis
Number of visibilities for use if vis_nvis not in json files
- --verbose
Verbose output?
Default: “False”
- --results
Directory for results, default is current directory
Default: “./”