Configuring the Script

This section describes the configuration of the vis-recieve script, covering the options available in the JSON configuration string supplied to AssignResources, the required environment variables, the helm chart arguments, and setting receive addresses.

Configuration Parameters

Simplified ParamsReference API

The vis-receive script can be configured through the following Processing Block parameters, all of which are optional:

SummaryVisReceiveParams

vis-receive script parameters

Name	Description	Default
channels_per_port	Number of frequency channels that will be sent to each port.	1
processes_per_node	Number of processes to start per node, useful if CPU usage becomes a problem.	null
num_nodes	Number of nodes to allocate.	null
port_start	The first port each receiver will open to listen for data.	21000
transport_protocol	The transport protocol the receiver should use ("tcp" or "udp").	"udp"
reception_network	The network the receiver will bind itself to. Overrides use_network_definition when set. hardware indicates the host’s NIC software indicates the internal Kubernetes network auto lets the script determine if hardware support is available	"auto"
allow_hostnames_in_receive_addresses	Whether hostnames can be written into the SDP receive addresses or not. This setting is only relevant for cases where the receiver binds itself to the Kubernetes network. In these case resolving hostnames into IPs has the effect that the Processing Block will take longer to become READY, which in turn means the subarray will take longer to transition to IDLE. When the receiver binds itself to the host’s NIC this setting has no effect, as the receive addresses are already filled with IPs.	false
dry_run	When set, produces no deployment of the vis-receive chart. Useful for debugging of the output Helm values and the generated receive addresses, which are filled with IPs in the 127.0.0.0/8 link-local network.	false
telescope_model	Configuration for Telescope Model access.	{"layout_key": "instrument/ska1_low/layout/low-layout.json", "station_data_key_path": "stations"}
signal_display	Settings for the signal-display processor.	{"image": null, "max_message_bytes": 52428800, "metrics": ["all"], "nchan_avg": 5, "rounding_sensitivity": 5, "version": null, "window_count": 5}
mswriter	Settings for the mswriter processor.	{"expires_at": "2099-12-31T00:00:00Z", "phase": "SOLID", "telescope_name": null, "upper_triangular_baselines": false}
rcal	Settings for the rcal processor.	{"accumulation_time": 10, "autoflag": true, "ddcal": null, "enabled": false, "flip_uvw": false, "flux_limit": null, "fov": 5.0, "gain_headroom": 0.1, "low_gain_threshold": 0.1, "output_h5": null, "preproc_ave_frequency": null, "preproc_ave_time": null, "rfi_masks": null, "scale_factor": null, "scale_gains": false, "sky_model": "", "threshold_headroom": false, "uv_min": null}

An example set of parameters can be found here.

Processors

The processors parameter is a dictionary with processor specifications. The keys in the dictionary are a simple name of the processor, while the value is a dictionary with a container-like specification as understood by the vis-receive Helm Chart.

This script comes with built-in processor definitions, which are defined as files under the [processors subdirectory](processors/):

• mswriter: takes incoming visibility data and writes it into a Measurement Set.

• rcal: performs gain calibration on incoming visibility data.

• signal-display-metrics-all: takes incoming visibility data and calculates all available Signal Display metrics, which are then published to the SDP data queues. (deprecated, use configuration above)

• signal-display-metrics-basic: takes incoming visibility data and calculates basic and fast Signal Display metrics, which are then published to the SDP data queues. (deprecated, use configuration above)

• signal-display-metrics-amplitude: takes incoming visibility data and calculates Amplitude vs Frequency Signal Display metrics, which are then published to the SDP data queues. (deprecated, use configuration above)

• signal-display-metrics-phase: takes incoming visibility data and calculates Phase vs Frequency Signal Display metrics, which are then published to the SDP data queues. (deprecated, use configuration above)

If a key in the processors dictionary refers to one of the built-in processor definitions, then the corresponding value is used to override the built-in definition; otherwise the value must fully define a processor.

Examples:

• Only use built-in mswriter processor, with its default settings:

  mswriter: {}
  

• Use the built-in mswriter processor, with its default settings, plus a processor called my-signal-display-metrics, which is described in full:

  mswriter: {}
  my-signal-display-metrics:
    name: my-signal-display-metrics
    image: artefact.skao.int/ska-sdp-qa-metric-generator
    version: 0.20.0
    command:
    - plasma-processor
    - ska_sdp_qa_metric_generator.plasma_to_metrics.SignalDisplayMetrics
    - --readiness-file
    - /tmp/processor_ready
    - --use-sdp-metadata
    - "False"
    - --metrics
    - all
    readinessProbe:
    - initialDelaySeconds: 5
      periodSeconds: 5
      exec:
        command:
        - cat
        - /tmp/processor_ready
  

• Use a different version of the built-in mswriter and rcal processors:

  mswriter:
    version: 1.2.3
  rcal:
    version: 2.3.4
  

Chaining Processors

Processors have the ability to be chained together. This interface between each is a Visibility object.

Any processor can be a chainable processor, but at the level of the script we have provided one example, the time-mswriter processor. So called because it writes visibility data to a Measurement Set, but also calculates the time average of the data before writing it.

- name: "sdp-processor"
- image: "registry.gitlab.com/ska-telescope/sdp/ska-sdp-realtime-receive-integration/ska-sdp-realtime-receive-integration"
- version: "0.3.0"
- command:
  - "sdp-processor"
- args:
  - "['ska_sdp_realtime_receive_integration.processors.AverageTimeProcessor','realtime.receive.processors.sdp.mswriter_processor.MSWriterProcessor']"
  - "--readiness-file"
  - "/tmp/processor_ready"
  - "--timestep 2"
  - "output.ms"
- readinessProbe:
  file: "/tmp/processor_ready"

The args parameter contains a list of processor classes that will be chained together. In this case the AverageTimeProcessor will be called first, followed by the MSWriterProcessor. The output of the first processor is passed to the second processor.