ioos_qartod.qc_tests.qc

class ioos_qartod.qc_tests.qc.QCFlags

Primary flags for QARTOD.

ioos_qartod.qc_tests.qc.add_qartod_ident(qartod_id, qartod_test_name)

Adds attributes to the QARTOD functions corresponding to database fields. Mostly for internal use

Parameters:

• qartod_id – The QARTOD test identifier, as represented by an integer
• qartod_test_name – The test name as stored in the database.

ioos_qartod.qc_tests.qc.attenuated_signal_check(arr, times, min_var_warn, min_var_fail, time_range=(None, None), check_type='std', prev_qc=None)

Check for near-flat-line conditions where the range of values (max-min) or standard deviation are below minimum thresholds.

Parameters:

• arr – The input array of observed values
• time – An array of timestamps corresponding to the observed values
• min_var_warn – range or standard deviation value prior to flagging values as suspect
• min_var_fail – range or standard deviation value prior to flagging values as bad
• time_range – a 2-tuple indicating the range of times to run the test over
• check_type – A string of either ‘std’ for standard devation or ‘range’ for max - min

ioos_qartod.qc_tests.qc.climatology_check(time_series, clim_table, group_function)

Takes a pandas time series, a dict of 2-tuples with (low, high) thresholds as values, and a grouping function to group the time series into bins which correspond to the climatology lookup table. Flags data within the threshold as good data and data lying outside of it as bad. Data for which climatology values do not exist (i.e. no entry to look up in the dict) will be flagged as Unknown/not evaluated.

ioos_qartod.qc_tests.qc.flat_line_check(arr, low_reps, high_reps, eps, prev_qc=None)

Check for repeated consecutively repeated values within a tolerance eps.

Parameters:

• arr – An array of observed data
• low_reps – number of repetitions prior to data being flagged suspect.
• high_reps – number of repetitions prior to being flagged bad.
• eps – a floating point number indicated the bounds within which the absolute difference of the previous and current measurement will be considered a repeated vlaue.

ioos_qartod.qc_tests.qc.location_set_check(lon, lat, bbox_arr=[[-180, -90], [180, 90]], range_max=None)

Checks that longitude and latitude are within reasonable bounds defaulting to lon = [-180, 180] and lat = [-90, 90]. Optionally, check for a maximum range parameter in great circle distance defaulting to meters which can also use a unit from the quantities library.

Parameters:

• lon – Longitude expressed as a floating point value
• lat – Latitude expressed as a floating point value

Bbox_arr:

A 2x2 array expressed in (lon, lat) pairs

Range_max:

Maximum range expressed in terms of geodesic curve distance. Defaults to units of meters

ioos_qartod.qc_tests.qc.qc_compare(vectors)

Returns an array of flags that represent the aggregate of all the vectors.

Parameters:vectors – An array of arrays of flags of uniform length Returns:An array of aggregated flag data

ioos_qartod.qc_tests.qc.range_check(arr, sensor_span, user_span=None)

Given a 2-tuple of sensor minimum/maximum values, flag data outside of range as bad data. Optionally also flag data which falls outside of a user defined range.

Parameters:

• arr – An array of data
• sensor_span – A 2-tuple consisting of the sensor limits. Data exceeding this threshold will be flagged as bad
• user_span – An optional 2-tuple consisting of the expected data bounds. Data outside of these bounds but within sensor span will be flagged as suspect. If the parameter is not present or is None, then onlyh the sensor_span will be considered and no data can be flagged as suspect.

ioos_qartod.qc_tests.qc.rate_of_change_check(times, arr, thresh_val, prev_qc=None)

Checks the first order difference of a series of values to see if there are any values exceeding a threshold. These are then marked as suspect. It is up to the test operator to determine an appropriate threshold value for the absolute difference not to exceed. Threshold may be expressed as a python quantities unit. For example: import quantities as pq

# more code ... # threshold of 2.5 per hour threshold = 2.5 / pq.hour # run the test with the new threshold results = rate_of_change_check(times, arr, threshold, old_qc)``

Defaults to a rate expressed in terms of seconds if not specified.

Parameters:

• times – An array of times
• arr – An array of observed values

Thresh_val:

Either a float value representing a rate of change over time or a quantities object represneting a value’s rate of change over time

ioos_qartod.qc_tests.qc.set_prev_qc(flag_arr, prev_qc)

Takes previous QC flags and applies them to the start of the array where the flag values are not unknown.

Parameters:

• flag_arr – An array of flag values
• prev_qc – An array of previous QC values corresponding to the start postition of flag_arr

ioos_qartod.qc_tests.qc.spike_check(arr, low_thresh, high_thresh, prev_qc=None)

Determine if there is a spike at data point n-1 by subtracting the midpoint of n and n-2 and taking the absolute value of this quantity, seeing if it exceeds a a low or high threshold. Values which do not exceed either threshold are flagged good, values which exceed the low threshold are flagged suspect, and values which exceed the high threshold are flagged bad.

The flag is set at point n-1.

Parameters:

• arr – The input array of values
• low_thresh – The low value threshold
• high_threshold – The high threshold value
• prev_qc – An array of any previous QC values which were applied. The first element is assumed to correspond to the position of the first element of arr

ioos_qartod.qc_tests.qc.time_series_flat_line_check(arr, low_reps=3, high_reps=5, eps=None, prev_qc=None)

Check for invariate observations and can be applied to all bulk wave parameters.

Parameters:arr – An array of observed data