IOOS QC Releases and Migration Guide

2.1.0 (October 2022)

Changes from 2.0.1:

Special thanks to @ocefpaf for pushing web compatibility forward!

• Core

  • Added ability to run ioos_qc in various python web integration technologies (WASM, pyscript). See PR log for details

  • Updated project to use setuptools_scm

  • Moved CI from Travis to GHA

  • Cleaned up pandas deprecations

  • Added work around for CubicSpline scipy bug

• Docs

  • Improved content and process in #73 and #87

  • Fixed the netCDF example notebook

2.0.1 (August 2021)

• Core

  • Added spike test methods

  • Fixed masked array bug in spike test

2.0.0 (July 2021)

As of this release, backwards compatibility with 1.0.0 has been broken for the QcConfig and NcQcConfig (they no longer exist). Look at the documentation for the new Config objects. The basic test interface and functionality did not change with this release and if you were not using any of the internal complex objects you should be safe to upgrade. Of course please test your own implementations first.

Special thanks to @jessicaaustin, @JessyBarette, @yosoyjay, and @eldobbins for reviewing code, improving documentation, and submitting issues and pull requests.

Changes from 1.0.0:

• Core

  • Added the concepts of Config, Streams, and Stores, see usage documentation for details.

  • Added ability to generate Config objects based on climatologies

• Tests

  • Added a Density Inversion test

  • Attenuated signal test was reworked

  • Bug fix in spike test

  • Added the argo module including speed and pressure tests

  • Improved documentation and notebook examples

1.0.0 (March 2020)

As of this release, all of the existing tests in the QARTOD API should be stable.

Special thanks to Andrew Reed, Seth Foote, Elizabeth Dobbins, Jesse Lopez, and Charles Seaton for reviewing code, improving documentation, and submitting issues and pull requests.

Changes from 0.2.1:

• NcQcConfig updates

  • Improve documentation and tests

  • Update save_to_netcdf method to make it compliant with latest CF and IOOS metadata profiles

    • See cf-conventions #216 and ioos_qc #14)

  • Add option for NcQcConfig to generate aggregate flag (see #15)

• Attenuated signal test rewrite

  • See #12 and #29

• Climatology test updates

  • Use relative time for ranges (#19)

  • span bugfixes

  • periods bugfix

• Fix datetime64[ns] bugs (#16)

• Added performance test suite

• Support for python 3.8

• Removed python 3.5 support

• Improved documentation and notebook examples

If you were previously using https://github.com/ioos/qartod , see the Migrating section below.

0.2.1 (Sept 9, 2019)

• Bugfix: flat line test handles short inp

0.2.0 (Sept 9, 2019)

• Flat line test rewrite

  • Includes: a bug fix to handle a flat line starting at the beginning of the timeseries, massive speed improvements, and a slight change to the algorithm to find flat lines

  • See (PR #11) for more information

• Add Jupyter notebook examples to the docs

0.1.1 (Feb 18, 2019)

• Remove dask as a requirement (#5)

0.1.0 (Feb 8, 2019)

• Complete test overhaul

  • Rewrite of tests to improve readability and maintainability

  • Introduce QcConfig concept

  • Tests use time interval parameters instead of counts (#2)

  • Split out threshold parameters into explicit suspect and fail parameters (#4)

  • Bug fixes

• Remove support for python 2

• Upgrade numpy to 1.14

• Build, test, and documentation improvements

Migrating from 0.0.3 to 0.1.0

Both versions implement the same QARTOD tests using essentially the same algorithms, however the test implementations are much different.

We recommend going test by test and comparing the documentation: 0.0.3 versus latest.

Specific changes to be aware of include:

• Removing support for python 2 and requiring numpy>=1.14.

• Module rename: ioos_qartod.qc_tests.qc.* to ioos_qc.qartod.*

• Test renames

  • attenuated_signal_check --> attenuated_signal_test

  • climatology_check --> climatology_test

  • flat_line_check --> flat_line_test

  • location_set_check --> location_test

  • range_check --> gross_range_test

  • rate_of_change_check --> rate_of_change_test

  • spike_check --> spike_test

• Tests use time interval parameters instead of counts (See #2)

  • This makes the test agnostic about sampling frequency, and thus more generic and human-readable

  • For example, you can specify a rate of change threshold of 0.5 units/second instead of 0.5 units/count

  • Test example: old flat_line_check versus new flat_line_test

• Text use explicit suspect_threshold and fail_threshold parameters instead of a single threshold parameters (See #3)

  • This improves readability, especially for users not familiar with the code

• Introduction of QcConfig object

  • While you can still call test methods directly, we highly recommend using the QcConfig object instead

  • This object encapsulates multiple test configurations, including test parameters, into a single object that can be serialized as JSON for extra portability

  • See the Quickstart notebook example and QcConfig Usage page for more info and examples

0.0.3 (Sept 8, 2016)

See https://github.com/ioos/qartod and https://ioos.github.io/qartod/