Information and Technical Documenation from the DIF Project - NOAA IOOS Data Integration Framework
The NOAA IOOS program initiated development of a Data Integration Framework (DIF) to improve management and delivery of an initial subset of ocean observations. The DIF will establish the technical infrastructure, standards, and protocols needed to improve delivery of atleast six of 20 IOOS core oceanographic variables defined in the U.S. IOOS Development Plan, as well as winds. Read More
The following services are the first to be established by the NOAA IOOS program and its partners to provide access to data.
- NDBC Sensor Observation Service (SOS) This server provides in-situ temperature, salinity, currents, water level, waves and winds data
from National Data Buoy Center (NDBC) moorings, IOOS Regional Coastal Ocean Observing Systems, Deep-ocean Assessment and
Reporting of Tsunamis (DART) buoys, and Tropical Atmosphere Ocean (TAO) buoys. The server is
operated by the NOAA National Weather Service (NWS) NDBC. SOS is an Open Geospatial Consortium (OGC) standard.
- CO-OPS SOS This server provides in-situ temperature, conductivity, currents, water level, and waves data
from the National Water Level Observing Network (NWLON) and the Physical Oceanographic Real-Time
System (PORTS). The server is operated by the NOAA National Ocean Service (NOS) Center for
Operational Oceanographic Products and Services (CO-OPS).
- NDBC THREDDS Data Server (TDS) This server provides gridded surface currents derived from high-frequency radar (HFR) installations
along the coasts. The server supports Web Coverage Service (WCS) and Open-source Project for a Network Data Access Protocol (OpenDAP) and is operated by NDBC. WCS is an OGC standard.
- SECOORA SOS This server provides a variety of in-situ parameters from a collection of data providers. Server is operated by SECOORA (Southeast Coastal Ocean Observing Regional Association). Software documentation on creating this service from the supporting database is at http://code.google.com/p/xenia/wiki/XeniaSOS
- CoastWatch TDS (coming soon!) This server will provide access to chlorophyll concentrations derived from satellite ocean color observations.
(Expected November 2008). This server is operated by NOAA National Environmental Satellite, Data, and Information Service (NESDIS) CoastWatch program.
- IOOS DIF Draft Architecture & Recommendations This set of slides includes a partial set of architecture
diagrams and provides an overview of the data access services used
by the DIF.
- XML Encoding: IOOS DIF XML Schema beta v0.6.1 - This Geography Markup Language (GML)
application schema defines the XML encoding used by the NDBC and CO-OPS SOS to
serve in situ data. This encoding is designed to be complete
and machine-readable, and may be transformed to simplified representations
using code such as the Extensible Styleheet Language Transformation (XSLT) example below.
- Text encoding: IOOS conventions for comma-separated value (CSV) (PDF, 84kB) and tab-separated value (TSV) (PDF, 95kB) encoding. CSV and TSV are offered by the NDBC SOS and eventually by other IOOS SOS.
The CSV is directly readable by common spreadsheet programs. The TSV includes a few specialized column headers to make it more easily readable by the Ocean Data View (ODV) software package.
- DIF Design Document:(pdf, 1,039kb) This document defines the high-level DIF architecture, standards,
conventions, and services to be employed, intersections with NOAA Enterprise
Architecture, and risks and mitigation strategies.
The following software is provided on an as-is basis with no support or warranty. Read the
full disclaimer before downloading.
Server Code
- NDBC SOS software - version 1: This software was developed by the National Data Buoy Center (NDBC) to provide a SOS interface to their MySQL database back-end. Offers data in GML, CSV, TSV and KML encodings.
- GCOOS SOS software beta version 0.6.1(zip, 88kb)This PHP script was modified from the NDBC version to support SQLite database engine. This beta software is still under active development and test.
- THREDDS Data Server (TDS) is open-source software from Unidata that provides OpenDAP and WCS functionality. It is used by the NDBC and CoastWatch.
Format Converters
- IOOS SOS to CSV v0.6.1This XSLT file converts the IOOS SOS v0.6.1 observation values from GML to comma-separated value (CSV) format suitable for use by Excel and other simple applications.
- netCDF to BUFR converter beta v0.1 This sample application was developed at AOML. It converts data from a netCDF format accepted by DIF to the Binary Universal Form for the Representation of meteorological data (BUFR) format maintained by the World Meteorological Organization (WMO). The archive contains C source code, descriptor file, and brief help on compilation and execution.
Other Tools
- Tcl/Tk Example These Tcl/Tk routines are used by the Coastal Inundation customer for retrieving water level observations from the CO-OPS SOS.
- IOOS REGIONAL DATA INTEGRATION FRAMEWORK
IMPLEMENTATION WORKSHOP:(pdf, 13mb)The Integrated Ocean Observing System (IOOS) Regional Data Integration Framework
(DIF) Implementation Workshop was organized with the goal of improving the overall production and execution of IOOS data management across the eleven IOOS Regions. To
accomplish the goal of DIF implementation, a set of objectives were developed in the workshop planning process.
- The NOAA IOOS Data Integration Framework: Initial Implementation Report:Slides (pdf, 315kb) and associated paper (pdf, 125kb)
presented at MTS/IEEE Oceans 2008 conference in Quebec, Canada 2008-09-17.
- Project Plan:(pdf, 168kb)The Project Plan records planning ideas and agreement details, approved scope, cost, and schedule baselines.
- Functional Requirements:(pdf, 333kb)This document defines the functional requirements of the DIF and is a guide to further system documentation and design.
- Concept of Operations:(pdf, 410kb)The Concept of Operations (ConOps) document is the translation of the DIF functional requirements into operational scenarios. The ConOps provides important input into the high-level system design.
- DIF Customer Implementation Project Summary and Performance Assessment Plan:(pdf, 457kb) This document
describes the 4 customer implementation projects used to assess the value of DIF's integrated data. Provides overview of benchmarks, metrics, and
performance assessment methodology.
- IOOS High Level Functional Requirements:(pdf, 168kb)As a part of the IOOS implementation planning effort, the NOAA IOOS Program collected functional requirements from a variety of existing sources and compiled them into this single document. This IOOS High-Level Functional Requirements Document will be used to assist developing more detailed documents as the program focuses its efforts in developing and deploying a National DMAC to serve all IOOS stakeholders.
- IOOS DMAC Concept of Operations:(pdf, 168kb)This document describes the initial high-level concept of operations (ConOps) for the DMAC subsystem; it is not intended to be an implementation or transition plan. The focus of the document is to define the functions and services that IOOS stakeholders desire the DMAC to perform. It does not address the technology or architecture of how it will perform those functions and services. NOAA is currently running a pilot project, the Data Integration Framework (DIF), to address some of the DMAC functionality and some of the technology issues.
[ top ]