Project Description

There is growing interest in connecting Large Language Models (LLMs) to IOOS data to enable users to ask questions and receive reliable, data-backed responses. The Model Context Protocol (MCP) provides a standard approach for connecting LLMs to external data sources, APIs, and tools.

Several MCP implementations for ERDDAP are already under development, providing a starting point for broader integration with IOOS data systems.

This project will explore how MCP can improve access to IOOS data, identify gaps in existing implementations, and develop new capabilities based on community needs. Participants are encouraged to propose datasets, workflows, or use cases they would like to make accessible through LLMs.

An important focus will be evaluating the reliability of LLM-generated responses. How can we verify that an LLM retrieves the correct data, interprets it appropriately, and produces an accurate response? We will explore approaches for testing and measuring these capabilities.

Potential Development Activities

Activities will be guided by participant interests and may include:

  • Evaluate existing MCP servers: Identify capabilities, limitations, and opportunities for improvement.
  • Expand data access: Develop MCP tools for additional IOOS datasets, APIs, or workflows.
  • Build example applications: Demonstrate how LLMs can use MCP tools to answer questions or support operational workflows.
  • Develop evaluation methods: Establish test cases and metrics for assessing data retrieval, accuracy, and response quality.
  • Identify future priorities: Recommend approaches for improving MCP capabilities across the IOOS community.

Example Use Case: HF Radar Monitoring

One potential application is developing MCP tools to support HF radar network monitoring.

HF radar operators routinely review station availability, data coverage, quality indicators, and ingest history. An MCP server could expose these checks as read-only tools, allowing an LLM-based agent to retrieve information and summarize network conditions.

For example, an agent could help identify station dropouts, compare coverage between time periods, or generate periodic monitoring summaries. These capabilities could reduce manual effort while providing reproducible, data-backed results.

GCOOS HF radar data could serve as a reference implementation, with the goal of developing reusable tools and approaches that other Regional Associations could adopt.

This is one example of a possible development activity rather than a required project outcome.

Expected Outcomes

  • Identify capability gaps in existing MCP solutions.
  • Identify IOOS datasets and workflows that would benefit from MCP integration.
  • Develop or improve MCP tools based on participant interests.
  • Establish practical approaches for evaluating the accuracy and reliability of LLM responses.
  • Demonstrate MCP capabilities through example use cases.
  • Document lessons learned and recommend priorities for future MCP development.

Skills required

Participants with a range of technical backgrounds are welcome. Useful experience includes:

  • Basic understanding of data access patterns and APIs.
  • Familiarity with LLMs and their use of external tools.
  • Python development or experience working with MCP servers.
  • Domain expertise to propose use cases and evaluate the accuracy of responses.

Expertise

Novice

Topic Lead(s)

@jonmjoyce

Functioning Prototype

TBD

Workflow

Tasks:

Updates