NC file info stripped out of README.md
Edit me

Requirements

  • ERDDAP is deployed using the Docker container as described in the Quick Start.
  • You have access to the source netCDF file.

Adding a new dataset

Steps

  1. Add your nc file to datasets/.
  2. Run GenerateDatasetsXml.sh. Results will output to the console and logs/. 
    $ ./GenerateDatasetsXml.sh
  3. Copy the resulting dataset xml snippet to erddap/content/datasets.xml
  4. Flag dataset for reloading/Restart ERDDAP.

Example

  1. Say you have a file called seward-sealife-center-astra.nc that you’ve added to datasets/.

  2. Navigate to your ~/erddap-gold-standard directory and run the following command:

     $ ./GenerateDatasetsXml.sh

    You should be able to see something similar to this:

     *** GenerateDatasetsXml ***
 Press Enter or type the word "default" (but without the quotes)
   to get the default value.
 Type the word "nothing" (but without quotes) or "" (2 double quotes)
   to change from a non-nothing default back to nothing (a 0-length string).
 Press ^D or ^C to exit this program at any time.
 Or, you can put all the answers as parameters on the command line.
 Results are shown on the screen and put in
 /erddapData/logs/GenerateDatasetsXml.out
 DISCLAIMER:
   The chunk of datasets.xml made by GenerateDatasetsXml isn't perfect.
   YOU MUST READ AND EDIT THE XML BEFORE USING IT IN A PUBLIC ERDDAP.
   GenerateDatasetsXml relies on a lot of rules-of-thumb which aren't always
   correct.  *YOU* ARE RESPONSIBLE FOR ENSURING THE CORRECTNESS OF THE XML
   THAT YOU ADD TO ERDDAP'S datasets.xml FILE.
 For detailed information, see
 http://coastwatch.pfeg.noaa.gov/erddap/download/setupDatasetsXml.html

 The EDDType options are:
 EDDGridAggregateExistingDimension   EDDTableFromDatabase
 EDDGridFromAudioFiles               EDDTableFromEML
 EDDGridFromDap                      EDDTableFromEMLBatch
 EDDGridFromEDDTable                 EDDTableFromErddap
 EDDGridFromErddap                   EDDTableFromFileNames
 EDDGridFromMergeIRFiles             EDDTableFromInPort
 EDDGridFromNcFiles                  EDDTableFromIoosSOS
 EDDGridFromNcFilesUnpacked          EDDTableFromMultidimNcFiles
 EDDGridFromThreddsCatalog           EDDTableFromNcFiles
 EDDGridLonPM180FromErddapCatalog    EDDTableFromNcCFFiles
 EDDTableFromAsciiFiles              EDDTableFromNccsvFiles
 EDDTableFromAudioFiles              EDDTableFromOBIS
 EDDTableFromAwsXmlFiles             EDDTableFromSOS
 EDDTableFromBCODMO                  EDDTableFromThreddsFiles
 EDDTableFromCassandra               EDDTableFromWFSFiles
 EDDTableFromColumnarAsciiFiles      EDDsFromFiles
 EDDTableFromDapSequence             ncdump

  3. Start answering ERDDAP questions:

    Which EDDType (default="EDDGridFromDap")
? EDDTableFromMultidimNcFiles
Starting directory (default="")
? /datasets
File name regex (e.g., ".*\.nc") (default="")
? seward-sealife-center-astra.nc
Full file name of one file (or leave empty to use first matching fileName) (default="")
?
DimensionsCSV (or "" for default) (default="")
?
ReloadEveryNMinutes (e.g., 10080) (default="")
?
PreExtractRegex (default="")
?
PostExtractRegex (default="")
?
ExtractRegex (default="")
?
Column name for extract (default="")
?
Remove missing value rows (true|false) (default="")
?
Sort files by sourceNames (default="")
?
infoUrl (default="")
?
institution (default="")
?
summary (default="")
?
title (default="")
?
standardizeWhat (-1 to get the class' default) (default="")
? 
treatDimensionsAs (default="")
? 
cacheFromUrl (default="")
? 

working...

*** EDDTableFromMultidimNcFiles.generateDatasetsXml
fileDir=/datasets fileNameRegex=seward-sealife-center-astra.nc
sampleFileName= useDimensionsCSV= reloadEveryNMinutes=10080
extract pre= post= regex= colName=
removeMVRows=true sortFilesBy=
infoUrl=
institution=
summary=
title=
externalAddGlobalAttributes=null
Found/using sampleFileName=/datasets/seward-sealife-center-astra.nc

*** generateDatasetsXml finished successfully.

Ctrl-c to interrupt/terminate the program!
  • The resulting file will be generated in ~/erddap-gold-standard/logs/GenerateDatasetsXml.out.

TODO

  1. Copy the results from GenerateDatasetsXml.out into ~/erddap-gold-standard/erddap/content/datasets.xml. 
    <?xml version="1.0" encoding="ISO-8859-1" ?>
      
</erddapDatasets>
   <requestBlacklist />
   <dataset type="EDDTableFromMultidimNcFiles" datasetID="this is the first dataset" active="true"...>
   <dataset type="EDDTableFromMultidimNcFiles" datasetID="this is the second dataset" active="true"...>
   <dataset type="EDDTableFromMultidimNcFiles" datasetID="another_data_set" active="true"...>
   <dataset ... Your dataset from `GenerateDatasetsXml.out` goes here ... >

</erddapDatasets>
  2. Reload ERDDAP by typing the following command: 
    $ docker-compose restart

Troubleshooting

The DasDds tool can help you find errors in datasets.xml, it is an interactive tool that will prompt you for a dataset ID.

$ sh DasDds.sh

You can find a log of what happened at logs/DasDds.out

Tags: