Hello Richard
At present these netCDF4/HDF5 user defined data types are not allowed in CF-netCDF, as you can see from the output of the checker; this is also stated in the CF conventions document.
The same is true for groups, these are not allowed within CF-netCDF at version 1.6.
There has been a fair amount of discussion on this topic recently. I attended a GoESSP meeting last year and a Unidata NetCDF meeting this year where the need to understand the path to get netCDF4 features adopted for CF was much in evidence.
There is also concern about how to manage the data sets not using these features as this are adopted in new versions of CF.
There is a discussion space for a future major version of CF2
https://github.com/cf-convention/CF-2/issues
It would be great to capture your requirements in a new issue here.
I don't think that this should preclude further discussion on adopting these types in a CF 1.x version, CF1.7 may be able to accommodate such things.
It would be interesting to understand reasons why this particular feature may or may not be suitable for CF1.7.
Are there further insights from others in the community on whether these user defined data types can be incorporated into CF1.7?
all the best
mark
________________________________________
From: CF-metadata [cf-metadata-***@cgd.ucar.edu] on behalf of Richard Olav Rud [***@nilu.no]
Sent: 06 July 2016 14:08
To: cf-***@cgd.ucar.edu
Subject: Re: [CF-metadata] Support for compound data types
Dear Alison and Maarten,
Thanks for your quick reply.
To be a little more specific, we are not in need of implementing a new standard name.
By "Compound data types", I mean the netCDF-4 user defined data type (http://www.unidata.ucar.edu/software/netcdf/docs/user_defined_types.html). Using the netcdf-4 format you have support for this data type, but I can not seem to find any information about this subject related to the CF-convention.
Since we have metadata per-variable that changes over time, we were thinking about packing it in a nested structure (compound type). We split up our variable in multiple instances, where each instance could potentially hold a new set of metadata. This could be the case when for example a new data-submitter has taken over the responsibility of reporting data from a specific station or that the instrument-model has changed etc.
So for a given instance of a variable, we could easily have over 100 metadata fields related to stations, data originators, instruments etc.
Basically I have played around creating metadata as compound types, but this does not pass the CF compliance checker (http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl).
From my understanding Maarten, you use groups to separate your metadata from your product? I will look more into this.
So I was wondering if there are any plans to allow for this custom data type in future revisions of the CF-convention, or if there are any other approaches that you might suggest regarding this issue.
All the best,
Richard Rud
Atmosphere and Climate Department (ATMOS)
________________________________
NILU - Norsk institutt for luftforskning
Norwegian Institute for Air Research
PO Box 100, 2027 KJELLER
Besøk/visit: Instituttveien 18, 2007 Kjeller
E-mail: ***@nilu.no
www.nilu.no
________________________________________
Fra: CF-metadata <cf-metadata-***@cgd.ucar.edu> på vegne av Maarten Sneep <***@knmi.nl>
Sendt: 4. juli 2016 18:45
Til: cf-***@cgd.ucar.edu
Emne: Re: [CF-metadata] Support for compound data types
On 07/04/2016 12:29 PM, Richard Olav Rud wrote:
> Dear CF community,
>
> We are currently working on exporting our NASA Ames formated data to cf-compliant
> netCDF for our atmospheric composition database.
>
> Since our metadata changes over time, for most of our variables, we were thinking
> about using compound data types.
> Most of our metadata is of little interest to our average user and therefore we would
> avoid listing them as variables. Even so, we are still obligated to present this
> information.
>
> So I am wondering if there any plans in the near future to implement compound data
> types for the CF metadata conventions?
What level of metadata? per-variable or per-granule or even per-dataset? If the
latter (2): take a look at the methods employed for Sentinel 5 precursor.
http://www.tropomi.eu/documents/level-2-products
http://www.tropomi.eu/science/level-2-products
(the first link contains documentation, the second has (pre-launch) sample products).
Best,
Maarten Sneep
--
KNMI
T: 030 2206747
E: ***@knmi.nl
R: A2.14
_______________________________________________
CF-metadata mailing list
CF-***@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
P Please consider the environment before printing this email and attachments