I'm being asked to investigate the feasibility of importing data from a NetCDF file into SQL server.
The file could be upwards of 300-400Mb and provided 4 times a day. However I'm only interested in a small percentage of the data. (Maybe as little as 3-5%). We estimate importing ~4000 rows on each run and accumulating the data.
From what I have read a NetCDF file is an XML file. Which may also include binary data.
What would be the best options for importing the data? Should I import the whole file into working table and then select what I want? Or would it be possible to parse/query the file using SQL XML features and only import the data i'm interested in.
I'd appreciate any recommendations from anyone who has first hand experience with NetCDF and what tools they used.