weather_data.md

Specification of external weather data source for VIPS

Tor-Einar Skog, Senior developer, NIBIO

Updated: 2023-02-23

Preface

This is a description of how a weather data provider can make weather data available for forecasting models running in VIPS, NIBIO's system for forecasting agricultural pests and diseases.

General description

• Weather data should be available over the HTTP protocol
• The service should be able to provide a requested set of parameters in a requested time period.
• Input parameters to the service are given as request parameters in either GET or POST format.
• The data should be presented in Json format (http://www.json.org/).
• We do not distinguish between measured data and forecast data. They are presented seamlessly as continuous time series.
• The common way to establish this service is to have a web server in front of a database with weather data.

Output format

-->TODO create a Json schema<--

A sample output is displayed below.

[
    {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"TM", "logIntervalId":1, "value":11.03},
    {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"RR", "logIntervalId":1, "value":0},
    {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"BT", "logIntervalId":1, "value":0}
]

Explanation:

• The square brackets signal the start ([) and end (]) of a list of weather data
• The curly brackets signal the start ({) and end (}) of one weather observation (or forecast)
• Comma separates each weather observation. No comma after the last weather observation.
• Each observation (or forecast) has a time stamp (timeMeasured) in the ISO-8601 format (http://en.wikipedia.org/wiki/ISO_8601), which always includes time zone.
• Each value has a parameter code (elementMeasurementTypeId) that describes what kind of measurement (or, indeed, forecast) this is, and implicitly also the measurement unit. Se appendix A for a table of parameter codes and their description.
• Each value has a logIntervalId, which describes the aggregation or frequency of the measurement. A value of 1 = hourly measuring frequency. See appendix B for a table of logintervalIds.

Input parameters

Parameter name	Description	Format	Example
startDate	Date for first observation in series	ISO-8601 date	2013-08-01
startTime	Hour of day for first observation in series	Zero padded integer	01
endDate	Date for last observation in series	ISO-8601 date	2013-08-15
endTime	Hour of day for last observation in series	Zero padded integer	23
timeZone	Timezone for input data and output data (timeMeasured)	IANA TimeZone database ( http://en.wikipedia.org/wiki/Tz_database )	Europe/Oslo
logIntervalId	aggregation or frequency of the measurement	Number	See appendix B
elementMeasurementTypes	The requested weather parameters	Array of strings	elementMeasurementTypes[]=TM

URL pattern for external weather data source

All weather stations (or measuring points) in the same external weather data source must have a uniform URL pattern. E.g.:

https://lmt.nibio.no/services/rest/vips/getdata/?weatherStationId=[weatherStationId]&elementMeasurementTypes[]=TM&elementMeasurementTypes[]=RR&elementMeasurementTypes[]=UM&logInterval=1h&startDate=2019-05-01&startTime=00&endDate=2019-05-20&endTime=23&timeZone=Europe/Oslo&elementMeasurementTypes[]=BT

The [weatherStationId] is a placeholder for the unique identificator for a particular weather station. It need not be numerical (although that is recommended), but it must be able to easily pass on as part of a URL.

Quality of service

• VIPS expects the series of weather data to be complete, meaning that there are no holes (missing values for one hour, for instance) in it. Some interpolation might be attempted on the receiving end (VIPS), but it's the weather service's responsibility to ensure a complete data set.
• The time series may not contain duplicates, meaning two entries with the same combination of
• timestamp
• parameter type
• log interval
• In the case of missing data (which ideally should not happen), a value/object should not be output for the missing parameter/timestamp/station. If the service contains no data for the given station/parameters/period, an empty array should be output: «[]». A server error status (500) with an error message should be returned unless the GET/POST parameter ignoreErrors=true is set.

Appendix A: Element measurement types (excerpt)

Due to a wish to standardize the names of the codes using international standards, the names of the codes may change in the future. Its definitions will stay the same.

unit	upper_limit	lower_limit	short_name	name
Degrees celcius	45	-55	TM	Mean air temperature in 2m height
Degrees celcius	47	-52	TX	Maximum air temperature in 2m height
Degrees celcius	49	-59	TN	Minimum air temperature in 2m height
Degrees celcius	50	-40	TMf	Mean air temperature in 2m height in fruit tree
Degrees celcius	50	-40	TXf	Maximum air temperature in 2m height in fruit tree
Degrees celcius	50	-40	TNf	Minimum air temperature in 2m height in fruit tree
Degrees celcius	29	-30	TJM5	Mean soil temperature in 5 cm depth
Degrees celcius	29	-13	TJM10	Mean soil temperature in 10 cm depth
Degrees celcius	29	-13	TJM20	Mean soil temperature in 20 cm depth
Degrees celcius	29	-10	TJM50	Mean soil temperature in 50 cm depth
Millimeters	100	-0.5	RR	Rainfall (aggregated)
Percent	145	5	UM	Mean relative humidity in 2m height
Percent	145	5	UX	Maximum relative humidity in 2m height
Percent	145	5	UMf	Mean relative humidity in 2m height in fruit tree
Minutes	60	0	BT	Leaf wetness
Minutes	60	0	BTf	Leaf wetness in fruit tree
Minutes	60	0	BTg	Leaf wetness at ground level (in grass)
Meters/second	30	0	FF	Wind speed in 10m height, last 10 minutes mean
Meters/second	30	0	FF2	Wind speed in 2m height, last 10 minutes mean
Meters/second	30	0	FM	Wind speed in 10m height, 60 minutes mean
Meters/second	21	0	FM2	Wind speed in 2m height, 60 minutes mean
Watt/m2	1000	-25	Q0	Global radiation
mm	1000	-25	EPP	Potential Evaporation (Penman)

Appendix B: Log interval Ids

Log interval id	Log interval
1	Hourly
2	Daily

Appendix C: Existing VIPS weather services

Organization	URL pattern	Contact person
NIBIO	https://lmt.nibio.no/services/rest/vips/getdata/?weatherStationId=[weatherStationId]	Tor-Einar Skog
Swedish University of Agricultural Sciences	http://www.ffe.slu.se/lm/json/downloadJS.cfm?weatherStationID=[weatherStationId]	LantMet