From 6dafc60a1f8f19da8d7424f4c86af7772d7a20c0 Mon Sep 17 00:00:00 2001
From: Tor-Einar Skog <tor-einar.skog@nibio.no>
Date: Thu, 22 Sep 2022 16:29:29 +0200
Subject: [PATCH] doc: Add weather data specification
---
README.md | 7 +++-
weather_data.md | 103 ++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 108 insertions(+), 2 deletions(-)
create mode 100644 weather_data.md
diff --git a/README.md b/README.md
index b5626ea..892af19 100644
--- a/README.md
+++ b/README.md
@@ -2,10 +2,13 @@
Tor-Einar Skog, NIBIO
-Last edit: 2021-02-04
+Last edit: 2022-09-22
## About this project
-Since VIPS consists of multiple subsystems, this project is a starting point for VIPS documentation, describing the system as a whole. Documentation to each subsystem is linked from here.
+Since VIPS consists of multiple subsystems, this project is a starting point for VIPS documentation, describing the system as a whole. Documentation to each subsystem is linked from here.
+
+## Further reading
+* [VIPS Weather data specification](./weather_data.md)
## Introduction to VIPS
### What is VIPS?
diff --git a/weather_data.md b/weather_data.md
new file mode 100644
index 0000000..9b1934d
--- /dev/null
+++ b/weather_data.md
@@ -0,0 +1,103 @@
+# Specification of external weather data source for VIPS
+
+## Preface
+This is a description of how a weather data provider can make weather data available for forecasting models running in VIPS, NIBIO's system for forecasting agricultural pests and diseases.
+
+## General description
+* Weather data should be available over the HTTP protocol
+* The service should be able to provide a requested set of parameters in a requested time period.
+* Input parameters to the service are given as request parameters in either GET or POST format.
+* The data should be presented in Json format (http://www.json.org/).
+* We do not distinguish between measured data and forecast data. They are presented seamlessly as continuous time series.
+* The common way to establish this service is to have a web server in front of a database with weather data.
+
+## Output format
+**-->TODO create a Json schema<--**
+
+A sample output is displayed below.
+
+```json
+[
+ {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"TM", "logIntervalId":1, "value":11.03},
+ {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"RR", "logIntervalId":1, "value":0},
+ {"timeMeasured": "2013-08-01T00:00:00+02:00","elementMeasurementTypeId":"BT", "logIntervalId":1, "value":0}
+]
+```
+
+Explanation:
+* The square brackets signal the start (`[`) and end (`]`) of a list of weather data
+* The curly brackets signal the start (`{`) and end (`}`) of one weather observation (or forecast)
+* Comma separates each weather observation. No comma after the last weather observation.
+* Each observation (or forecast) has a time stamp (`timeMeasured`) in the ISO-8601 format (http://en.wikipedia.org/wiki/ISO_8601), which always includes time zone.
+* Each value has a parameter code (`elementMeasurementTypeId`) that describes what kind of measurement (or, indeed, forecast) this is, and implicitly also the measurement unit. Se appendix A for a table of parameter codes and their description.
+* Each value has a `logIntervalId`, which describes the aggregation or frequency of the measurement. A value of `1` = hourly measuring frequency. See appendix B for a table of logintervalIds.
+
+## Input parameters
+|Parameter name |Description |Format |Example |
+|-------------- |----------- |------ |------- |
+|startDate |Date for first observation in series |ISO-8601 date |2013-08-01 |
+|startTime |Hour of day for first observation in series |Zero padded integer |01 |
+|endDate |Date for last observation in series |ISO-8601 date |2013-08-15 |
+|endTime |Hour of day for last observation in series |Zero padded integer |23 |
+|timeZone |Timezone for input data and output data (timeMeasured) |IANA TimeZone database ( http://en.wikipedia.org/wiki/Tz_database ) |Europe/Oslo |
+|logIntervalId |aggregation or frequency of the measurement |Number |See appendix B |
+|elementMeasurementTypes |The requested weather parameters |Array of strings |elementMeasurementTypes[]=TM |
+
+## URL pattern for external weather data source
+All weather stations (or measuring points) in the same external weather data source must have a uniform URL pattern. E.g.:
+
+```
+https://lmt.nibio.no/services//rest/vips/getdata/?weatherStationId=[weatherStationId]&elementMeasurementTypes[]=TM&elementMeasurementTypes[]=RR&elementMeasurementTypes[]=UM&logInterval=1h&startDate=2019-05-01&startTime=00&endDate=2019-05-20&endTime=23&timeZone=Europe/Oslo&elementMeasurementTypes[]=BT
+```
+
+The `[weatherStationId]` is a placeholder for the unique identificator for a particular weather station. It need not be numerical (although that is recommended), but it must be able to easily pass on as part of a URL.
+
+## Quality of service
+* VIPS expects the series of weather data to be complete, meaning that there are no holes (missing values for one hour, for instance) in it. Some interpolation might be attempted on the receiving end (VIPS), but it's the weather service's responsibility to ensure a complete data set.
+* The time series may not contain duplicates, meaning two entries with the same combination of
+* timestamp
+* parameter type
+* log interval
+* In the case of missing data (which ideally should not happen), a value/object should not be output for the missing parameter/timestamp/station. If the service contains no data for the given station/parameters/period, an empty array should be output: «[]». A server error status (500) with an error message should be returned unless the GET/POST parameter ignoreErrors=true is set.
+
+## Appendix A: Element measurement types (excerpt)
+Due to a wish to standardize the names of the codes using international standards, the names of the codes may change in the future. Its definitions will stay the same.
+
+|unit |upper_limit|lower_limit|short_name |name |
+|-------------------|-----------|-----------|-----------|---------------------------------------------------|
+|Degrees celcius |45 |-55 |TM |Mean air temperature in 2m height |
+|Degrees celcius |47 |-52 |TX |Maximum air temperature in 2m height |
+|Degrees celcius |49 |-59 |TN |Minimum air temperature in 2m height |
+|Degrees celcius |50 |-40 |TMf |Mean air temperature in 2m height in fruit tree |
+|Degrees celcius |50 |-40 |TXf |Maximum air temperature in 2m height in fruit tree |
+|Degrees celcius |50 |-40 |TNf |Minimum air temperature in 2m height in fruit tree |
+|Degrees celcius |29 |-30 |TJM5 |Mean soil temperature in 5 cm depth |
+|Degrees celcius |29 |-13 |TJM10 |Mean soil temperature in 10 cm depth |
+|Degrees celcius |29 |-13 |TJM20 |Mean soil temperature in 20 cm depth |
+|Degrees celcius |29 |-10 |TJM50 |Mean soil temperature in 50 cm depth |
+|Millimeters |100 |-0.5 |RR |Rainfall (aggregated) |
+|Percent |145 |5 |UM |Mean relative humidity in 2m height |
+|Percent |145 |5 |UX |Maximum relative humidity in 2m height |
+|Percent |145 |5 |UMf |Mean relative humidity in 2m height in fruit tree |
+|Minutes |60 |0 |BT |Leaf wetness |
+|Minutes |60 |0 |BTf |Leaf wetness in fruit tree |
+|Meters/second |30 |0 |FF |Wind speed in 10m height, last 10 minutes mean |
+|Meters/second |30 |0 |FF2 |Wind speed in 2m height, last 10 minutes mean |
+|Meters/second |30 |0 |FM |Wind speed in 10m height, 60 minutes mean |
+|Meters/second |21 |0 |FM2 |Wind speed in 2m height, 60 minutes mean |
+|Watt/m2 |1000 |-25 |Q0 |Global radiation |
+
+## Appendix B: Log interval Ids
+
+|Log interval id|Log interval|
+|---------------|------------|
+|1 |Hourly |
+|2 |Daily |
+
+# Appendix C: Existing VIPS weather services
+
+|Organization|URL pattern|Contact person|
+|------------|-----------|--------------|
+|NIBIO|http://lmt.nibio.no/agrometbase/export/getVIPS3JSONWeatherData.php?weatherStationId=[weatherStationId]|[Tor-Einar Skog](https://nibio.no/ansatte/tor-einar-skog)|
+|Swedish University of Agricultural Sciences|http://www.ffe.slu.se/lm/json/downloadJS.cfm?weatherStationID=[weatherStationId]|[LantMet](mailto:lantmet@slu.se)|
+
\ No newline at end of file
--
GitLab