Skip to content
Snippets Groups Projects
Commit 4b7cd71c authored by Tor-Einar Skog's avatar Tor-Einar Skog
Browse files

Merge branch 'main' into 'release'

Main

See merge request !3
parents 3648ad56 55305667
No related branches found
No related tags found
1 merge request!3Main
......@@ -4,12 +4,14 @@ languages=en,nb
[i18n.en]
no_forecast = No forecast
no_risk = No risk
possible_risk = Possible risk
high_risk = High risk
no_risk = No infection risk
possible_risk = Possible infection risk
high_risk = High infection risk
risk = Infection risk
[i18n.nb]
no_forecast = Varsel beregnes ikke
no_risk = Ingen infeksjonsrisiko
no_risk = Ingen fare for angrep
possible_risk = Mulig fare for angrep
high_risk = Høy fare for angrep
high_risk = Stor fare for angrep
risk = Infeksjonsrisiko
#!/usr/bin/python3
"""
Copyright (C) 2023 NIBIO <https://www.nibio.no/>.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
"""
# Author: Brita Linnestad <brita.linnestad@nibio.no>
import os
import subprocess
import glob
import subprocess,glob
from dotenv import load_dotenv
from datetime import datetime, timedelta
from jinja2 import Environment, FileSystemLoader
......
# Naerstad Poteto Blight - spatial version
# Naerstad Potato Blight Model - spatial version
The model is based on several years of experiments where trap plants have been placed next to a potato field infected with late blight. Infections on these plants have been correlated with weather conditions the day of exposure, and this is the data the model is based on. The model predicts if there are favorable conditions for spore production and the following spread, survival and infection of these spores. The model produces an infection risk, and a value of 2.5 corresponds to one spot of late blight on each trap plant exposed. This value of 2.5 is the threshold where the model issues a warning.
## Technical description
###Software requirements
### Software requirements
The model can only be run on Linux, as some of the tools mentioned below are only available on Linux. The development and testing of the model has been done using Ubuntu Linux 22.04LTS.
####CDO and GDAL
#### CDO and GDAL
The heavy lifting in this model is done by the tools CDO and GDAL. These tools need to be installed and available. CDO is only available on Linux.
####Python requirements
The Python requirements are specified in requirements.txt file, and are included in the virtualenv created by the run_NAERSTADMO.sh (see below).
#### Python requirements
The Python requirements are specified in the `requirements.txt` file, and are included in the virtualenv created by the `run_NAERSTADMO.sh` (see below).
#### Input data requirements
The model assumes weather data files named `met_1_0km_nordic-[YYYY-MM-DD].nc` with hourly values of:
* Temperature (TM)
* Relative humidity (UM)
* Precipitation (RR)
* Wind speed (FF)
* Global radiation (Q0)
#### Running the model
It is required that you have set the following environment variables:
```bash
# Path to this code (HOME_DIR + NAERSTADMO)
HOME_DIR=/foobar/gridmodels/
# Path to the weather data files. Expecting hourly values in files named met_1_0km_nordic-[YYYY-MM-DD].nc
WEATHER_DATA_DIR=/foobar/met_1_0km_nordic/2024/
# Start date for the model
START_DATE=2024-05-15
# Use this file to crop the output of the grid
MASK_FILE=Norge_landomrader.csv
# Where the GeoTIFF files will be output
DATA_DIR=/foobar/mapserver/data/NAERSTADMO/
# Where the mapfile will be output
MAPFILE_DIR=/foobar/mapserver/wms/NAERSTADMO/
# Path to the GeoTIFF files used by mapserver (written to mapfile)
MAPSERVER_DATA_DIR=/foobar2/mapserver/data/NAERSTADMO/
# Path to the mapfile used by mapserver (written to mapfile)
MAPSERVER_MAPFILE_DIR=/foobar2/mapserver/wms/NAERSTADMO/
# Path to the mapserver log file for this WMS (written to mapfile)
MAPSERVER_LOG_FILE=/foobar2/mapserver/log/NAERSTADMO.log
# Path for mapserver to store tmp images (written to mapfile)
MAPSERVER_IMAGE_PATH=/foobar2/mapserver/tmp/
# Extent of map (written to mapfile)
MAPSERVER_EXTENT="-1.5831861262936526 52.4465003983706595 39.2608060398730458 71.7683216082912736"
```
...this is the contents of the `env-sample` file
```bash
$ ./run_PSILARTEMP.sh
```
This creates a Python virtualenv, installs all the Python dependencies, runs the model and stores output in a log file.
Alternatively, primarily for development purposes, you can run the Python script PSILARTEMP directly:
```bash
$ ./PSILARTEMP.py
```
####Input data requirements
#### Viewing the result of the model
####Running the model
**TODO** Add more details
####Viewing the resault of the model
![WARNING_STATUS example. Showing Norway and surrounding area](./wms_illustration.png)
# Use this as an example to create your own .env file
# Where your application resides
HOME_DIR=/home/foo/model/
# Path to the weather data
WEATHER_DATA_DIR=in/
# The models start date
# Use this example to create your own .env file
# Path to this code (HOME_DIR + NAERSTADMO)
HOME_DIR=/foobar/gridmodels/
# Path to the weather data files. Expecting hourly values in files named met_1_0km_nordic-[YYYY-MM-DD].nc
WEATHER_DATA_DIR=/foobar/met_1_0km_nordic/2024/
# Start date for the model
START_DATE=2024-05-15
# Path to optional CSV file with polygons for masking result.
# Use this file to crop the output of the grid
MASK_FILE=Norge_landomrader.csv
# Path to the output (GeoTIFF) files as seen from the running model code
DATA_DIR=out/
# Path to the generated mapfile as seen from the running model code
MAPFILE_DIR=mapfile/
# The path to the output (GeoTIFF files) as seen from Mapserver
MAPSERVER_DATA_DIR=/foo/mapserver/data/SEPTREFHUM/
# Where your generated MAPFILE and query templates should be placed
MAPSERVER_MAPFILE_DIR=/foo/mapserver/wms/SEPTREFHUM/
# Where mapserver logs for this WMS are written
MAPSERVER_LOG_FILE=/foo/mapserver/log/SEPTREFHUM.log
# Path to the temporary directory for writing temporary files and images. Must be writable by the user the web server is running as
MAPSERVER_IMAGE_PATH=/foo/mapserver/tmp/
# The value of the EXTENT parameter in Mapserver's mapfile. Units are DD (Decimal degrees)
MAPSERVER_EXTENT="-1.5831861262936526 52.4465003983706595 39.2608060398730458 71.7683216082912736"
\ No newline at end of file
# Where the GeoTIFF files will be output
DATA_DIR=/foobar/mapserver/data/NAERSTADMO/
# Where the mapfile will be output
MAPFILE_DIR=/foobar/mapserver/wms/NAERSTADMO/
# Path to the GeoTIFF files used by mapserver (written to mapfile)
MAPSERVER_DATA_DIR=/foobar2/mapserver/data/NAERSTADMO/
# Path to the mapfile used by mapserver (written to mapfile)
MAPSERVER_MAPFILE_DIR=/foobar2/mapserver/wms/NAERSTADMO/
# Path to the mapserver log file for this WMS (written to mapfile)
MAPSERVER_LOG_FILE=/foobar2/mapserver/log/NAERSTADMO.log
# Path for mapserver to store tmp images (written to mapfile)
MAPSERVER_IMAGE_PATH=/foobar2/mapserver/tmp/
# Extent of map (written to mapfile)
MAPSERVER_EXTENT="-1.5831861262936526 52.4465003983706595 39.2608060398730458 71.7683216082912736"
......@@ -41,23 +41,31 @@ WEB
"wms_languages" "{{ language_codes|join(",")}}" # The first is the default
{% endif %}
"wms_abstract.nb" "
"wms_abstract.nb" "<div id='preamble'>
<h2>Modellbeskrivelse</h2>
<p>Modellen er laget av forsker Ragnhild N&aelig;rstad, p&aring; grunnlag av resultater fra forskingsprosjektet NORPHYT. Modellen beregner risiko for sporeproduksjon med p&aring;f&oslash;lgende spredning, overlevelse og infeksjon p&aring; grunnlag av v&aelig;rdata. For at infeksjonen virkelig skal skje m&aring;det v&aelig;re smitte til stede. </p>
<p>Modellen er basert p&aring; fors&oslash;k over flere &aring;r hvor testplanter av potet, dyrket i veksthus, har blitt eksponert et d&oslash;gn for smitte fra et potetfelt med t&oslash;rrt&aring;e p&aring; friland. </p>
<p>T&oslash;rr&aring;teangrepet p&aring; disse testplantene har blitt sammenholdt med v&aelig;rforholdene dagen de ble eksponert, og basert p&aring; dette er det laget en modell. Modellen beregner hvor gunstig v&aelig;ret er for sporeproduksjon med p&aring;f&oslash;lgende sporespredning, overlevelse og infeksjon av sporene.</p>
<h2>Tolking av varsel</h2>
<p>Modellen er laget slik at risikotallene &oslash;ker med angrepet p&aring; testplantene. Terskelverdien p&aring; 2,5 som gj&oslash;r at varslet blir r&oslash;dt, tilsvarer at det i gjennomsnitt ble funnet en t&oslash;rr&aring;teflekk per testplante. H&oslash;ye verdier betyr at v&aelig;rforholdene gir stor risiko for infeksjon hvis det er smitte til stede.</p>
<p>Kartet viser risiko for angrep av tørråte i potet. Modellen beregner hvor gunstig været er for spredning og infeksjon av tørråte-sporene. Det er størst fare for infeksjon ved varsel om angrep to eller flere dager etter hverandre, og når det er påvist smitte i området.</p>
<p>Varslene beregnes i rutenett a 1x1km basert på reanalyserte data og værprognoser fra Meteorologisk institutt. Varslene oppdateres en gang per døgn, med mulighet for å navigere i tid fra modellens startdato og inntil 2 døgn frem i tid.</p>
<p>Beregning av varsler startes 15.mai og avsluttes 15. oktober.</p>
</div>
<div id='body'>
Risiko for angrep er basert på <a href='https://www.vips-landbruk.no/forecasts/models/NAERSTADMO/' target='new'>Nærstads modell</a>. Nærstads modell beregnes også med data fra værstasjoner, hvor mer detaljerte beregninger kan hentes opp via visningen i kartet på VIPS-forsida.
</div>
"
"wms_abstract.en" "
"wms_abstract.en" "<div id='preamble'>
<h2>Description of the model</h2>
<p>The model is based on several years of experiments where trap plants have been placed next to a potato field infected with late blight. Infections on these plants have been correlated with weather conditions the day of exposure, and this is the data the model is based on. The model predicts if there are favorable conditions for spore production and the following spread, survival and infection of these spores. The model produces an infection risk, and a value of 2.5 corresponds to one spot of late blight on each trap plant exposed. This value of 2.5 is the threshold where the model issues a warning.</p>
<p>The map shows the risk of attack by Potato Late Blight. The model calculates how favourable the weather is for the spread and infection of the Late Blight spores. Ther isk of infection is at its highestt when an infection is predicted two or more days in a row, and when an infection has been confirmed detected in the area. </p>
<p>The warnings are calculated in a 1x1km grid based on reanalyzed data and weather forecasts from the Norwegian Meteorological Institute. The map is updated once per day, with the option to navigate in time from the model's start date and up to 2 days into the future. </p>
<p>Calculation of risk starts on 15 May and ends on 15 October.</p>
</div>
<div id='body'>
Infection risk is based on <a href='https://www.vips-landbruk.no/forecasts/models/NAERSTADMO/' target='new'>Nærstad's model</a>. Nærstad's model is also calculated with data from weather stations, where more detailed calculations can be retrieved via the display in the map on the VIPS front page.
</div>
"
"wms_enable_request" "*"
"wms_title.en" "Potato Late Blight model "
"wms_title.nb" "Potettørråte Nærstads model"
"wms_title.en" "Nærstad's model for Potato Late Blight"
"wms_title.nb" "Potettørråte - Nærstads model"
#"wms_onlineresource" "https://testvips.nibio.no/cgi-bin/NAERSTADMO"
"wms_getfeatureinfo_formatlist" "text/plain,text/html,text/xml"
"wms_accessconstraints" "none"
......@@ -100,7 +108,8 @@ LAYER
STATUS ON
METADATA
"wms_title.nb" " Potettørråte Nærstads model {{ timestep_date }}"
"wms_title.nb" "Potettørråte - Nærstads model {{ timestep_date }}"
"wms_title.en" "Potato Late Blight - Nærstad's model {{ timestep_date }}"
{% for language in languages %}
"wms_abstract.{{language.language_code}}" "
<ul style=\"list-style: none; padding: 0;\">
......@@ -139,7 +148,7 @@ LAYER
END
END
CLASS
NAME "Fare for angrep"
NAME "Stor fare for angrep"
EXPRESSION ([pixel] >= 4)
STYLE
COLOR 255 0 0
......@@ -159,11 +168,12 @@ END # Layer
STATUS ON
METADATA
"wms_title" " Potettørråte Nærstads model risiko {{ timestep_date }}"
"wms_title.nb" "Potettørråte - Nærstads model infeksjonsrisiko {{ timestep_date }}"
"wms_title.en" "Potato Late Blight - Nærstad's model infection risk {{ timestep_date }}"
{% for language in languages %}
"wms_abstract.{{language.language_code}}" "
<ul style=\"list-style: none; padding: 0;\">
<li style=\"margin-bottom: 5px;\"><span style=\"display: inline-block; width: 25px; background: linear-gradient(to right, #0000FF, #FF0000);\">&nbsp;</span> {{language.day_degrees}}</li>
<li style=\"margin-bottom: 5px;\"><span style=\"display: inline-block; width: 25px; background: linear-gradient(to right, #0000FF, #FF0000);\">&nbsp;</span> {{language.risk}}</li>
</ul>
"
{% endfor %}
......@@ -187,40 +197,6 @@ END # Layer
END
END
END # Layer
LAYER
NAME "NAERSTADMO.IR.{{ timestep_date }}"
DATA "{{mapserver_data_dir}}IR_result_{{ timestep_date }}.tif"
TEMPLATE "{{mapserver_mapfile_dir}}query_template_IR.xml" TOLERANCE 1 TOLERANCEUNITS PIXELS
TYPE RASTER
#PROCESSING "BANDS=1" # IR band on top (others invisible, but band values are available in the query template)
#PROCESSING "SCALE=AUTO"
#PROCESSING "NODATA=-1"
STATUS ON
METADATA
"wms_title" "Potettørråte Nærstads model infeksjonsrisiko {{ timestep_date }}"
{% for language in languages %}
"wms_abstract.{{language.language_code}}" "
<ul style=\"list-style: none; padding: 0;\">
<li style=\"margin-bottom: 5px;\"><span style=\"display: inline-block; width: 25px; background: linear-gradient(to right, #0000FF, #FF0000);\">&nbsp;</span> {{language.day_degrees}}</li>
</ul>
"
{% endfor %}
END
CLASSITEM "[pixel]"
CLASS
NAME "IR [0-1]"
EXPRESSION ([pixel] >= 0 AND [pixel] <= 1)
STYLE
DATARANGE 0 1
COLORRANGE 255 255 0 0 0 255
END
END
END # Layer
{% endfor %}
......
......@@ -53,8 +53,8 @@ cd $APP_PATH
# Create and activate the virtual environment
python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --upgrade pip
pip install -r $REQUIREMENTS
python3 -m pip install -q --upgrade pip
pip install -q -r $REQUIREMENTS
# Run the model
echo "==== `date`: Running model" &>> "$LOG_FILE"
......
wms_illustration.png

545 KiB

0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment