grid_models.md

VIPS Grid models

Tor-Einar Skog, Senior developer, NIBIO

Updated: 2023-11-17

What you will learn

This document describes how to set up a VIPS compatible gridded DSS model, such as in the screenshot below. It also describes briefly how to set up a client to a VIPS compatible DSS gridded model.

The gridded models are using the same "traffic light" color pattern for indicating risk as the location based models. In addition, any other numeric parameter can be displayed as a separate layer, with its own legend and colour scheme.

Prerequisites

• You should be familiar with web maps and WMS services
• You should be familiar with using and manipulating gridded weatherdata files, e.g. NetCDF files or GRIB files.

Architecture of the VIPS gridded model system

Referring to the architecture illustration below:

• How you collect input data and calculate your results is entirely up to you
• The results must be presented as a WMS service following the VIPS conventions. Scroll down for details.

Get started

Get gridded weather data

Gridded weather data typically comes in NetCDF or GRIB format. In VIPS, we have been using NetCDF files, which can be inspected with command-line tools like ncdump and ncview.

ncview is a great tool to familiarize yourself with a NetCDF file

Manipulate data, perform calculations

Performing calculations on gridded data are typically done using tools like R, numPy/netCDF4 or CDO. The latter is by far the quickest, but - depending on the problem - could be cumbersome to work with from a programmer's point of view.

$ cdo -selname,TM -seldate,2023-04-01T00:00:00,2023-12-31T00:00:00 met_1_0km_nordic-2023.nc TM_from_april.nc

The above example extracts one variable (TM - mean temperature) for the period starting with April 1st 2023 and outputs this to a new NetCDF file. To see all CDO operators, run cdo --operators

Create output files

The output files must be data files for a WMS service. A common format for this purpose is GeoTIFF. To convert data from NetCDF to GeoTIFF, we use GDAL, for example:

$ gdal_translate -ot Int16 -of GTiff -b 1 NETCDF:"tmp/result.nc":WARNING_STATUS out/result_WARNING_STATUS_2023-04-01.tif

This selects the first timestep (-b 1) of result.nc, specifically the WARNING_STATUS variable, and creates a GeoTIFF file from this.

Set up a WMS Service

The structure of the WMS Service

VIPS expects the WMS Service to follow certain patterns in order to display it in a map.

Model metadata

You need at least these metadata:

• wms_title - Used to display the model's name (see illustration above)
• wms_abstract - This is used to display the model's description (see illustration above)

WMS Layers

In order for VIPS to be able to utilize your WMS, you need to structure the layers in a specific way:

• The layer names must follow this namespacing pattern: [MODEL_ID].[PARAMETER ID].[YYYY-MM-DD]. For instance, the PSILARTEMP (Carrot Rust Fly) model has these layers (for one day):
  • PSILARTEMP.WARNING_STATUS.2023-04-01
  • PSILARTEMP.DD.2023-04-01

There is one mandatory layer, and that is of course the WARNING_STATUS layer. It must have an integer data type - at least if you're using Mapserver for the WMS

For each layer, you need to specify metadata for classification and legend building, e.g. like in this mapfile excerpt:

LAYER
        NAME "PSILARTEMP.WARNING_STATUS.2023-04-01"
        DATA "/disks/data01/mapserver/data/PSILARTEMP/result_WARNING_STATUS_2023-04-01.tif"
        TEMPLATE "/disks/data01/mapserver/wms/PSILARTEMP/query_template.xml" TOLERANCE 1 TOLERANCEUNITS PIXELS
        TYPE RASTER
        PROCESSING "BANDS=1" # WARNING_STATUS band on top (others invisible, but band values are available in the query template)
        PROCESSING "NODATA=-1"
       
        
        STATUS ON
        METADATA
          "wms_title"     "Carrot rust fly (Psila rosae) temperature model 2023-04-01"  
        END  
          CLASSITEM "[pixel]"
      
        # class using simple string comparison, equivalent to ([pixel] = 0)
      
        CLASS
          NAME "Model not running"
          EXPRESSION ([pixel] >= 0 AND [pixel] < 2) 
          STYLE
              COLOR 112 112 112
          END
        END
        CLASS
          NAME "No infection risk"
          EXPRESSION ([pixel] >= 2 AND [pixel] < 3) 
          STYLE
              COLOR 0 180 87
          END
        END
        CLASS
          NAME "Possible infection risk"
          EXPRESSION ([pixel] >= 3 AND [pixel] < 4) 
          STYLE
              COLOR 255 204 0
          END
        END
        CLASS
          NAME "High infection risk"
          EXPRESSION ([pixel] >= 4) 
          STYLE
              COLOR  255 0 0
          END
        END
    END # Layer

This will result in correct color presentation in the map, and the availability of a legend for the map (see the upper right corner in the screenshot above)