diff --git a/docs/data_model.md b/docs/data_model.md index 9004f4fcd0190e53b369c67b24194d56642d5e4c..1624ce8ea191c19ef26dc293c6292d307948bf36 100644 --- a/docs/data_model.md +++ b/docs/data_model.md @@ -1,7 +1,40 @@ -#Data model +# Data model + +## Organizations +For many aspects of VIPSLogic, the organization is the top level object. One organization may be "VIPS Norge" (VIPS in Norway). Entities that belong to one organization only are: + +* Organization groups (e.g. adviser organizations) +* Users, including their forecast configuratins, field observations, weather stations and farms +* Crop categories +* News messages +* Preparations (pesticides) +* History of batch jobs (task_history) +* Quantification schemas of field observations + + + + +## Forecast configurations and forecast results +This is a key part of VIPSLogic. The forecast configuration decribes how and where a DSS model is being run: +* It is owned by a user +* It is connected to a location +* It is connected to a weather station +* It specifies which model should be run +* It specifies which crop the model is run in +* It specifies which pest the model is run for +* It specifies when (e.g. from March 15th 2020 to August 31st 2020) a model should be run + +The batch system starts regularly (could be every three hours, for instance) and queries the system for forecast configurations that should be run at the given date and time. All these forecast configurations are then queued. For each forecast configuration, the input data are being prepared (weather data, sowing dates etc.) and a request is sent to VIPSCore/VIPSCoremanager. The results from the model run are stored in the table forecast_result. They can be retrieved by any client through the appropriate [web service endpoint](https://logic.vips.nibio.no/public/RESTdocs/apidocs/resource_LogicService.html). + + + + ## Organisms (crops, pests, diseases and weeds) Crops, pests, diseases and weeds are all organisms subject to [classification](https://en.wikipedia.org/wiki/Taxonomy_(biology)#Modern_system_of_classification). In VIPS, they are all placed in the organisms table. Following the rules of the classification system, the table is a tree structure. So each node (bar the root/top nodes) have a parent organism - which may be a family, genus etc. + + + ### Crop categories Each organization (NIBIO, Jordbruksverket, ADAS etc.) has their own set of crop categories. They can be e.g. cereals, fruit, vegetables. Each of these links to one or more organisms. \ No newline at end of file diff --git a/docs/illustrations/data_model_forecast_configuration.png b/docs/illustrations/data_model_forecast_configuration.png new file mode 100644 index 0000000000000000000000000000000000000000..ce9ab178f5fcf78a17e29985c8639893b8634495 Binary files /dev/null and b/docs/illustrations/data_model_forecast_configuration.png differ diff --git a/docs/illustrations/data_model_organism.png b/docs/illustrations/data_model_organism.png new file mode 100644 index 0000000000000000000000000000000000000000..c35184fc630913d0eaaf2cca62419d013a412da0 Binary files /dev/null and b/docs/illustrations/data_model_organism.png differ diff --git a/docs/illustrations/data_model_organization.png b/docs/illustrations/data_model_organization.png new file mode 100644 index 0000000000000000000000000000000000000000..36b270a46b164f82fdec68c7c0d36d25dc30eaab Binary files /dev/null and b/docs/illustrations/data_model_organization.png differ diff --git a/docs/illustrations/vipslogic_login_screen.png b/docs/illustrations/vipslogic_login_screen.png new file mode 100644 index 0000000000000000000000000000000000000000..9e61d25845de673fe932335726217b4d9d0c8e40 Binary files /dev/null and b/docs/illustrations/vipslogic_login_screen.png differ diff --git a/docs/illustrations/vipslogic_start_page.png b/docs/illustrations/vipslogic_start_page.png new file mode 100644 index 0000000000000000000000000000000000000000..62d4ce46cba8b73de8c64574b76471f4caebe185 Binary files /dev/null and b/docs/illustrations/vipslogic_start_page.png differ diff --git a/docs/index.md b/docs/index.md index 69f0b91b3bf9fdeffd2bac305acf8c58275b4b03..40f3346d9e9e72652f93309cf40e31dd974bd482 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,22 +1,53 @@ -# Building VIPSLogic -## But first: A deployment note -Whether you deploy using Docker or Wildfly (or another JEE server), the context path of the app is always /VIPSLogic. However, the app is designed to be accessible at the root level. So you always need to proxy it behind e.g. Apache or Nginx. Here's an example vhost for Apache: +# VIPSLogic development and deployment guide -``` apache -<VirtualHost *:80> - ServerName vipslogic-local.no +[The data model](./data_model.md) - AddDefaultCharset UTF-8 +[Troubleshooting](./troubleshooting.md) - ProxyPass / http://localhost:18080/VIPSLogic/ - ProxyPassReverse / http://localhost:18080/VIPSLogic/ - ProxyPassReverseCookiePath /VIPSLogic / -</VirtualHost> +## Building VIPSLogic + +### Recommended: Building with Docker +#### First: the database +The docker container needs a database. Important parts of the data model [is described in the Data Model chapter](./data_model.md). + +So make sure you have PostgreSQL >= 10 and PostGIS >=2.4 installed. Then create a user called vipslogic. Make sure you enter the vipslogic username and password in the `standalone.xml` file of Wildfly (see chapter below). + + Next up is initalizing the database. Here's how to do it, as the PostgreSQL superuser (In Ubuntu, typically `postgres`) + +``` sql +-- This script must be run as superuser (postgres) +-- Before you deploy VIPSLogic + +SET statement_timeout = 0; +SET lock_timeout = 0; +SET client_encoding = 'UTF8'; +SET standard_conforming_strings = on; +SET check_function_bodies = false; +SET client_min_messages = warning; +SET row_security = off; + +CREATE USER vipslogic WITH PASSWORD :vipslogic_password; + +CREATE DATABASE vipslogic + WITH OWNER = vipslogic + ENCODING = 'UTF8' + TABLESPACE = pg_default + LC_COLLATE = 'en_US.UTF-8' + LC_CTYPE = 'en_US.UTF-8' + CONNECTION LIMIT = -1; + + +-- WE NEED TO DO THIS IN ORDER TO MAKE flywaydb RUN THE INITIAL MIGRATION +-- MAKE SURE YOU REVOKE THIS PRIVILEGE AFTER FIRST DEPLOYMENT OF VIPSLOGIC +-- (Like this: ALTER ROLE vipslogic NOSUPERUSER;) + +ALTER ROLE vipslogic SUPERUSER; ``` -## Docker -### Building + + +#### Building the VIPSLogic image You need the Make sure you're located in the parent folder of the VIPSLogic project. You need these resource files/folders in your current folder: * standalone.xml (see below) @@ -26,14 +57,84 @@ Run `sudo docker build --tag vips/logic:TEST01 -f VIPSLogic/Dockerfile .` -#### Contents of standalone.xml +##### Contents of standalone.xml VIPSLogic is using system properties for instance specific settings. In Wildfly (JBoss), these can be provided in the standalone.xml file. Examples for this is available in the `wildfly_config_examples/` folder. -### Running +#### Running VIPSLogic `sudo docker run --publish 18080:8080 --add-host=vipslogicdb:[YOUR_HOSTS_IP_ADDRESS] --detach --name vipslogic vips/logic:TEST01` -### Troubleshooting by logging in +#### Troubleshooting by logging in `sudo docker exec -it <containername> bash` + +#### Post build database operations +You need to complete these steps in order to have an organization and a super user in the system. Replace `$variable_name` with your info. + +``` sql +INSERT INTO public.organization(organization_name,address1,address2,postal_code,country_code,default_locale,default_map_center,default_map_zoom,default_time_zone,city) +VALUES('$organization_name','$address_1','$address_2','$postal_code','$country_code','$default_language',ST_GeomFromText('POINT($longitude $latitude)',4326),4,'$timezone','$city'); + +-- You need the auto generated organization_id from the first INSERT +INSERT INTO public.vips_logic_user(email,first_name,last_name,organization_id,user_status_id,preferred_locale) +VALUES('$user_email','$first_name','$last_name',(SELECT organization_id FROM public.organization WHERE organization_name='$organization_name'),4,'$default_language'); + +-- You need the auto generated user_id from the previous INSERT +-- To create the $passwordhash, see the python script below +INSERT INTO public.user_authentication(user_id,user_authentication_type_id,username,password) VALUES($userid,1,'$username','$passwordhash'); + +INSERT INTO public.user_vips_logic_role(vips_logic_role_id,user_id) VALUES(1,$userid); +``` +To create the $passwordhash for the INSERT above, you can use this python script + +``` python + +# Python 3 code to demonstrate the +# working of MD5 (string - hexadecimal) + +import hashlib + +# initializing string +# salt must match the no.nibio.vips.logic.MD5_SALT in +# WildFly's standalone.xml file +salt = "FoobarBarfoLoremipsum123" +p = "YourUserPassword" +str2hash = p + salt + +# encoding GeeksforGeeks using encode() +# then sending to md5() +result = hashlib.md5(str2hash.encode()) + +# printing the equivalent hexadecimal value. +print("The hexadecimal equivalent of hash is : ", end ="") +print(result.hexdigest()) # This is the $passwordhash to be used in the SQL above + +``` + +#### Deployment +Whether you deploy using Docker or Wildfly (or another JEE server), the context path of the app is always /VIPSLogic. However, the app is designed to be accessible at the root level. So you always need to proxy it behind e.g. Apache or Nginx. Here's an example vhost for Apache: + +``` apache +<VirtualHost *:80> + ServerName vipslogic-local.no + + AddDefaultCharset UTF-8 + + ProxyPass / http://localhost:18080/VIPSLogic/ + ProxyPassReverse / http://localhost:18080/VIPSLogic/ + ProxyPassReverseCookiePath /VIPSLogic / +</VirtualHost> + +``` + +After successful deployment in Apache (or Nginx), you should see this screen: + + + +Log in with your newly generated superuser credentials, and you get to this screen: + + + + + diff --git a/wildfly_config_examples/20.0.0/standalone.xml b/wildfly_config_examples/20.0.0/standalone.xml index 2f8381f7f0698a6a5cdaabc600f2dcd35d03fb25..e2ff43e3aa77736eb1c41745afa2f93d8384da4f 100644 --- a/wildfly_config_examples/20.0.0/standalone.xml +++ b/wildfly_config_examples/20.0.0/standalone.xml @@ -43,24 +43,24 @@ <property name="no.nibio.vips.logic.VIPSLOGIC_PROTOCOL" value="http"/> <property name="no.nibio.vips.logic.SYSTEM_TIME_OFFSET_MONTHS" value="-13"/> <property name="no.nibio.vips.logic.START_SCHEDULING_ON_BOOT" value="false"/> - <property name="no.nibio.vips.logic.MD5_SALT" value="lkfajoihure98877683736575KLJBYUEFIUYEBUYF"/> + <property name="no.nibio.vips.logic.MD5_SALT" value="XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"/> <property name="no.nibio.vips.logic.VIPSCOREMANAGER_URL" value="http://localhost:8080/VIPSCoreManager"/> <property name="no.nibio.vips.logic.CORE_BATCH_USERNAME" value="vipsbatch"/> - <property name="no.nibio.vips.logic.CORE_BATCH_PASSWORD" value="testpass2014"/> + <property name="no.nibio.vips.logic.CORE_BATCH_PASSWORD" value="XXXXXXXX"/> <property name="no.nibio.vips.logic.SMTP_SERVER" value="mail4.nibio.no"/> - <property name="no.nibio.vips.logic.MESSAGE_ILLUSTRATION_PATH" value="/home/treinar/prosjekter/vips/temp/static/images/messages"/> - <property name="no.nibio.vips.logic.OBSERVATION_ILLUSTRATION_PATH" value="/home/treinar/prosjekter/vips/temp/static/images/observations"/> - <property name="no.nibio.vips.logic.GOOGLE_OPENID_CLIENT_ID" value="852658596506-npgg3d8mkej1asog8jftpabeve7gafsl.apps.googleusercontent.com"/> - <property name="no.nibio.vips.logic.GOOGLE_OPENID_CLIENT_SECRET" value="jmstWQ3LTDQXlOKqLGxgVjfM"/> + <property name="no.nibio.vips.logic.MESSAGE_ILLUSTRATION_PATH" value="/home/developer/vips/temp/static/images/messages"/> + <property name="no.nibio.vips.logic.OBSERVATION_ILLUSTRATION_PATH" value="/home/developer/vips/temp/static/images/observations"/> + <property name="no.nibio.vips.logic.GOOGLE_OPENID_CLIENT_ID" value="XXXXXXXXXXXXXXXXXXXXXX"/> + <property name="no.nibio.vips.logic.GOOGLE_OPENID_CLIENT_SECRET" value="XXXXXXXXXXXXXXXXXXXXXX"/> <property name="no.nibio.vips.logic.ALLOWED_X_DOMAINS" value="vipsweb,localhost"/> <property name="no.nibio.vips.logic.AVAILABLE_LANGUAGES" value="en,nb,zh_CN"/> <property name="no.nibio.vips.logic.DISABLE_MESSAGING_SYSTEM" value="true"/> <property name="no.nibio.vips.logic.weather.VIPS_WEATHER_PROXY_BASE_URL" value="http://localhost:8080/VIPSWeatherProxy"/> <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_USERNAME" value="nibiovips"/> - <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_PASSWORD" value="q22bspFVPwkaohImV21m"/> - <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_CLIENT_ID" value="MetosDemo"/> - <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_CLIENT_SECRET" value="aa8f4b62b72986bac7c84be78836c2c6"/> - <property name="no.nibio.vips.logic.weather.METNOTHREDDS_TMP_FILE_PATH" value="/home/treinar/prosjekter/vips/projects/2017_SpotIT/Task 3.2/"/> + <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_PASSWORD" value="XXXXXXXXXXXX"/> + <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_CLIENT_ID" value="XXXXXXXXXXXXXXXX"/> + <property name="no.nibio.vips.logic.weather.FIELDCLIMATE_API_CLIENT_SECRET" value="XXXXXXXXXXXXXXXXXXXXXX"/> + <property name="no.nibio.vips.logic.weather.METNOTHREDDS_TMP_FILE_PATH" value="/home/tdeveloper/vips/projects/2017_SpotIT/Task 3.2/"/> <property name="no.nibio.vips.logic.USER_COUNTRY_CODES" value="NO,SE,BA,LV,US,FI,LT,CH"/> <property name="no.nibio.vips.logic.AVAILABLE_TIMEZONES" value="Europe/Oslo,Europe/Zurich,Europe/Stockholm,Europe/Vilnius,Europe/Helsinki"/> </system-properties>