Commit bef7478d authored by Jorge S. Mendes de Jesus's avatar Jorge S. Mendes de Jesus
Browse files

update docs and Dockerfile

parent 27b0b8a5
Pipeline #25319 passed with stage
in 1 minute and 20 seconds
......@@ -10,13 +10,6 @@ LABEL "maintainer"="jorge.mendesdejesus@isric.org"
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV DEBIAN_FRONTEND noninteractive
ENV DB_HOST scomp1270.wurnet.nl
ENV DB_NAME isric
ENV DB_USER hass005
ENV DB_PASSWORD 1515NewPW
ENV DB_PORT 5479
ENV MONOLITH_PATH /data/monoliths
RUN apt-get update \
&& apt-get install -y --no-install-recommends postgresql-client build-essential libpq-dev tini
......
WSM
---
# WSM
World Soil Museum - REST API
App structure
-------------
## App structure
REST APP is on [appWSM.py](./appWSM.py) and then calling modules like `resources`. This app is based on:
......@@ -25,17 +23,22 @@ App accesses ISRIC database on `scomp1270.wurnet.nl`.
**NOTE:** To copy files check section below [Monolith picture location](#monolith-picture-location)
### DB access configuration / Data path
### DB access configuration
DB options and monolith path location is picked up from env variables on script: [common/secret.py](./common/secret.py).
DB options are picked up from file [common/secret.py](./common/secret.py). This file needs to be changed if a new DB is used.
NOTE: Jorge did a bad job on leaving DB password on the repository. Password should be on envs.
```bash
DB_HOST=scomp1270.wurnet.nl
DB_NAME=isric
DB_USER=hass005
DB_PASSWORD=<removed>
DB_PORT=5479
MONOLITH_PATH=/data/monolith
```
### DB queries
In the code we have the following queries:
| code path | method/function | SQL |
|:-----------------:|:---------------------:|:-----------------------------:|
......@@ -44,40 +47,35 @@ NOTE: Jorge did a bad job on leaving DB password on the repository. Password sho
**NOTE:** Views need to be refreshed when new data is added to the upstream tables
Monolith picture location
-------------------------
### Monolith picture location
Monoliths used by REST are stored on ISRIC data volume, the access to this volume is very restricted and based on IP. Rserver (scomp1457.wurnet.nl), on this server users haas005, turku001 and ruipe001 have a symbolic link to the monoliths.
There the monolith pictures can be updated or modified
Monolith file copy
------------------
### Monolith file copy
On the Rserver is it better to copy the monoliths from workspace scratch into `ISRIC_Monoliths`. On the home folder of user the command would look like:
```
```bash
turdu001@scomp1457:~$ cp -rv ./ISRIC_Workspace/scratch/Upload_ToRest/combined_processed/* ./ISRIC_Monoliths/
```
The FB-IT mounts are not reliable, check if "ISRIC_Monolith" is accessible, it not mount it:
```
```bash
# mount all volumes from /etc/fstab
sudo mount -a
```
In case of batch file permission it is advisable (NOTE: Only sudo can do it):
```
```bash
find /mnt/ISRIC_Monoliths -type d -exec chmod 777 {} \; #for directories
find /mnt/ISRIC_Monoliths -type f -exec chmod 666 {} \; #for files
```
Monolith image update and implementation
----------------------------------------
## Monolith image update and implementation
To update monolith images, it is necessary:
......@@ -89,49 +87,55 @@ It is advisable to use a rsync software when moving files around.
REST API doesn't make any assumptions on files and will crawl the fold structure for the pictures. It maybe necessary some code changes if there are to many changes.
Local Deployment
----------------
## Local Deployment
This API can be deployed using a docker container. The code resides in the
host system, together with the monoliths images to be served. These images
must be accessible in a folder named `monoliths` at the folder root (same level
as this file). This code folder is then mapped into the container at startup.
as this file). This code folder is then mapped into the container at startup.
The Dockerfile present in this project sets up the API with Gunicorn and nginx.
To build the container image change to the project folder and run:
`docker build -t ldesousa/wsm.rest .`
`docker build -t wsm.rest .`
The resulting image is named `wsm/rest`.
The resulting image is named `ldesousa/wsm.rest`.
To run the image it is necessary to map the right environment variables to connect to the database as well as mount the source code folder and bind port 80 (where nginx listens):
To run the image it is necessary to map the right environment variables to connect to the database as well as mount the source code folder and bind port 8000 (where gunicorn listens):
Docker run command with envfile:
```bash
docker run -dit --env-file /path/to/envfile -v /data/monoliths:/data/monoliths --name wsm.rest -p 8081:80 ldesousa/wsm.rest
docker run -dit \
--env-file /path/to/envfile \
-v /data/monoliths:/data/monoliths \
--name wsm.rest -p 8000:8000 wsm.rest
```
Docker run command with variables inline:
```bash
docker run -dit -e DB_HOST=scomp1270.wurnet.nl -e DB_NAME=isric -e DB_USER=haas005 -e DB_PASSWORD=wur -e DB_PORT=5479 -v /data/monoliths:/data/monoliths --name wsm.rest -p 8081:80 ldesousa/wsm.rest
docker run -dit \
-e DB_HOST=scomp1270.wurnet.nl \
-e DB_NAME=isric \
-e DB_USER=haas005 \
-e DB_PASSWORD=<REMOVED>
-e MONOLITH_PATH=/data/monolith
-e DB_PORT=5479 \
-v /data/monoliths:/data/monoliths
--name wsm.rest -p 8000:8000 wsm.rest
```
Your container will now be available on [localhost:8081](http://localhost:8081)
Your container will now be available on [localhost:8000](http://localhost:8000)
OC deployment
=============
## K8s deployment
OC uses the gitlab webhook to deploy the application on project `isric-files`. Pod application is named `rest-wsm`, and external route is `rest-wsm.isric.org`.
Image build is based on the `python3.6` OC image `openshift/python:3.6` and WSGI system is configure with envs:
```
APP_MODULE=appWSM:app
```
- `ENV` variables are set as k8s secrets on the [ISRIC deployment repository](https://git.wur.nl/isric/ict/k8s-deployments)
Envs are stored on configuration file [here](.s2i/environment)
- Application follows the standard k8s deployment.
Pod mounts the `isric-files/monoliths` on `/opt/app-root/src/monoliths (ro)`
- TODO: Write about semanatic tagging
Image processing and metadata
=============================
## Image processing and metadata
Please consult the specific folder **[README](imageProcessing/README.md)**
"""Password of database"""
"""This is just an example, change to the correct values and rename the file to
secret.py"""
DB={"host":"85.214.241.121","database":"isric", "user":"isis", "password":"@#%951GHKaef"}
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment