Commit 2002ad0c authored by Jorge S. Mendes de Jesus's avatar Jorge S. Mendes de Jesus
Browse files

Updated documentation

	modified:   .gitignore
	deleted:    .project
	modified:   README.md
	modified:   imageProcessing/README.md
parent c0c5b34a
......@@ -4,3 +4,5 @@ app.log
wsm.rest.env
wsm.rest.sock
logs
.project
.pydevproject
<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
<name>wsm.rest</name>
<comment></comment>
<projects>
</projects>
<buildSpec>
<buildCommand>
<name>org.python.pydev.PyDevBuilder</name>
<arguments>
</arguments>
</buildCommand>
</buildSpec>
<natures>
<nature>org.python.pydev.pythonNature</nature>
</natures>
</projectDescription>
# WSM
WSM
---
World Soil Museum - REST API
# Copyright
(c) ISRIC - Worlds Soil Information
App structure
-------------
REST APP is on [appWSM.py](./appWSM.py) and then calling modules like `resources`. This app is based on:
- Flask
- Flask_restfull
App requires access to ISRIC database to retrieve profile information.
This app was build before the openapi standard and generic request and information can be found at the URL root level: `https://rest-wsm.isric.org/` Please consult the URL for the following examples:
- GET monolith pictures
- GET content filter (WRB classes, country)
- GET limit number of requested pictures
## Database
App accesses ISRIC database on `scomp1270.wurnet.nl`
### DB access configuration
DB options are picked up from file [common/secret.py](./common/secret.py). This file needs to be changed if a new DB is used.
**NOTE:** Jorge did a bad job on leaving DB password on the repository. Password should be on envs.
### DB queries
In the code we have the following queries:
# Licence
Some content is protected and paid, therefore distibution is not allowed.
| code path | method/function | SQL |
|:-----------------:|:---------------------:|:-----------------------------:|
| common/filters.py | filter_classification()|select * from web_isis.vw_isis_profiles_with_classification_geo_img_rest where classification ilike %s"|
| common/filters.py | filter_classification() |select * from web_isis.vw_isis_profiles_with_classification_geo_img_rest where classification ilike %s|
# Deployment
** NOTE: ** Views need to be refreshed when new data is added to the upstream tables
This API is to be deployed within a docker container. The code resides in the
Monolith picture location
-------------------------
Monoliths used by REST are stored on ISRIC data volume, the access to this volume is very restricted and based on IP. Rserver (scomp1457.wurnet.nl), on this server users haas005, turku001 and ruipe001 have a symbolic link to the monoliths.
There the monolith pictures can be updated or modified
Monolith image update and implementation
----------------------------------------
To update monolith images, it is necessary:
- Process images to have the different sizes
- Add them to the ISRIC data volume
- Update DB (web_isis.profiles_with_rest_images?) and views
It is advisable to use a rsync software when moving files around.
REST API doesn't make any assumptions on files and will crawl the fold structure for the pictures. It maybe necessary some code changes if there are to many changes.
Local Deployment
----------------
This API can be deployed using a docker container. The code resides in the
host system, together with the monoliths images to be served. These images
must be acessible in a folder named `monoliths` at the folder root (same level
must be accessible in a folder named `monoliths` at the folder root (same level
as this file). This code folder is then mapped into the container at startup.
The Dockerfile present in this project sets up the API with Gunicorn and nginx.
......@@ -37,11 +93,25 @@ Finally, detach orderly from the running container with the escape sequence:
`Ctrl+P Ctrl+Q`
OC deployment
=============
OC uses the gitlab webhook to deploy the application on project `isric-files`. Pod application is named `rest-wsm`, and external route is `rest-wsm.isric.org`.
Image build is based on the `python3.6` OC image `openshift/python:3.6` and WSGI system is configure with envs:
```
APP_MODULE=appWSM:app
```
Envs are stored on configuration file [here](.s2i/environment)
Pod mounts the `isric-files/monoliths` on `/opt/app-root/src/monoliths (ro)`
Image processing
================
Image processing and metadata
=============================
Pleae consult the specific folder ** (README)[imageProcessing/README.md] **
Please consult the specific folder ** [README](imageProcessing/README.md) **
......
......@@ -193,16 +193,3 @@ Script addCoordsComments.py
This script will add coordinates + comments, this script will use the CSV file with filenames to set the properties. It had some geocoding capabilities but is to old to be properly working
Copy files to location
----------------------
monolith files are accessible on the rserver on `/home/mende012/ISRIC_DATA_RW/monoliths` better to first copy to a tmp folder and them sudo to move them
```
from local computer
scp -v -r * rserver:/tmp/monoliths/
on rserver
~/ISRIC_DATA_RW/monoliths$ sudo cp -r /tmp/monoliths/* .
```
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment