README.md 6.23 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
Image processing information
============================

This documentation concerns the image manipulation of the tiff images generated by the museum.

Conversion: script Orginal2ReizeImage.py
----------------------------------------

**LET OP** Conversion only after the picture have metadata

The original images are stored in folder monoliths/original/tif.  The originals are converted into jpg,png and tif based on the following properties: "10per", "1per", "20_200", "40_400", "50per", "original" bellow we have the folder structure

 
```
.
├── 10per
│   ├── jpg
│   ├── png
│   └── tif
├── 1per
│   ├── jpg
│   ├── png
│   └── tif
├── 20_200
│   ├── jpg
│   ├── png
│   └── tif
├── 40_400
│   ├── jpg
│   ├── png
│   └── tif
├── 50per
│   ├── jpg
│   ├── png
│   └── tif
├── 80_800
│   ├── jpg
│   ├── png
│   └── tif
└── original
    ├── jpg
    ├── png
    └── tif
```

Conversion is done using `magick library` that can be used on the command line, for example

```
convert -delete 1--1 ./original/tif/GB-006-def.tif -resize 1%  ./1per/tif/GB-006-def.1per.tif
convert -delete 1--1 ./original/tif/CN-053-def.tif -resize 40x400 ./40_400/png/CN-053-def.40_400.png
```

Note: the -delete 1--1 is due to the fact that some tiffs have 3 layers, one with the full profile, another with the profile without measurement tape and the third with a measurement tape.

To facilitate conversion we use the python script as command: `tiff2Images.py` 

```
tiff2Images.py --makepath --inpath=/home/jesus/git/wsm.rest/imageProcessing/tmp/2019-10-15/original/tif  --outpath=/home/jesus/git/wsm.rest/imageProcessing/tmp2
```

Note: the script uses  `shell=True` (bad, bad, bad !!!)


Adding extra information
------------------------

Extra tags are generated but to include WOSIS, ISIS information. This is done with extra scripts. We have different information according to the possibilities of the image format since JPEG can have more tags than PNG and in turn tiff has only generic tags

**TIFF:**

```
e,g: gdalinfo NL-039-def.50per.tif
Metadata:
  TIFFTAG_ARTIST=ISRIC - Theo Jacobs
  TIFFTAG_COPYRIGHT=ISRIC - CC BY 4.0 http://creativecommons.org/licenses/by/4.0/
  TIFFTAG_DOCUMENTNAME=ZM-010-def.50per.tif
  TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
  TIFFTAG_XRESOLUTION=96.000008
  TIFFTAG_YRESOLUTION=96.000008
```

**JPEG:**

```
e.g: exif  NL-039-def.50per.jpg

EXIF tags in 'FR-002-def.50per.jpg' ('Intel' byte order):
--------------------+----------------------------------------------------------
Tag                 |Value
--------------------+----------------------------------------------------------
Image Description   |WOSIS:63625,ISIS:FR002
X-Resolution        |72
Y-Resolution        |72
Resolution Unit     |Inch
User Comment        |WOSIS:63625,ISIS:FR002
Exif Version        |Exif Version 2.1
FlashPixVersion     |FlashPix Version 1.0
Color Space         |Internal error (unknown value 65535)
GPS Tag Version     |2.0.0.0
North or South Latit|N
Latitude            |48,  7, 59,999999
East or West Longitu|E
Longitude           | 7,  3, 43,999999
Altitude Reference  |Sea level7
Altitude            |1007,700000
Geodetic Survey Data|WGS-84
--------------------+----------------------------------------------------------
```

**PNG:**

```
identify -verbose ZM-004-def.50per.png

 Properties:
    date:create: 2018-09-13T12:39:44+02:00
    date:modify: 2018-09-12T13:49:56+02:00
    exif:Artist: ISRIC - Theo Jacobs
    exif:Copyright: ISRIC - CC BY 4.0 http://creativecommons.org/licenses/by/4.0/
    exif:ExifOffset: 192
    exif:GPSAltitude: 1771100000/1000000
    exif:GPSAltitudeRef: 0
    exif:GPSInfo: 240
    exif:GPSLatitude: 8/1, 55/1, 1200/1000000
    exif:GPSLatitudeRef: S
    exif:GPSLongitude: 31/1, 16/1, 59998799/1000000
    exif:GPSLongitudeRef: E
    exif:GPSMapDatum: WGS-84
    exif:GPSVersionID: 2, 0, 0, 0
    exif:ImageDescription: WOSIS:63211,ISIS:ZM004
    exif:UserComment: 0, 0, 0, 0, 0, 0, 0, 0, 87, 79, 83, 73, 83, 58, 54, 51, 50, 49, 49, 44, 73, 83, 73, 83, 58, 90, 77, 48, 48, 52

 Profiles:
    Profile-exif: 412 bytes
    Profile-iptc: 40 bytes
      Caption[2,120]: WOSIS:63211,ISIS:ZM004
```


To check the image information we can use the following commands:

```
# for png and also tif
identify -verbose AU-002-def.40_400.png
# for jpeg we have exif
exif KE-051a-def.80_800.jpg
```

Python scripts
==============

There are several python script that can be run according to their functionalities. They require `GExiv2` library, which can be a problem to install. The best strategy is using packages:

```
sudo apt install python3-gi python3-gi-cairo gir1.2-gtk-3.0
python3.6 -c "from gi.repository import GExiv2" 
```

**Jorge, Let Op!!** your default python is 3.7 and therefore will not see GExiv2


Database information
--------------------

First step on processing is to get the profiles from WOSIS into a CSV: 

``` 
select distinct p.country_id,p.geom,p.profile_id ,p.profile_code,cl.classification
from web_isis.vw_isis_profile as p 
left join  web_isis.vw_isis_wrb_classification_profile as cl ON cl.profile_id=p.profile_id 
where (p.geom is NOT NULL and cl.classification<>'');   
```
 
Notice that the Postgresql export makes CSV with `;` and we need to change it or save it on a proper format 


Script profilesCodeGeomPicture.py
---------------------------------

This script converts the output CSV into a correct CSV with extra information. The script will crawl the files present and generate a CSV with filename and only with the images found. Therefore the script needs to be set with proper file names and outputs. For example the DB information like `PE021` needs to be transformed into `PE-021-def.80_800.tif`

TODO: Change this script into a python command using click package and rename  it 


Script addCopyright.py
---------------------

This script will add metadata content mainly *Artist* and *Copyright*
 

Script addCoordsComments.py
---------------------------

This script will add coordinates + comments, this script will use the CSV file with filenames to set the properties.  It had some  geocoding capabilities but is to old to be properly working
195