maps with different projection ?

Smallpox vs. largepox, gen2 vs gen5, early war vs. peaceful alliances. Which is your favourite gaming style?
Post Reply
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

meynaf wrote:but i can't handle GeoTiff files at all
^ You already have! I think that I may have contributed to some confusion about that. When I first started looking for a way to do a live lookup of land classification for a given spot on Earth via coordinates, I immediately found Google Earth Pro, which I posted an image of several posts back in this topic. In that image, the software displays high-resolution satellite images of Earth, and a second layer of classification data. From that I concluded that data sets included both the photographic data and the land classification data. Which seemed annoying because I don't need the photographic layer at all, but rather only the land class data -- but it appeared that I would be forced into downloading both. That is -not- true of the GeoTiff data "sets". The tiff files are just tiff files but they do not represent photographic images. They are images of the land classification values. There aren't two sets of data in the GeoTiff format, but only one set -- the land classification values in sequence within a tiff file. The tiff file of GlobCover 2009 data which I resized and posted for you -is- a GeoTiff. It has no photographic information at all, just the land class values in place of photographic pixel values. That's the only difference in a GeoTiff. It's still a proper tiff file. The pixel data is not of a photo, but rather classification values.

You're right about the somewhat futile nature of trying to translate real-world land into Freeciv maps, but short of completely changing the entire Freeciv software to support a totally different map structure, I don't see any options other than trying any one of several imperfect methods... In v2.6 at least, there are only 14 tile types.

ocean
deep ocean
lake

inaccessible

arctic/glacier
desert
forest
grassland
hill
jungle
mountain
plains
swamp
tundra

Of those, only a few types inherently indicate elevation/depth: Four land: mountain, hill, plains, grassland. Two water: ocean, deep ocean.

A few others -may- indicate the extent of elevation when used near some of the prior four: arctic/glacier when used near mountains may indicate high elevation. Forest used near mountains may indicate lower elevation of mountains (below the tree line.)

A couple indicate relative elevation to nearby tiles: Swamp indicates the swamp is lower elevation relative to adjacent flat-land tiles which aren't swamp/lake. Same with lake tiles, the lake must have lower elevation relative to nearby flat-land tiles such as grassland or plains.

But all of it can be rendered nonsensical in a Freeciv map, even during game-play. Engineers can create a swamp in the middle of a desert mountain range, for instance. A map can have a lake tile in the middle of the surface of Antarctica, in which a boat can be produced, despite that the lake should be frozen solid.

In short, there is no perfect way to translate varying real-world terrain maps into Freeciv maps. We are left with two choices: just translate the land cover classifications into single tile types without trying to make it make sense, or try to use a consistent system to represent more information about the land than individual Freeciv tile types convey such as by consistent/systematic use of mixed terrain tiles to indicate something more than individual tile types convey. Mountains mixed with forest to show that the land is -both- mountainous and forest-covered, for example. ::shrugs::

Then there are some additional issues, such as rivers that run uphill. :)
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: ^ You already have! I think that I may have contributed to some confusion about that. When I first started looking for a way to do a live lookup of land classification for a given spot on Earth via coordinates, I immediately found Google Earth Pro, which I posted an image of several posts back in this topic. In that image, the software displays high-resolution satellite images of Earth, and a second layer of classification data. From that I concluded that data sets included both the photographic data and the land classification data. Which seemed annoying because I don't need the photographic layer at all, but rather only the land class data -- but it appeared that I would be forced into downloading both. That is -not- true of the GeoTiff data "sets". The tiff files are just tiff files but they are not photographic images. They are images of the land classification values. There aren't two sets of data in the GeoTiff format, but only one set -- the land classification values in sequence within a tiff file. The tiff file of GlobCover 2009 data which I resized and posted for you -is- a GeoTiff. It has no photographic data at all, just the land class values in place of photographic pixel values. That's the only difference in a GeoTiff. It's still a proper tiff file. The pixel data is not of a photo, but rather classification data.
What you gave me (and i could use) was pbm, not tiff. That, i can handle. Png too. But not tiff, at least not compressed ones (and especially if they are enormous).

Molo_Parko wrote: You're right about the somewhat futile nature of trying to translate real-world land into Freeciv maps, but short of completely changing the entire Freeciv software to support a totally different map structure, I don't see any options other than trying any one of several imperfect methods... In v2.6 at least, there are only 14 tile types.

ocean
deep ocean
lake

inaccessible

arctic/glacier
desert
forest
grassland
hill
jungle
mountain
plains
swamp
tundra

Of those, only a few types inherently indicate elevation/depth: Four land: mountain, hill, plains, grassland. Two water: ocean, deep ocean.

A few others -may- indicate the extent of elevation when used near some of the prior four: arctic/glacier when used near mountains may indicate high elevation. Forest used near mountains may indicate lower elevation of mountains (below the tree line.)

A couple indicate relative elevation to nearby tiles: Swamp indicates the swamp is lower elevation relative to adjacent flat-land tiles which aren't swamp/lake. Same with lake tiles, the lake must have lower elevation relative to nearby flat-land tiles such as grassland or plains.

But all of it can be rendered nonsensical in a Freeciv map, even during game-play. Engineers can create a swamp in the middle of a desert mountain range, for instance. A map can have a lake tile in the middle of the surface of Antarctica, in which a boat can be produced, despite that the lake should be frozen solid.

In short, there is no perfect way to translate varying real-world terrain maps into Freeciv maps. We are left with two choices: just translate the land cover classifications into single tile types without trying to make it make sense, or try to use a consistent system to represent more information about the land than individual Freeciv tile types convey such as by consistent/systematic use of mixed terrain tiles to indicate something more than individual tile types convey. Mountains mixed with forest to show that the land is -both- mountainous and forest-covered, for example. ::shrugs::
I know. What i fear is more big mistakes than small interpretation issues. Like putting forests where in real life there is vegetation but no tree, or a desert in a place that's not really barren.
But maybe i should just use arbitrary rules, generate a "nice" map and be happy with it.

Molo_Parko wrote: Then there are some additional issues, such as rivers that run uphill. :)
In civ1 rivers were a terrain type by themselves. I could eventually do the same ;)
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Yes, you're right. I gave you a GeoPBM not a GeoTiff, but aside from file formatting it would be the same with an uncompressed tiff.
I know. What i fear is more big mistakes than small interpretation issues. Like putting forests where in real life there is vegetation but no tree, or a desert in a place that's not really barren.
^ Right, if we try to indicate both elevation and land cover, one over-rides the other on at least some tiles. If land cover is applied first, then trying to mix-in hills and mountains might eradicate forests/jungle. If land cover is last, then forest, etc. may eradicate hills and mountains on some tiles. I don't see any perfect method. If we use only one of the two datasets, it would give either accurate land cover but no elevation at all (no hills or mountains), or accurate elevation with somewhat inaccurate land cover (since a tile can't be both mountain and forest simultaneously.) Between the two choices, I think that accurate land cover without elevation would result in an "easier" game. More mountains/hills would result in a less-easy game. I suppose one could simply make three versions of the map for three scenarios instead of one: Accurate land cover; accurate elevation; mix of both. Then the player could try all three and pick whichever map is desirable at any time.

It is possible to programmatically check e03_ layer river paths to ensure that they only run "downhill". I did that in a scenario I posted, "Rapids Canyon Valley". Basically I determined which tiles indicate elevation and ranked them in order from highest to lowest "mhpgs" and then automate following the river tiles and check that the corresponding terrain layer tile (using same x,y from river layer tile) is always equal or lower in height to the prior river tile. So a river which starts on mountain may proceed to mountain, hill, plains, or grassland (or forest, jungle, etc.) But a river tile which has reached grassland may never go back upward to plains, hill, or mountain, and so on. Once a river hits swamp or ocean, it's done. It would probably be better-still to do that by actual elevation data for the corresponding tile.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Yes, you're right. I gave you a GeoPBM not a GeoTiff, but aside from file formatting it would be the same with an uncompressed tiff.
Sure, but the GeoTiff files i've seen weren't uncompressed ;)

Molo_Parko wrote: ^ Right, if we try to indicate both elevation and land cover, one over-rides the other on at least some tiles. If land cover is applied first, then trying to mix-in hills and mountains might eradicate forests/jungle. If land cover is last, then forest, etc. may eradicate hills and mountains on some tiles. I don't see any perfect method. If we use only one of the two datasets, it would give either accurate land cover but no elevation at all (no hills or mountains), or accurate elevation with somewhat inaccurate land cover (since a tile can't be both mountain and forest simultaneously.) Between the two choices, I think that accurate land cover without elevation would result in an "easier" game. More mountains/hills would result in a less-easy game. I suppose one could simply make three versions of the map for three scenarios instead of one: Accurate land cover; accurate elevation; mix of both. Then the player could try all three and pick whichever map is desirable at any time.
It is not necessary to have one dataset totally pre-empt another.
There could be specific rules for each land cover type, when applying elevation data.

Currently elevation data is ignored. I start with land cover, then ocean depth.
And there are already issues.
Normally, depth value=255 indicates land. In south pole area (which does not have land cover data), implies arctic.
Land cover=21 indicates water. Normally they should match but they don't always do.
I supposed water with value=255 is just shallow water.
But land with other value, what does it mean ? There are few pixels, but there are some. I used a distinct color to see them, but it's not a terrain type.

Now, to use elevation data, what to do ?
If i just use a threshold, you can bet mountains will always have hills nearby. High plains would be hills. Hills located at lower altitudes would be removed.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

meynaf wrote:But land with other value, what does it mean ? There are few pixels, but there are some. I used a distinct color to see them, but it's not a terrain type.
The only "other" values I can find in the pbm file are the four text lines of header info at the beginning of the file.

Code: Select all

00000000  50 35 0a 32 31 36 30 30  0a 39 33 30 30 0a 32 35  |P5.21600.9300.25|
00000010  35 0a                                             |5.|
I had the computer count how many occurrences of each byte's decimal value are in the file, which produced the list below.

Code: Select all

 10: 4
 11: 807836
 14: 2764183
 20: 3039062
 30: 3266736
 40: 3775241
 48: 4
 49: 1
 50: 2479018
 51: 1
 53: 3
 54: 1
 57: 1
 60: 950213
 70: 1312878
 80: 1
 90: 5354835
100: 1520383
110: 2216058
120: 1975077
130: 2843467
140: 2980602
150: 7575963
160: 235026
170: 37244
180: 532119
190: 120563
200: 7521670
210: 146437106
220: 3124973
230: 9749
As an example, 10 decimal is 0A hex and it occurs four times in the file. Each instance is one of four newline/line endings from the file header where the format info is specified. The first line is the "magic number" that represents file type, "P5" is a PBM file. The second line shows the width for each line of pixels (21600 is the width you requested), the third line is the quantity of lines. The fourth and last line of header/file-format info is the maximum value for any single pixel which is 255 (8 bit grayscale.) The 18 bytes from those first four (text) lines aren't part of the pixel data, they are format metadata.

The decimal value 53 (hex 35) is ASCII character "5" and there are three of them in the file, all in the file header/format info - one from the first line where the file type is indicated as "P5". The other two are from the fourth line where the maximum pixel value is "255".

The first byte of the file is hex value 50 which is decimal value 80, and ASCII character "P" -- it is the P from "P5" that indicates the file is PBM.

Those first four lines should not be converted to a Freeciv map or to pixel data, but rather skipped during data conversion.

The single most-used land cover classification in the file is decimal value "210 - water bodies", as in ocean.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:
meynaf wrote:But land with other value, what does it mean ? There are few pixels, but there are some. I used a distinct color to see them, but it's not a terrain type.
The only "other" values I can find in the pbm file are the four text lines of header info at the beginning of the file.

Code: Select all

00000000  50 35 0a 32 31 36 30 30  0a 39 33 30 30 0a 32 35  |P5.21600.9300.25|
00000010  35 0a                                             |5.|
I had the computer count how many occurrences of each byte's decimal value are in the file, which produced the list below.

Code: Select all

 10: 4
 11: 807836
 14: 2764183
 20: 3039062
 30: 3266736
 40: 3775241
 48: 4
 49: 1
 50: 2479018
 51: 1
 53: 3
 54: 1
 57: 1
 60: 950213
 70: 1312878
 80: 1
 90: 5354835
100: 1520383
110: 2216058
120: 1975077
130: 2843467
140: 2980602
150: 7575963
160: 235026
170: 37244
180: 532119
190: 120563
200: 7521670
210: 146437106
220: 3124973
230: 9749
As an example, 10 decimal is 0A hex and it occurs four times in the file. Each instance is one of four newline/line endings from the file header where the format info is specified. The first line is the "magic number" that represents file type, "P5" is a PBM file. The second line shows the width for each line of pixels (21600 is the width you requested), the third line is the quantity of lines. The fourth and last line of header/file-format info is the maximum value for any single pixel which is 255 (8 bit grayscale.) The 18 bytes from those first four (text) lines aren't part of the pixel data, they are format metadata.

The decimal value 53 (hex 35) is ASCII character "5" and there are three of them in the file, all in the file header/format info - one from the first line where the file type is indicated as "P5". The other two are from the fourth line where the maximum pixel value is "255".

The first byte of the file is hex value 50 which is decimal value 80, and ASCII character "P" -- it is the P from "P5" that indicates the file is PBM.

Those first four lines should not be converted to a Freeciv map or to pixel data, but rather skipped during data conversion.

The single most-used land cover classification in the file is decimal value "210 - water bodies", as in ocean.
Terrible misunderstanding here. This is not at all what i was asking :roll:
I know what the pbm header is and i can count bytes myself, thanks.

Let's restart.
I was talking about ocean depth.
Normally, land should have depth value=255.
But sometimes other values are found. I marked them as a non existing terrain type when showing my image.
I was asking what these inconsistent values could mean...
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Ahh, that makes more sense. Where did you get the depth values? I mean, what is the data source? Maybe there is some documentation at the site which wasn't included with the image file?
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Ahh, that makes more sense. Where did you get the depth values? I mean, what is the data source? Maybe there is some documentation at the site which wasn't included with the image file?
This is simple bathymetry. Nothing special, just one-byte levels. I linked to it in a previous post :
http://forum.freeciv.org/f/viewtopic.ph ... 64#p104164
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

The page at your link says that the image was created with data from "the General Bathymetric Chart of the Oceans (GEBCO) produced by the British Oceanographic Data Centre" but the NASA page doesn't give any details about which GEBCO data set(s) were used, other than mentioning that the images were created in 2005.

The GEBCO site has some documentation for their -current- 2021 data sets but so far I haven't found anything for older data ~ 2005.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:The page at your link says that the image was created with data from "the General Bathymetric Chart of the Oceans (GEBCO) produced by the British Oceanographic Data Centre" but the NASA page doesn't give any details about which GEBCO data set(s) were used, other than mentioning that the images were created in 2005.

The GEBCO site has some documentation for their -current- 2021 data sets but so far I haven't found anything for older data ~ 2005.
So there is nothing for me to find out how to handle my land/ocean conflicts. Which, btw, are not always located at the coasts.
Post Reply