maps with different projection ?

Smallpox vs. largepox, gen2 vs gen5, early war vs. peaceful alliances. Which is your favourite gaming style?
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

I don't need a small map for testing, i need a big map.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Download links are at the bottom of the last post! :D (I just added them 3 minutes ago.)
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Download links are at the bottom of the last post! :D (I just added them 3 minutes ago.)
Thanks, let me see this.

Molo_Parko wrote: "GLOBCOVER_L4_200901_200912_V2.3_21600w.pbm.7z" is pbm (portable bit map) with binary/hex values, pbm file compressed externally with 7z. Uncompressed size is 200 MB, compressed it's 12.8 MB. :D
https://www9.zippyshare.com/v/m27MNagq/file.html
https://www.sendspace.com/file/i1k33e
I could get both. Identical files they are ;)
Counted 23 distinct values inside.
But, 21600x9300. How can i possibly merge that with the two 21600x10800 images i have for ocean depth / mountain height layers ?
Have they been cut ? Resized ? Both ? Centered differently ? Using different projection ? Missing data for poles ? A combination of all that ?
If you know why they can have different size, please tell me. Else i'll have to experiment (now i can).

Molo_Parko wrote: ^ The free file host sites have a ton of ads, pop-ups and crap. Sendspace download button is white text on blue. ZippyShare is white text on orange/red but usually it gets a redirect on first click to an ad site, try again and it downloads.
First click redirects to normal Cloudflare intermediate, that's not an ad site.
If there were ads somewhere, my ABP removed that anyway...
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

I've started to fiddle a bit with all the data i now have.

Enjoy da 4k image :
http://meynaf.free.fr/upload/tst.png

It has 31 distinct colors :
- 22 original ones (23 minus water)
- 8 levels for water (not needed for civ but it's nice)
- a special color for where bathymetric data collided with land (few places but it happened)

Land elevation is still missing. I need to think before any attempt.
Colors aren't necessarily well chosen, but, hey, i don't know how to interpret most of them.

I also need to redo the projection. No idea for the formulas to use.
I don't know what they used here. North part looks squashed.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Beautiful image! Well done!

The image file from Globcover2009_V2.3_Global data set has no south pole data at all. It's not in the data set, which is why the bottom of the image seems to be cut short. I resized the giant tiff, constraining proportions, down to the 21600 pixel width you specified. The pic is all the image there is in that data set. Since it is a "land cover" set, I can sort-of see why they didn't bother including Antarctica. On the other hand, they also didn't include sea ice in the Arctic. https://nsidc.org/arcticseaicenews/2009/03/

Perhaps the polar land / sea ice regions are included in some other type of data set, just not in "land cover" data. Probably not in "land use" data, nor "vegetation" data either. The GlobCover image is misleading because it only shows ice on land, as far as I can ascertain.

Are the rivers in the image distinct enough for Freeciv conversion? I can see some of them clearly, but the central North American rivers have a lot of mixed terrain colors that makes it hard to follow them on the image.

The red pixels in the map appear to be be dense city areas. Presumably, on conversion to Freeciv map, those would be changed to some terrain type... but what terrain type(s)? Perhaps we might avg the surrounding non-red terrain and use whichever is most common. For instance

g g g
g r g
g p +

^ Since grassland predominates around the red / city tile, change the red to grassland on conversion to Freeciv map ???


The equations for changing projection to Winkel Tripel are listed on Wikipedia and better explained in a PDF shown and linked below.

Image
https://www.researchgate.net/publicatio ... Projection

It also appears that various mapping software can do the conversion from one projection to another. Take a look at this one from NASA: https://www.giss.nasa.gov/tools/gprojector/

The "GlobCover 2009 Read me.pdf" includes the following: "The map is in geographic coordinates in a Plate-Carrée projection "
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: Beautiful image! Well done!
Yeah, it looks quite good :)
For now it is purely generated from data, it hasn't been hand edited (aside of color picking). Resizing was just pixel skipping...

Molo_Parko wrote: The image file from Globcover2009_V2.3_Global data set has no south pole data at all. It's not in the data set, which is why the bottom of the image seems to be cut short. I resized the giant tiff, constraining proportions, down to the 21600 pixel width you specified. The pic is all the image there is in that data set. Since it is a "land cover" set, I can sort-of see why they didn't bother including Antarctica. On the other hand, they also didn't include sea ice in the Arctic. https://nsidc.org/arcticseaicenews/2009/03/
I feared the images would be incompatible due to a different size, but it was just truncated at the bottom.

Molo_Parko wrote: Perhaps the polar land / sea ice regions are included in some other type of data set, just not in "land cover" data. Probably not in "land use" data, nor "vegetation" data either. The GlobCover image is misleading because it only shows ice on land, as far as I can ascertain.
North pole is indeed missing. It is not land, after all.
But south pole is land, so for me there is no reason it shouldn't be here. Hopefully i could recreate it from ocean depth data.

Molo_Parko wrote: Are the rivers in the image distinct enough for Freeciv conversion? I can see some of them clearly, but the central North American rivers have a lot of mixed terrain colors that makes it hard to follow them on the image.
As my image is a resized one (5:1) maybe rivers are big enough for some conversion, but maybe not.
There is also the problem of special resources...

Molo_Parko wrote: The red pixels in the map appear to be be dense city areas. Presumably, on conversion to Freeciv map, those would be changed to some terrain type... but what terrain type(s)? Perhaps we might avg the surrounding non-red terrain and use whichever is most common. For instance

g g g
g r g
g p +

^ Since grassland predominates around the red / city tile, change the red to grassland on conversion to Freeciv map ???
Red pixels in my map are "190 - artificial surfaces and associated areas".
So, basically, cities. They should be flat terrain with nothing special, not too wet, not too dry.
I think i'll convert them to plains, as plains are a good type for founding cities.

More problematic for me are 11 to 180 in the list.

Molo_Parko wrote: The equations for changing projection to Winkel Tripel are listed on Wikipedia and better explained in a PDF shown and linked below.

https://www.researchgate.net/publicatio ... Projection

It also appears that various mapping software can do the conversion from one projection to another. Take a look at this one from NASA: https://www.giss.nasa.gov/tools/gprojector/

The "GlobCover 2009 Read me.pdf" includes the following: "The map is in geographic coordinates in a Plate-Carrée projection "
Thanks for information. I don't know if i will be able to perform the conversion, though.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

^ Nasa's GProjector software is free. Requires Java 9 or later and can transform projections into a long list of different projections all by itself!

List of supported projections to convert from / to: https://www.giss.nasa.gov/tools/gprojec ... ojections/

On the land cover types...

11 sounds like grassland that indicated surface moisture - after rainfall, flood, or irrigation. Could convert that to just grassland, or to -irrigated- grassland, I suppose.

14 sounds very similar to 11 except that the cause of the ground moisture is certain - rain. That might be a good indication that there should not be any Freeciv "plains" tiles in the area but rather all nearby flat land would be either grassland or forest/jungle due to adequate rainfall.

20 is majority cropland (cultivated and probably irrigated) so grassland again.

30 sounds like uncultivated cropland - grassland area but with lots of brush/scrub vegetation growing wild. Pasture as opposed to farm land so probably plains.

40 is too ambiguous. >15% forest but no explanation what the rest is. That might include "savannah" for instance which seems like sparse trees/jungle on desert. Might also include some orchards with large distance between trees, and in which case the rest is probably grassland. This might be a case where which Freeciv tile is used could depend on location. If Africa, desert tile may be best match. If northern area, probably plains. If South America, probably grassland or jungle... or something like that.

50 = > 40% forest so "f" in Freeciv although > 50 would be a better measure for that perhaps.

60 = 15 to 40% forest which makes it majority not forest, so probably plains.

70 >40% forest so probably forest.

90 = Same as 60, just different type of trees in the minority forest.

110 = forest

120 = grassland

130 & 140 = > 15% too ambiguous again. 130=jungle? 140=plains?

150 = desert or tundra depending on proximity to poles.

160, 170, 180 = swamp

For some areas a mix of tiles might be a better match. If the map has a 3x3 cell area of all type 180 = "regularly flooded vegetation", then a mix of swamp, grassland and jungle might be a better representation of the area than just swamp, just grassland, or just jungle. Most Freeciv maps aren't really big enough to support mixing tiles to indicate terrain but your map is going to be huge in Freeciv, so it might work well.

I found this: https://neo.sci.gsfc.nasa.gov/view.php? ... MOD_NDVI_M
Which offers both image files -or- text files of data. The data files have intensity values of "leaf index" which show relative vegetation cover but they don't classify what type of vegetation. The only use for Freeciv conversion that I can think of, for that data, would be to differentiate areas of desert, plains, or grassland/forest/jungle all together. Low vegetation = desert or tundra, medium = plains, high=grassland/forest/jungle. But we already have that info (mostly) from the GlobCover maps land cover classifications.

This link looks interesting: https://data.apps.fao.org/map/catalog/s ... 559c4bff38
It has a download of "dominant land cover" with the same sort of grayscale GeoTiff image wherein the grayscale pixel byte values are the classification # from the list included. I -assume- that "dominant" means >50% which would likely be an easier conversion to Freeciv tiles because it is unambiguous about whether a tile is mostly/majority forest or not, for example. The tiff file is 37 MB and 43200 × 21600 pixels, so not too huge for processing. They also have individual data sets for "bare soil", "snow and glaciers", "water bodies" and so on.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: 11 sounds like grassland that indicated surface moisture - after rainfall, flood, or irrigation. Could convert that to just grassland, or to -irrigated- grassland, I suppose.
Nile delta and northern India contain a lot of these. Swamps are out of question then, so probably grassland indeed.

Molo_Parko wrote: 14 sounds very similar to 11 except that the cause of the ground moisture is certain - rain. That might be a good indication that there should not be any Freeciv "plains" tiles in the area but rather all nearby flat land would be either grassland or forest/jungle due to adequate rainfall.
Sounds like temperate grassland. Locations on map indicate it can't be jungle.

Molo_Parko wrote: 20 is majority cropland (cultivated and probably irrigated) so grassland again.
That's starting to make lots of grassland. Some countries will be mostly grassland then.

Molo_Parko wrote: 30 sounds like uncultivated cropland - grassland area but with lots of brush/scrub vegetation growing wild. Pasture as opposed to farm land so probably plains.
Dunno, but we clearly needed more plains to oppose to grassland...

Molo_Parko wrote: 40 is too ambiguous. >15% forest but no explanation what the rest is. That might include "savannah" for instance which seems like sparse trees/jungle on desert. Might also include some orchards with large distance between trees, and in which case the rest is probably grassland. This might be a case where which Freeciv tile is used could depend on location. If Africa, desert tile may be best match. If northern area, probably plains. If South America, probably grassland or jungle... or something like that.
Locations on map strongly suggest jungle.

Molo_Parko wrote: 50 = > 40% forest so "f" in Freeciv although > 50 would be a better measure for that perhaps.
Only thing i can tell : northeast USA consists mostly of that.

Molo_Parko wrote: 60 = 15 to 40% forest which makes it majority not forest, so probably plains.
Lots of these can be found in sub-saharian Africa, so plains work.

Molo_Parko wrote: 70 >40% forest so probably forest.
Land type scattered in US, China, Europe. I can't tell.

Molo_Parko wrote: 90 = Same as 60, just different type of trees in the minority forest.
Huge forest in Russia is of that type.

Molo_Parko wrote: 110 = forest

120 = grassland
Locations on map suggest plains instead. Unless Australia is more verdant than i think.

Molo_Parko wrote: 130 & 140 = > 15% too ambiguous again. 130=jungle? 140=plains?
130 : north Australia, west USA, Asia. Several places suggest mountains.
140 : near deserts, center USA, Ireland. Maybe plains.

Molo_Parko wrote: 150 = desert or tundra depending on proximity to poles.
Too ambiguous, this type is present at too many places, esp. temperate ones, to be only that.
Adding plains at the center might be fine, but filtering with y coordinate only will lead to stripes.

Molo_Parko wrote: 160, 170, 180 = swamp
Rare land types, scattered here and there. I can't tell anything but i needed candidates for swamps.

Molo_Parko wrote: For some areas a mix of tiles might be a better match. If the map has a 3x3 cell area of all type 180 = "regularly flooded vegetation", then a mix of swamp, grassland and jungle might be a better representation of the area than just swamp, just grassland, or just jungle. Most Freeciv maps aren't really big enough to support mixing tiles to indicate terrain but your map is going to be huge in Freeciv, so it might work well.
Source maps i have are a lot bigger than a freeciv map would be. I don't currently do any averaging, but the map i've shown you was already showing 5x5 areas reduced to single pixels.
Adding a bit of pseudo random to some parts wouldn't be a big deal technically, though.

Molo_Parko wrote: I found this: https://neo.sci.gsfc.nasa.gov/view.php? ... MOD_NDVI_M
Which offers both image files -or- text files of data. The data files have intensity values of "leaf index" which show relative vegetation cover but they don't classify what type of vegetation. The only use for Freeciv conversion that I can think of, for that data, would be to differentiate areas of desert, plains, or grassland/forest/jungle all together. Low vegetation = desert or tundra, medium = plains, high=grassland/forest/jungle. But we already have that info (mostly) from the GlobCover maps land cover classifications.
Might be useful to solve ambiguous cases. Perhaps, maybe. :)
There is also a temperature map on the site. Probably more accurate than y coordinate on map.

Molo_Parko wrote: This link looks interesting: https://data.apps.fao.org/map/catalog/s ... 559c4bff38
It has a download of "dominant land cover" with the same sort of grayscale GeoTiff image wherein the grayscale pixel byte values are the classification # from the list included. I -assume- that "dominant" means >50% which would likely be an easier conversion to Freeciv tiles because it is unambiguous about whether a tile is mostly/majority forest or not, for example. The tiff file is 37 MB and 43200 × 21600 pixels, so not too huge for processing. They also have individual data sets for "bare soil", "snow and glaciers", "water bodies" and so on.
Could be interesting to experiment with, but i can't handle GeoTiff files at all (and i'm always reluctant to install extra software ;) ).
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

On the issue of Plains (tiles) vs grassland, it appears that in the U.S., at least, the differentiation of plains and grassland is a matter of elevation.

Image

Image
^ Elevation side-view of Interstate highway 70 across the state of Kansas appears to indicate that elevation below 1,000 feet is grassland or "lowlands" while land above 1,000 feet elevation is plains. If that distinction is used globally, then flat portions of some countries being almost entirely grassland wouldn't be too surprising.

Within Freeciv, grassland to forest takes 10 turns, plains to forest takes 15 turns. Areas of mostly grassland might result in both faster city growth and also comparatively fast change to forest for higher productivity. Which would likely accelerate the development of tribes in mostly grassland areas (assuming that the "A.I." controlled tribes bother to mine any tiles at all to balance growth and productivity.) Looking at the U.S. map linked above, that seems to be about right, historically speaking. The "lowland" areas of the map (at least those which aren't swamps / below sea level, like New Orleans) have had both greater growth and greater productivity than have the plains areas which were predominantly farms and ranches. The majority of large cities / centers of industrial activity haven't been located in the plains or mountain areas.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: Within Freeciv, grassland to forest takes 10 turns, plains to forest takes 15 turns. Areas of mostly grassland might result in both faster city growth and also comparatively fast change to forest for higher productivity. Which would likely accelerate the development of tribes in mostly grassland areas (assuming that the "A.I." controlled tribes bother to mine any tiles at all to balance growth and productivity.) Looking at the U.S. map linked above, that seems to be about right, historically speaking. The "lowland" areas of the map (at least those which aren't swamps / below sea level, like New Orleans) have had both greater growth and greater productivity than have the plains areas which were predominantly farms and ranches. The majority of large cities / centers of industrial activity haven't been located in the plains or mountain areas.
In freeciv they haven't both growth and productivity. Plain has 1 production unit where grassland has none.
Besides, turn your grassland into a forest, then remove said forest : now you've changed land elevation without any special tech...
So land elevation doesn't work for me.
Post Reply