maps with different projection ?

Smallpox vs. largepox, gen2 vs gen5, early war vs. peaceful alliances. Which is your favourite gaming style?
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

NASA Blue Marbe ftp site ftp://public.sos.noaa.gov/oceans/2009_i ... tion/4096/ has more images showing sea ice (both north and south) throughout calendar year 2009. Northern sea ice maximum extent would be about 1/4 of the way down the list of images. Southern would be about 3/4 of the way through the images. I elected to use a less-than-maximum-extent image because huge areas of arctic tiles in Freeciv aren't much fun. I did not use the actual minimum extents for north or south though. Blue Marble also has a color tint legend/key for sea ice concentration/density as shown in the images.

Image
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Question : has Greenland very high altitude ?
I keep finding incredibly high land elevation values !
It starts to become mountain long before all normal features appear !
(f.e. at 80/255 most of European mountains are reduced to next to nothing but Greenland has most of its bulk in mountain hue)
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Yes, the glacier peaks at about 3,200 meters / 10,000 feet. The highest land (without the ice) is currently at about 2,800 meters but it might actually be higher if the glacier wasn't depressing the land.

I'm experimenting with absolute elevation value ranges for conversion to Freeciv tiles. I'm using a GEBCO grayscale image with ocean mask (value 0), and values 1-222 for (land) elevation (which works, but is not what their documentation claims they provide.) The values 1-222 represent elevation from 0 to 9,000 or 10,000 meters presumably, which works out to about 40 meters per value (assuming an equal spread across the entire value range.)

Code: Select all

	Elevation range		# Tiles, Label
	 0	to	  0		85595 Ocean
	 1	to	 12		16386 Grassland
	13	to	 29		 9328 Plains
	30	to	 89		10337 Hills
	90	to	255		 9426 Mountains
^ Those ranges result in too many hills and too many mountains (by my preference.) I'll likely reduce hills and expand plains to near equal grassland's # of tiles. The problem with reducing the value range for mountains is that in some cases, there are only one or two mountains left in an entire (short) mountain range.

Image
^ That is Absolute Elevation + Ocean Depth 2 levels + Sea Ice (both poles)
The elevation data (for land only) causes the Qattara Depression to appear as ocean, presumably because a value of 1 is above sea level and thus any land below sea level = 0 or "ocean" whether there is water there or not.

EDIT: I changed plains to range 13 to 39 (total 17341 plains tiles), hills to range 40 to 89 (total 2324 hills tiles) and I like the result. Much less hills, much more plains. I think that I will keep mountains as they are, at least until I change my mind again. :)

Image

Image
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Yes, the glacier peaks at about 3,200 meters / 10,000 feet. The highest land (without the ice) is currently at about 2,800 meters but it might actually be higher if the glacier wasn't depressing the land.
Then this will become a huge mountain, contradicting all previous civ maps which showed it as ice...
Attempting to 'fix' that will put ice in others areas (where there was none), obviously.

Molo_Parko wrote: I'm experimenting with absolute elevation value ranges for conversion to Freeciv tiles. I'm using a GEBCO grayscale image with ocean mask (value 0), and values 1-222 for (land) elevation (which works, but is not what their documentation claims they provide.) The values 1-222 represent elevation from 0 to 9,000 or 10,000 meters presumably, which works out to about 40 meters per value (assuming an equal spread across the entire value range.)
Using 40 meters as scale so a value of 20 was my 800m target did put mountains everywhere, a real flood.
A value of 70 proved to be more realistic but it's not perfect either.

Molo_Parko wrote:

Code: Select all

	Elevation range		# Tiles, Label
	 0	to	  0		85595 Ocean
	 1	to	 12		16386 Grassland
	13	to	 29		 9328 Plains
	30	to	 89		10337 Hills
	90	to	255		 9426 Mountains
^ Those ranges result in too many hills and too many mountains (by my preference.) I'll likely reduce hills and expand plains to near equal grassland's # of tiles. The problem with reducing the value range for mountains is that in some cases, there are only one or two mountains left in an entire (short) mountain range.
Mountain value of 90 makes the Pyrenees disappear completely.

Molo_Parko wrote: ^ That is Absolute Elevation + Ocean Depth 2 levels + Sea Ice (both poles)
The elevation data (for land only) causes the Qattara Depression to appear as ocean, presumably because a value of 1 is above sea level and thus any land below sea level = 0 or "ocean" whether there is water there or not.
Elevation isn't the right way to find ocean tiles ;)

Molo_Parko wrote: EDIT: I changed plains to range 13 to 39 (total 17341 plains tiles), hills to range 40 to 89 (total 2324 hills tiles) and I like the result. Much less hills, much more plains. I think that I will keep mountains as they are, at least until I change my mind again. :)
Can settlers/workers change terrain elevation without having researched Explosives ?
What is a 400m elevation block inside of a 100m area if it's not a hill ?
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Once I have selected a land cover set, I'll probably expand the value range of mountains and hills again, then replace a percentage of each of them with land cover so that mountains and hills will be more widely spread, but potentially not more numerous. The Pyrenees will reappear then. Greenland and Antarctica will become almost entirely glacier tiles.

The GEBCO elevation source GeoTiff file which I am using (gebco_08_rev_elev) has the entire ocean masked (as value 0, black.) The only non-zero values in the file are supposed to be land/elevation. I'm not too happy with that data source because even at original size (21600x10800) some land that is just above sea level is already missing (Florida's coasts as an example), while some land which actually is underwater is shown as above sea level (between Spain and Africa for instance.) It also shows inland areas which lie below sea level as ocean even when they aren't filled with water (the Qattara Depression.)

For ocean depths, I'm using a "gebco_08_rev_bath" image with white land mask. The data does not contain any elevation information, only depths. It is the opposite of the GEBCO file I've used for elevation (which contains no depth information, only elevation.) However, neither of those two files match with what GEBCO documentation says they should provide (16 bit values!)

I intend to find replacement datasets for both GEBCO images (which are from and created by NASA using GEBCO data) with other datasets which I hope will be more reliable. Meanwhile, I've automated the process of combining data for elevation, bathymetry, and sea ice (from Blue Marble series) into a Freeciv map. I'm working on rivers and lakes, unsurprisingly there are some problems with that data too. :evil:

And I've been fiddling around with ideas about relative elevation, but so far I don't like any of them and will probably just use absolute elevation with percent of tiles replaced by land cover.

Attached is a Freeciv map image showing longitudes and latitudes, centered at 0 (some images that I posted for you are centered at 11 degrees east for ocean on both sides without cutting off the eastern salient of Russia.)
Attachments
Freeciv map long lat.pdf.zip
(81.15 KiB) Downloaded 227 times
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Once I have selected a land cover set, I'll probably expand the value range of mountains and hills again, then replace a percentage of each of them with land cover so that mountains and hills will be more widely spread, but potentially not more numerous. The Pyrenees will reappear then. Greenland and Antarctica will become almost entirely glacier tiles.
That won't work. You'll get patterns and totally wrong cells mixed with right ones.

Molo_Parko wrote: The GEBCO elevation source GeoTiff file which I am using (gebco_08_rev_elev) has the entire ocean masked (as value 0, black.) The only non-zero values in the file are supposed to be land/elevation. I'm not too happy with that data source because even at original size (21600x10800) some land that is just above sea level is already missing (Florida's coasts as an example), while some land which actually is underwater is shown as above sea level (between Spain and Africa for instance.) It also shows inland areas which lie below sea level as ocean even when they aren't filled with water (the Qattara Depression.)

For ocean depths, I'm using a "gebco_08_rev_bath" image with white land mask. The data does not contain any elevation information, only depths. It is the opposite of the GEBCO file I've used for elevation (which contains no depth information, only elevation.) However, neither of those two files match with what GEBCO documentation says they should provide (16 bit values!)
That's not new. I've been handling this for quite some time now.
Land with zero elevation is normal. Treat it as land with very low altitude.
Water above sea level is normally lake.
Land with nonzero bathymetry is land under sea level (like Quattara). Same as if zero elevation.
Water with zero bathymetry is just too shallow to have a significant value but it's normal water.
Thus, water is handled.
From that, land cover directly gives grasslands, plains, forests, jungles and swamps.
That's 8/13 tile types already.
Then problems arise :
- Deserts are mismatched with tundras, regardless of the used method. Even with a temperature layer, still wrong.
- Mountains are mismatched with arctic - actually, with everything else.
- Hills are completely out if it. Absolute elevation fails miserably, relative elevation would be messed up with scale changes.

Oh, i forgot.
Applying the arctic mask, i see enormous ice mass - it's way way too big for a fciv map - touching the land and in a few areas even conflicting with it. Situation is even worse than antarctic.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

That won't work. You'll get patterns and totally wrong cells mixed with right ones.
Whether a tile displays elevation vs. land cover does not make it "wrong". It is either/or, not both. To mix elevation and land cover together will necessarily mean that some tiles will display elevation (such as hill or mountain) while others display land cover (such as forest or jungle.) There is no option to combine the two in a single tile within standard Freeciv.
Land with zero elevation is normal. Treat it as land with very low altitude.
In the source image, ocean pixels are value 0. Land with low elevation (the coasts of Florida) is -also- value 0. They are indistinguishable in the source. Both ocean and low land = 0 (and there are no depth values, so nothing sub zero to distinguish ocean from low land.) The source data method is flawed. Not all 255 values are used. It would have been simple enough to make ocean 0 and land ranging from 1 to 255 (actually 223 because the remaining values to 255 are unused in the data.) The source image, by itself, makes it impossible to determine any difference whatsoever between low coastal lands and ocean.
Water above sea level is normally lake.
The data source image does not differentiate between the values of ocean (zero) and "lake" (also zero.) There is no way to determine from the source image that a pixel with value 0 has any other elevation than sea level.
Land with nonzero bathymetry
The data source image for land/elevation is strictly elevation (land only) with a black mask (value 0) in place of ocean. Thus a 0 value anywhere in the image is indistinguishable from ocean. There is no bathymetric data in the source at all. I pull bathymetric data from a different data source image which has land masked in white (value 255.) It is not the bathymetric data source which causes the issues I mentioned regarding land, rather it is the land / elevation source itself which has those problems in the original source data (at least in the 21600x10800 NASA image which they created from GEBCO 2008 data.)

I have no problems at all with the bathymetric data source which I am using. Only the land/elevation data source has issues. However, given that both sources are based on GEBCO data which is supposed to be signed 16 bit integers, and that the land data image has obvious errors, I intend to replace both datasets as I have very little confidence in the data from the NASA/GEBCO images.
Applying the arctic mask, i see enormous ice mass - it's way way too big for a fciv map
The Arctic sea ice is roughly 15 million square kilometers at maximum extent this year. It actually is enormous. Antarctica is 14 million square kilometers (land not counting sea ice. It's even larger when sea ice is included.) https://cires.colorado.edu/news/arctic- ... xtent-2021

You could use minimum extent for Arctic ice for a smaller area, but then the area is ocean instead of ice -- either way, it's boring in Freeciv.

Image
^ Maximum and minimum Arctic sea ice extents.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: Whether a tile displays elevation vs. land cover does not make it "wrong". It is either/or, not both. To mix elevation and land cover together will necessarily mean that some tiles will display elevation (such as hill or mountain) while others display land cover (such as forest or jungle.) There is no option to combine the two in a single tile within standard Freeciv.
As you can't combine the two, one has to take precedence, and i'm afraid it shall be the mountain. Your method will show a mountain with holes in it !

Molo_Parko wrote: In the source image, ocean pixels are value 0. Land with low elevation (the coasts of Florida) is -also- value 0. They are indistinguishable in the source. Both ocean and low land = 0 (and there are no depth values, so nothing sub zero to distinguish ocean from low land.) The source data method is flawed. Not all 255 values are used. It would have been simple enough to make ocean 0 and land ranging from 1 to 255 (actually 223 because the remaining values to 255 are unused in the data.) The source image, by itself, makes it impossible to determine any difference whatsoever between low coastal lands and ocean.
This data does not need to differenciate coastal land and ocean. Not its job.
Use land cover to check if it's water or land.

Molo_Parko wrote: The data source image does not differentiate between the values of ocean (zero) and "lake" (also zero.) There is no way to determine from the source image that a pixel with value 0 has any other elevation than sea level.
Use land elevation to find lakes, not bathymetry.

Molo_Parko wrote:
Land with nonzero bathymetry
The data source image for land/elevation is strictly elevation (land only) with a black mask (value 0) in place of ocean. Thus a 0 value anywhere in the image is indistinguishable from ocean. There is no bathymetric data in the source at all. I pull bathymetric data from a different data source image which has land masked in white (value 255.) It is not the bathymetric data source which causes the issues I mentioned regarding land, rather it is the land / elevation source itself which has those problems in the original source data (at least in the 21600x10800 NASA image which they created from GEBCO 2008 data.)
Again, use land cover to find out if a cell is water. Don't attempt to make data sources give more infos than they're supposed to, it's already hard enough !

Molo_Parko wrote: I have no problems at all with the bathymetric data source which I am using. Only the land/elevation data source has issues. However, given that both sources are based on GEBCO data which is supposed to be signed 16 bit integers, and that the land data image has obvious errors, I intend to replace both datasets as I have very little confidence in the data from the NASA/GEBCO images.
I don't think you will find alternative datasets, especially ones that don't also have errors.

Molo_Parko wrote: The Arctic sea ice is roughly 15 million square kilometers at maximum extent this year. It actually is enormous. Antarctica is 14 million square kilometers (land not counting sea ice. It's even larger when sea ice is included.) https://cires.colorado.edu/news/arctic- ... xtent-2021
Enormous, but not that much. Due to flat projection, its size is grossly exagerated.
In my map south ice represents a little bit more than 10% cells of the overall map, that's 51 million square kilometers !
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Switched to newest GEBCO_Latest_2 data for absolute elevation and ocean levels. Download the 4096x2048 tiff from: https://www.gebco.net/data_and_products ... sion=1.3.0

Resized the tiff to 512x256 color PPM file using "nearest neighbor" (in Photoshop) which was the only interpolation that did not create a land bridge between Spain and Morocco (like the GEBCO 08 img from NASA already had at 21600x10800.) That's one annoying problem solved. Florida looks normal-ish now, unlike its pipe-shape in the aforementioned NASA GEBCO 08 image.

Image
^ GEBCO Latest 2 img resized to 512x256.

Next, automated a process of converting ranges of RGB values to simplified color values for conversion to Freeciv and output both the simplified image and the Freeciv map together.

Image
^ Simplified color image ready for conversion to Freeciv tile letters, or for editing/manipulation via photo/image software.

Image
^ Freeciv map as a binary PGM file using Freeciv tile letters, converted to png for posting.

Image
^ Map image saved from within Freeciv game.

Next I'll add sea ice for both poles, then back to trying to clean-up a rivers overlay file.

The color ranges and conversion values I used are here:

Code: Select all

	if [ ${r} -ge 0 -a ${r} -le 150 -a ${g} -ge 0 -a ${g} -le 127 -a ${b} -ge 0 -a ${b} -le 255 ] ; then
		r=0 g=50 b=100 # Deep ocean
		echo -n ":" >> "${freecivMap}"
	elif [ ${r} -ge 22 -a ${r} -le 217 -a ${g} -ge 109 -a ${g} -le 254 -a ${b} -ge 182 -a ${b} -le 247 ] ; then
		r=0 g=0 b=255  # Shallow ocean
		echo -n " " >> "${freecivMap}"
	elif [ ${r} -ge 100 -a ${r} -le 255 -a ${g} -ge 87 -a ${g} -le 160 -a ${b} -ge 50 -a ${b} -le 160 ] ; then
		r=255 g=255 b=255  # Highest mountains?
		echo -n "m" >> "${freecivMap}"
	elif [ ${r} -ge 145 -a ${r} -le 230 -a ${g} -ge 80 -a ${g} -le 190 -a ${b} -ge 0 -a ${b} -le 150 ] ; then
		r=230 g=230 b=230  # All mountains
		echo -n "m" >> "${freecivMap}"
	elif [ ${r} -ge 231 -a ${r} -le 245 -a ${g} -ge 170 -a ${g} -le 205 -a ${b} -ge 70 -a ${b} -le 150 ] ; then
		r=200 g=200 b=200  # Hills
		echo -n "h" >> "${freecivMap}"
	elif [ ${r} -ge 210 -a ${r} -le 255 -a ${g} -ge 200 -a ${g} -le 230 -a ${b} -ge 145 -a ${b} -le 190 ] ; then
		r=200 g=200 b=0  # Plains
		echo -n "p" >> "${freecivMap}"
	elif [ ${r} -ge 50 -a ${r} -le 220 -a ${g} -ge 200 -a ${g} -le 255 -a ${b} -ge 100 -a ${b} -le 240 ] ; then
		r=0 g=188 b=0  # Grassland
		echo -n "g" >> "${freecivMap}"
	else
		echo "UNKNOWN: x=${x},y=${y} RGB=$r $g $b"
		r=0 g=0 b=0
		echo -n "i" >> "${freecivMap}"
	fi
Last edited by Molo_Parko on Fri Nov 12, 2021 9:03 pm, edited 1 time in total.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

That doesn't help. This image contains obvious errors, like grassland in antarctica, missing tile types, etc.
Post Reply