maps with different projection ?

Smallpox vs. largepox, gen2 vs gen5, early war vs. peaceful alliances. Which is your favourite gaming style?
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: The main problem I see with most land cover maps on the 'net is that they have far too many types of land cover for an easy, simple conversion to Freeciv tile types.
Well, better slightly too many types than slightly too few. I can convert but i cannot create.

Molo_Parko wrote: ^ This one has both elevation and land cover maps. This land cover map has relatively few types of land cover compared to most of the maps, and fewer would be easier to convert. When converting to Freeciv tiles, one could alternate mountain or hill tiles with whichever land cover type is indicated at a particular high-elevation spot on the map. "fmfmfm" could be derived programmatically for an area of the Freeciv map where the corresponding area of the elevation map shows mountain, and the corresponding area of the land cover map shows forest. That would be simple. I'd match all forest areas to forest tiles, shrub and mangrove to jungle tiles, bare soil to desert tiles and in all cases, intermix either hill or mountain tiles if the area indicates medium to high elevation. All three water types would match to lake tiles since the ocean is not included in the "land" cover map. The conversion is very easy to figure out with a map that doesn't have umpteen-bajillion types of land cover.
I captured such an image and tried to convert it, leading to a miserable failure as you can guess.
Despite its legend showing only a few colors, it had several thousands due to some antialias that got used to resize it before showing to me ! Made it useless due to the numerous ambiguities.
But using "raw" (full size) images lead to huge files that no software i have can handle. Even online image conversion sites failed to convert the huge tiff images i found !

Molo_Parko wrote: ^ Their idea of "grassland" strikes me more as plains (in Freeciv.)
Grassland is a plain. A plain is just flat terrain, while a grassland is a plain covered with grass. Having them in civ did not match any reality that i could observe -- ok, maybe i live in a verdant country, but i've never seen bare earth not covered with some vegetation.
I guess we have to differenciate long grass vs short grass.

Molo_Parko wrote: "Dry cropland and pasture" is what?! exactly? A desert? Or again, plains???
Irrigated/farmland desert maybe ? I'd choose between desert and plains depending on their location on the map. If it's mixed up with desert areas, then desert. Else plains.

Molo_Parko wrote: How about "Savanna"... should that become forest or plains or a mix of both tile types in that area? Or a mix of desert and forest? Or all three?
Technically it's a transition between forest and plains. But as plains can also have occasionnal trees here and there, plains are my best bet.

Molo_Parko wrote: It would be easier with fewer, and more common designations than with many somewhat odd terrain types.
If you don't know, create a new terrain type !
If it does not end up with more than around 30 types overall, for me it should remain manageable.
At worse, i'll have to find the biggest concentration of some mysterious type and look at an existing fciv map to see what's there.

Civ does not handle everything anyway, there are nonexisting types such as salt pans that are clearly not sand deserts but are (probably) converted to them.
Actually, i would prefer keeping these because the map can then be reused for non civ games - like if i make my own (something i started long ago, never finished, but who knows).

So a few tens of types should still be fine, but hundreds or thousands of them aren't.
However, even a single missing type and the whole process fails - the only exception being lakes.

As an example, keep in mind that among our 13 types there are two oceanic types.
So even the most accurate land map will still fail.
Normally continental plates extend past the coasts before we get oceanic plates.
So the former would be shallow (normal) ocean, while the latter would be deep ocean.
If this information is missing, i can't create it.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Despite its legend showing only a few colors, it had several thousands
^ That's typical / common - the color key for the image may list only the few colors which are significant as data, but depending on image format and how it was generated it will likely be from 8 bit to 24 bit color range with lots of "blends" along borders between colors. That can be fixed easily by reducing the number of colors in the image.

Image Image
^ The color key lists 5 colors which represent data. The image also uses black, and white for text (and white for arctic areas), and gray for the longitude/latitude lines. Reducing the colors in the image to 8 will eliminate all the "blended" colors where two different colors are adjacent. "Graphic Converter" is the software shown in the above two images.

Image Image
Image Image
^ Same thing, but using Photoshop and saving as GIF Restrictive with 8 colors.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote: ^ That's typical / common - the color key for the image may list only the few colors which are significant as data, but depending on image format and how it was generated it will likely be from 8 bit to 24 bit color range with lots of "blends" along borders between colors. That can be fixed easily by reducing the number of colors in the image.
That's not easy to fix, no. Some colors are somewhat ambiguous and the software will not remove them properly as they can look closer to a terrain type which is neither of those present nearby.

Molo_Parko wrote: ^ The color key lists 5 colors which represent data. The image also uses black, and white for text (and white for arctic areas), and gray for the longitude/latitude lines. Reducing the colors in the image to 8 will eliminate all the "blended" colors where two different colors are adjacent.
Nice way to destroy information. Anyway, 8 colors aren't what's needed - we need 13.
When i reduced colors it started to merge different terrains into one before all blended colors got removed.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

Only the colors listed in the color key are valid data. Aside from text labels and grid lines, other colors in the image are unintentional "shades" or blends of the key's colors. Those shades and blends are not intentional as data, but rather are introduced by the graphics file format used and by processes such as resizing the image, or generating raster images from a vector graphic. Limiting the colors in the image file doesn't remove any significant data, it merely constrains the color of a pixel to the closest match among the intended colors which represent data. In short, it is not -losing- valid information but rather it is -clarifying- the valid information within the image file.

The 8 colors in the example are because that map only uses 5 colors as data (plus white for arctic areas where they have no data), along with black, white and gray for non-data map features such as text and the longitude and latitude lines. Freeciv has more tile types available but the map data from that particular map only has 5 data colors (those shown in the color key in the first image of my last post.)

Image
^ Only the five colors in the color key are valid data. Any other shade or blend of color introduced during image manipulation or conversion is not valid data but rather an artifact of the process. Constraining the colors to those in the key will correct the unintentional extra color shades or blends back to the limited colors which represent data.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Molo_Parko wrote:Only the colors listed in the color key are valid data. Aside from text labels and grid lines, other colors in the image are unintentional "shades" or blends of the key's colors. Those shades and blends are not intentional as data, but rather are introduced by the graphics file format used and by processes such as resizing the image, or generating raster images from a vector graphic. Limiting the colors in the image file doesn't remove any significant data, it merely constrains the color of a pixel to the closest match among the intended colors which represent data. In short, it is not -losing- valid information but rather it is -clarifying- the valid information within the image file.
I'll try again with an example.
Say we have a hill and a mountain nearby. Blending these will give an intermediate hue which may be identified by the graphics software as closer to a totally irrelevant color, like a forest or a jungle, effectively creating a forest where there is none.

Molo_Parko wrote: The 8 colors in the example are because that map only uses 5 colors as data (plus white for arctic areas where they have no data), along with black, white and gray for non-data map features such as text and the longitude and latitude lines. Freeciv has more tile types available but the map data from that particular map only has 5 data colors (those shown in the color key in the first image of my last post.)
That might eventually work for a map with very few cell types. But we are not in this case.

Molo_Parko wrote: ^ Only the five colors in the color key are valid data. Any other shade or blend of color introduced during image manipulation or conversion is not valid data but rather an artifact of the process. Constraining the colors to those in the key will correct the unintentional extra color shades or blends back to the limited colors which represent data.
Constraining invalid data to valid range doesn't make that data true.
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

I agree that any blending of colors from a map's color key results in incorrect data. If the source is an elevation map, there are no forests, there is only elevation. If the map's color key represents a hill as blue, and a mountain as yellow, and a pixel between a hill and mountain is incorrectly turned to some shade of green because of the way the image was generated or manipulated, then the pixel's green color is incorrect data. By constraining the colors, the green pixel would be changed to the closest match of either yellow or blue as used in the map's color key. It would be returned to either the color for hill, or the color for mountain depending on its particular -shade- (amount of yellow and blue in that shade of green.) Restraining the colors in the image doesn't -cause- blended pixels, rather it un-blends them and returns the pixel to a matching color in the map's color key.

Image
The colors in the map color key are the only valid data in the map. The Madagascar map color key which I posted before has 25 valid colors for land cover types. No other colors in a map using that color key are valid and -should- be interpreted as actually being one of the map key colors. Reducing the color set of the image does just that. The situation doesn't change by the number of colors in the key, or the number of incorrect shades/blends of cells/areas in the image file.

Pixels with colors that don't match the color key are not showing additional valid information, as in "this area is half hill and half mountain", but rather they are simply wrong -- if the key indicates only two classifications, "hill" or "mountain", then the area/cell is either hill or mountain not both, and the color should be changed to the nearest valid match from the color key. Both Photoshop and Graphic Converter do that well and not at all randomly. An incorrectly green colored pixel that is 60% yellow, 40% blue does not become pink when constraining the colors! It becomes yellow.

This particular issue of converting from pixel colors back to a given classification is exactly why I would prefer a text response for each area/cell and its classification value. A text/numeric value of 1=hill, 2=mountain doesn't ever become a green pixel! :)

A text/numeric example: 1Km areas/cells, 2 valid classifications: 0=less than 60% forest, 1= greater than 60% forest

Code: Select all

0 0 0 1 0
0 0 1 1 0
0 .5 0 0 0
^ When you see the .5 value you instantly know that the data is corrupt because it isn't a valid value. That area is not "half less than 60% forest and half more than 60% forest" it is either not >=60% or it is. There are only two valid values. The .5 is an error or mistake, not an indication that the cell's classification is half-way between the only two possible classes.
Last edited by Molo_Parko on Wed Oct 06, 2021 7:08 pm, edited 2 times in total.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

We are not using an elevation map in civ, we have all sorts of infos. Any kind of cell can be next to any other.
So, to take an example like yours, if we have water as blue and desert as yellow (a likely story), a pixel between them is some shade of green.
When reducing the colors, it should end up with either blue or yellow - but whatever program does it will choose the nearest green instead, i.e. another terrain type. Because another green is closer to that pixel than the two others.
And now we end up with a totally incorrect cell. Whether it's grassland, plains or forest does not matter : it's wrong. Not pink, sure, but nevertheless wrong.
What's difficult to understand in that ?!
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

What you are saying makes sense if you are dealing with a photograph of the Earth and attempting to classify each area/cell for conversion to Freeciv. You'd have to use your judgement to determine the class / type of each cell.

Land-cover and elevation images/maps are composed of areas which are -already- classified. That is what such images show, the classification of each area. They do not show an actual photograph of the area but rather only the classifications assigned to the area. There is nothing in the map from which to make a judgement call about what an invalid color should be, except the color of the pixel/area, and the valid colors listed in the color key. The invalid color doesn't mean anything other than that it was supposed to be one of the valid colors. It is not what was intended but rather an error from the image manipulation process. Each area/cell class can be one of several valid values, each expressed in the image as a specific color and the map's color key defines the valid color for each classification. Any other color than those valid values used for classification is simply an error in the source. It is not that the authors intended to convey that a cell is halfway between two valid classifications. The class for that area was supposed to be one or the other, but the process which was used to create the image file mixed or blended the colors.

So, if the source map includes an invalid color, there isn't any data available in the source for a judgement call about what it should be, other than COLOR and the list of valid colors provided in the key. Whatever the invalid color is, it should be changed to the closest matching valid color from the color key, which is effectively what the image editing programs do when constraining the number of colors.

It isn't constraining the colors that makes an area's color wrong. It was already wrong and the constraint forces it to a valid value using the only clues available - the colors.

If the color key shows that the only yellow used for classification is pure yellow (RGB values R=255, G=255, B=0) is desert, and a pixel in the image is yellow-ish (R=240, G=230, B=20), then that pixel ought to be changed to the closest matching valid color.
User avatar
meynaf
Hardened
Posts: 155
Joined: Sun Jan 21, 2018 10:27 am
Location: Lyon / France
Contact:

Re: maps with different projection ?

Post by meynaf »

Obviously the pixel itself was already wrong, but that does not make it right at the end of the process -- and, worse, we lose the information that it was wrong to start with.
I took such an image, reduced it to 256 colors, and did color usage statistics. You'd expect that "main" colors are overused and "blend" pixels are scarce but you'll find out that these color keys hardly find exact matches. Actually, if you don't tell it to keep original colors unchanged, it is possible that your color reduction software will just remove them to perform some kind of averaging.

The fact that these maps are already classified does not make them usable. They just go from "totally useless" to "usable but will give wrong results".

Of course, all of this won't help to solve the most critical problem : finding an actual map to convert...
Molo_Parko
Hardened
Posts: 158
Joined: Fri Jul 02, 2021 4:00 pm

Re: maps with different projection ?

Post by Molo_Parko »

When constraining the colors, it doesn't matter how many incorrect shades of blue for example, are in an image wherein the key defines only one blue color as valid. The constraining process makes all the blue variations as near-matches the same, which is the point. You start with one "correct" blue (from the key), and many incorrect blues, then end-up with only one blue for both the key's blue and the pixels that were various shades of blue. At which point, during conversion to Freeciv map, one only need deal with a single value of blue instead of potentially thousands of values of blue. It wouldn't even matter if all the blue shades became orange as long as orange is a distinct color and all the orange pixels match the (formerly blue) orange color in the color key. Whatever color the key is (or becomes), is the color that the near-matching pixels were supposed to be. To clarify, as long as the key color for "forest" is the same color as every pixel which was supposed to be "forest" then it is easy to convert to Freeciv. The computer tends to do a better job of correcting the variance than I could do manually (in graphics editing software with a "pencil" tool ), so I let the computer do it. On the other hand, the entire problem of the variance from a single original color is caused by computers in the first place, possibly just from image compression formats (although I would have thought less variance would have been better for compression, not more variance.)

The wiki SVG maps of Winkel Tripel Earth projections are excellent quality but don't include the land cover data. If you do find acceptable land cover raster images for each land area, it is really quite easy to "warp" the raster into the SVG outline. The map projections are -all- warps of views of Earth in the first place.

If none of the source images available are acceptably accurate, then the only other option for land cover data appears to be to install mapping software, download eleventeen GigaBytes or so of land cover data, and figure out how to extract a numeric land cover classification per pixel along with the pixel's coordinates (longitude/latitude would probably be easiest to map into a Freeciv x,y map format.) Then you'd have a simple data file with unambiguous numeric land cover classification by area coordinates which can be mathematically converted to tile locations for a Freeciv map. But... the time required to download, decompress, load, and export a few simple bytes of numeric data per pixel/cell from the enormous image-based data sets might exceed any amount that could be considered "sane". :D

Then there is the other problem of color keys created by obvious lunatics, such as this key... Why create multiple categories represented by the exact same color, when they are not distinguishable by color ??? Is watermelon lettuce? This key screams "YES!" :roll:
Image
Post Reply