Page 2 of 5

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 8:50 am
by cazfi
cazfi wrote:For the record, I took out some structure sizes from freeciv trunk:

Tile: 96 bytes -> 30000 tiles take 2.9 MB
Plrtile: 72 bytes -> 30000 tiles on plrmap of 250 players take 540 MB
City: 2392 bytes -> 30000 / 20 cities on plrmap of 250 players + on 1 real map take 900.6 MB
Unit: 328 bytes -> 30000 * 1 units take 9.8 MB
Diplstate: 28 bytes -> 250 * 250 players take 1.8 MB

So, yeah, server shouldn't take more than a couple of gigabytes at worst.


Updating that to 500 players, and double the tile count to have space for all of them

Tile: 96 bytes -> 60000 tiles take 5.8 MB
Plrtile: 72 bytes -> 60000 tiles on plrmap of 500 players take 2.2 GB
City: 2392 bytes -> 60000 / 20 cities on plrmap of 500 players + on 1 real map take 3.6 GB
Unit: 328 bytes -> 60000 * 1 units take 20 MB
Diplstate: 28 bytes -> 500 * 500 players take 7 MB

So server of 500 player game would peak around 6-7 GB, or, effectively it's 4 times the memory usage of 250 players game as the significant memory usages get multiplied by the tile count and the player count.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 9:17 am
by AndreasR
The current server can handle a 500 player game then, good. I have used zram memory compression with Freeciv-web in the past, that could be a possible solution for even larger games.

Do you have any ideas about how to reduce memory usage to allow even larger games? Could the Freeciv C server compress its datastructures internally? The current 250 players use very little CPU.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 10:31 am
by cazfi
To go beyond 500 player games we will need new nation rulesets. We currently have a bit over 500 of them.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 6:44 pm
by AndreasR
Plrtile: 72 bytes -> 60000 tiles on plrmap of 500 players take 2.2 GB
City: 2392 bytes -> 60000 / 20 cities on plrmap of 500 players + on 1 real map take 3.6 GB


Do you think it would be possible to reduce the memory usage of Plrtile and/or City in a large LongTurn game?
For example by using different datatypes or removing some information from the C memory objects/structs?
3.6 GB seems like a lot to store "some" cities. Also, I'm planning on having 40k map tiles, not 60k tiles.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 7:14 pm
by Caedo
Different idea: Could the map, in principle, be split up into smaller segments, and if a player has not discovered any tiles in a given segment, all those plrtiles aren't even created in memory? I assume it would cause minor complications later on when the player gets around to discovering those areas, but since a lot of players will probably die before exploring the whole map, it might save on total RAM.
Also, for the city info, there could be an additional bit vector per city where each bit corresponds to one player and is set to 1 iff that player knows that city, and for all players that haven't discovered a given city, no space is allocated in their plrmap at all?
I know, this would make RAM usage more squiggly, but by the time we're reaching MMO levels, it might save a lot.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 7:21 pm
by cazfi
- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 7:29 pm
by cazfi
Caedo wrote:players that haven't discovered a given city, no space is allocated in their plrmap at all?

That's already the case. Plrmap has those cities player thinks there to be - it can lack cities that player has not seen, or still have cities that are already destroyed. That's why "500 players game uses 4 times as much memory as 250 players game" is more accurate for typical case than any theoretical maximum.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 7:34 pm
by AndreasR
cazfi wrote:- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city


Yes! I think both of these improvements will help a lot. I hope we can include these two things before the 500 player game is started.

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 7:51 pm
by pungtryne
AndreasR wrote:
cazfi wrote:- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city


Yes! I think both of these improvements will help a lot. I hope we can include these two things before the 500 player game is started.


For a 500-player game, could it be smart to announce it some days ahead and have the first turn be longer, say 2-3 days, as to fill up with as many human players as possible before starting?

Re: LongTurn Game 2 for Freeciv-web

Posted: Thu Jun 15, 2017 8:18 pm
by cazfi
From b2d445b4244560769d24b7ee7e8dab0031d800a0 Mon Sep 17 00:00:00 2001
From: Marko Lindqvist <cazfi74@gmail.com>
Date: Thu, 15 Jun 2017 23:03:17 +0300
Subject: [PATCH] Pack struct city with single byte alignment

Signed-off-by: Marko Lindqvist <cazfi74@gmail.com>
---
common/city.h | 2 ++
1 file changed, 2 insertions(+)

diff --git a/common/city.h b/common/city.h
index 91eb2e1b97..17b2890605 100644
--- a/common/city.h
+++ b/common/city.h
@@ -302,6 +302,7 @@ struct tile_cache; /* defined and only used within city.c */

struct adv_city; /* defined in ./server/advisors/infracache.h */

+#pragma pack(push, 1)
struct city {
char name[MAX_LEN_CITYNAME];
struct tile *tile; /* May be NULL, should check! */
@@ -449,6 +450,7 @@ struct city {
} client;
};
};
+#pragma pack(pop)

struct citystyle {
struct name_translation name;
--
2.11.0