LongTurn Game 2 for Freeciv-web

Planning and discussing Freeciv Longturn gaming
cazfi
Elite
Posts: 3077
Joined: Tue Jan 29, 2013 6:54 pm

Re: LongTurn Game 2 for Freeciv-web

Post by cazfi »

cazfi wrote:For the record, I took out some structure sizes from freeciv trunk:

Tile: 96 bytes -> 30000 tiles take 2.9 MB
Plrtile: 72 bytes -> 30000 tiles on plrmap of 250 players take 540 MB
City: 2392 bytes -> 30000 / 20 cities on plrmap of 250 players + on 1 real map take 900.6 MB
Unit: 328 bytes -> 30000 * 1 units take 9.8 MB
Diplstate: 28 bytes -> 250 * 250 players take 1.8 MB

So, yeah, server shouldn't take more than a couple of gigabytes at worst.
Updating that to 500 players, and double the tile count to have space for all of them

Tile: 96 bytes -> 60000 tiles take 5.8 MB
Plrtile: 72 bytes -> 60000 tiles on plrmap of 500 players take 2.2 GB
City: 2392 bytes -> 60000 / 20 cities on plrmap of 500 players + on 1 real map take 3.6 GB
Unit: 328 bytes -> 60000 * 1 units take 20 MB
Diplstate: 28 bytes -> 500 * 500 players take 7 MB

So server of 500 player game would peak around 6-7 GB, or, effectively it's 4 times the memory usage of 250 players game as the significant memory usages get multiplied by the tile count and the player count.
AndreasR
Elite
Posts: 754
Joined: Thu May 02, 2013 10:26 pm

Re: LongTurn Game 2 for Freeciv-web

Post by AndreasR »

The current server can handle a 500 player game then, good. I have used zram memory compression with Freeciv-web in the past, that could be a possible solution for even larger games.

Do you have any ideas about how to reduce memory usage to allow even larger games? Could the Freeciv C server compress its datastructures internally? The current 250 players use very little CPU.
cazfi
Elite
Posts: 3077
Joined: Tue Jan 29, 2013 6:54 pm

Re: LongTurn Game 2 for Freeciv-web

Post by cazfi »

To go beyond 500 player games we will need new nation rulesets. We currently have a bit over 500 of them.
AndreasR
Elite
Posts: 754
Joined: Thu May 02, 2013 10:26 pm

Re: LongTurn Game 2 for Freeciv-web

Post by AndreasR »

Plrtile: 72 bytes -> 60000 tiles on plrmap of 500 players take 2.2 GB
City: 2392 bytes -> 60000 / 20 cities on plrmap of 500 players + on 1 real map take 3.6 GB
Do you think it would be possible to reduce the memory usage of Plrtile and/or City in a large LongTurn game?
For example by using different datatypes or removing some information from the C memory objects/structs?
3.6 GB seems like a lot to store "some" cities. Also, I'm planning on having 40k map tiles, not 60k tiles.
User avatar
Alien Valkyrie
Elite
Posts: 513
Joined: Sun Feb 10, 2013 10:21 pm
Location: Stuttgart, Germany

Re: LongTurn Game 2 for Freeciv-web

Post by Alien Valkyrie »

Different idea: Could the map, in principle, be split up into smaller segments, and if a player has not discovered any tiles in a given segment, all those plrtiles aren't even created in memory? I assume it would cause minor complications later on when the player gets around to discovering those areas, but since a lot of players will probably die before exploring the whole map, it might save on total RAM.
Also, for the city info, there could be an additional bit vector per city where each bit corresponds to one player and is set to 1 iff that player knows that city, and for all players that haven't discovered a given city, no space is allocated in their plrmap at all?
I know, this would make RAM usage more squiggly, but by the time we're reaching MMO levels, it might save a lot.
~ AVL
cazfi
Elite
Posts: 3077
Joined: Tue Jan 29, 2013 6:54 pm

Re: LongTurn Game 2 for Freeciv-web

Post by cazfi »

- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city
cazfi
Elite
Posts: 3077
Joined: Tue Jan 29, 2013 6:54 pm

Re: LongTurn Game 2 for Freeciv-web

Post by cazfi »

Caedo wrote:players that haven't discovered a given city, no space is allocated in their plrmap at all?
That's already the case. Plrmap has those cities player thinks there to be - it can lack cities that player has not seen, or still have cities that are already destroyed. That's why "500 players game uses 4 times as much memory as 250 players game" is more accurate for typical case than any theoretical maximum.
AndreasR
Elite
Posts: 754
Joined: Thu May 02, 2013 10:26 pm

Re: LongTurn Game 2 for Freeciv-web

Post by AndreasR »

cazfi wrote:- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city
Yes! I think both of these improvements will help a lot. I hope we can include these two things before the 500 player game is started.
pungtryne
Veteran
Posts: 72
Joined: Sat Jun 03, 2017 5:27 pm

Re: LongTurn Game 2 for Freeciv-web

Post by pungtryne »

AndreasR wrote:
cazfi wrote:- If you are ready to take the performance hit, #pragma pack(1) seems to result in 2360 bytes struct city
- I have plans to make city name storage size dynamic, instead of reserving enough space for the longest city name on every struct city
Yes! I think both of these improvements will help a lot. I hope we can include these two things before the 500 player game is started.
For a 500-player game, could it be smart to announce it some days ahead and have the first turn be longer, say 2-3 days, as to fill up with as many human players as possible before starting?
cazfi
Elite
Posts: 3077
Joined: Tue Jan 29, 2013 6:54 pm

Re: LongTurn Game 2 for Freeciv-web

Post by cazfi »

From b2d445b4244560769d24b7ee7e8dab0031d800a0 Mon Sep 17 00:00:00 2001
From: Marko Lindqvist <cazfi74@gmail.com>
Date: Thu, 15 Jun 2017 23:03:17 +0300
Subject: [PATCH] Pack struct city with single byte alignment

Signed-off-by: Marko Lindqvist <cazfi74@gmail.com>
---
common/city.h | 2 ++
1 file changed, 2 insertions(+)

diff --git a/common/city.h b/common/city.h
index 91eb2e1b97..17b2890605 100644
--- a/common/city.h
+++ b/common/city.h
@@ -302,6 +302,7 @@ struct tile_cache; /* defined and only used within city.c */

struct adv_city; /* defined in ./server/advisors/infracache.h */

+#pragma pack(push, 1)
struct city {
char name[MAX_LEN_CITYNAME];
struct tile *tile; /* May be NULL, should check! */
@@ -449,6 +450,7 @@ struct city {
} client;
};
};
+#pragma pack(pop)

struct citystyle {
struct name_translation name;
--
2.11.0
Post Reply