Page 1 of 1

[blog] Porting Freeciv to the Web

Posted: Wed Jan 02, 2019 4:36 am
by louis94
Welcome!

I use this thread to post "blog" entries about work (mostly mine) done towards porting the SDL client to the Web.

Happy reading!

Bringing Freeciv to the Web -- again

Posted: Wed Jan 02, 2019 4:36 am
by louis94
Since the beginning, Freeciv was written as a network game. This means that it is really made of two programs who talk to each other: the client and the server. The client is the user-facing part of the game that displays the map and allow you to give orders. The server, on the other hand, is mostly hidden. It is responsible for calculating the outcome of player actions. The details of combat, city growth and research all depend on the server, and the clients know nothing but their outcome.

Communication between the client and the server happens over the Internet, traditionally on port 5556. If the server runs on a public machine, it can serve several players: that's how multiplayer magic happens. The Longturn community has been using this feature for a long time to run their one-turn-per-day games.

Over the past few years, AndreasR has put a lot of effort into making Freeciv playable on the Web. He put up a website, play.freeciv.org, where one could start a game in a few clicks from their browser. The website kept the original server but used a completely new client, built from the ground off. While this allowed the introduction of innovative features that would have been impossible otherwise, the Web client also had some limitations: it only supported one tileset and was limited to square topology, and not all ruleset changes were handled correctly. The website was nevertheless very successful and a community was starting to build around it. This lasted until Andreas shut the server down.

Almost one year ago, the server at play.freeciv.org was shut down without notice. The young Freeciv-Web community quickly vanished, and today only some of the most active players remain. Since all of Andreas' code was there, an attempt was made at starting a new server; however, the task proved to be hard and no one really succeeded in bringing back momentum to the project (although there are two servers, one for single-player and one for Longturn games).

More importantly, no one in the community has enough time to maintain the Web client. This means that few new feature will happen. Even keeping up with changes to the mainstream server will be hard. If we want to keep playing Freeciv in the browser, we need to do something about it.

Fortunately, the browser ecosystem has evolved a lot since the foundation of Freeciv-Web. These evolutions may make it possible to run one of Freeciv's native clients in the browser. If we manage to do that, the advantages are numerous: not only can we share a lot of code supported by core devs (less work!), but maybe Web players could connect to any game "just" like normal players do. This can only be positive for the community.

Me and a long-time LT player (zoltan) started to work on bringing the SDL client to the Web. In 2019, this is still far for being trivial, but the game is actually playable if you know how to compile. The resolution is stuck at 640x480 and crashes happen (yes, Web pages can crash), but we're hoping to fix that.

Re: [blog] Porting Freeciv to the Web

Posted: Sun Jan 13, 2019 2:05 pm
by Lachu
Do you use emterpreter? I have ported 7kaa to the web. 7kaa uses many nested loops, so porting was a real problem. In first step I do a not trivial task to rewrite a very huge part of code to use emscripten_set_main_loop and exception:
- Write main loop, which use list of special structure, containing pointer to current level of last code nested, parameter and pointer to procedure to clean current procedure
- Put each variable of highly nested code as a field of special structure and main loop uses pointer to this structure as parameter (I say that structures of main loop list contains parameter?)
- Split each function contains loop dependent on input event into initialization, loop and free parts
- When we set new loop pointer, we call throw go_to_main_loop, so main loop could call our iterator

But my work isn't real necessary, because there's emterpreter. Emterpreter makes code slower, but it helps with developing. If you like, I will help you with port.

Compiling and running on the Web

Posted: Mon Jan 14, 2019 11:07 pm
by louis94
Freeciv is written in C, which is a programming language suited for native apps. On the Web, however, only one language is allowed: Javascript. Before you even think about running a program in a browser, you need to translate it to Javascript. Doing this by hand for the roughly 50,000 lines of code that make up a Freeciv client is simply not possible. That's where Emscripten comes into play.

Emscripten is a tool that comes with a promise: one can compile C code to run in the browser. It supports many of the tools needed by Freeciv, and zoltan could actually add the only one that was missing. But the Web as a platform is vastly different from native operating systems, and what you get when you try to run the SDL client is a black, unresponsive browser tab using 100% CPU. Not nice.

The reason for this behavior, and the source of much annoyance when porting native apps, is that Javascript doesn't have a "wait" command. A Web page cannot simply ask the browser to wait; instead, it must ask the browser to call it back after some time. So instead of having a code that waits when appropriate, one needs to move back and forth between the browser and the program, asking to be called back after some time. This is a major difference from the programming point of view, and converting between the two takes some work. Fortunately, the Freeciv code happened not to rely upon wait commends too much, and it was possible to port most of it to a browser-compatible form.

Once enough wait commands were converted into their Web-compatible counterpart, I was able to start the first hex-based Freeciv game ever played in a browser:
Screenshot_20180929_064709.png
That early version was painfully slow, downloaded 300MB of data every time it was started, and used a ridiculous 4GB of RAM. All this to display a 640x480 window. There was clearly some room for improvement, but the initial success was enough to drive me away from development as I was enjoying the game :geek:

Re: [blog] Porting Freeciv to the Web

Posted: Mon Jan 14, 2019 11:27 pm
by louis94
Hi Lachu!
Lachu wrote:Do you use emterpreter?
I use WASM, since it appears to be the fastest option on Firefox and performance is still not as good as I'd like. (Don't misunderstand me: it's not slow. Just not fast enough.)
Lachu wrote:I have ported 7kaa to the web. 7kaa uses many nested loops, so porting was a real problem.
Freeciv's event loops are all handled by a single function, so replacing calls to it was easy. I implemented a linked list to push and pop context data, and it works quite well. I'm not sure that the so-called "force-exit" mode is handled correctly, but I don't think it's needed if the window cannot be closed. The networking part still deserves some attention because I had to disable all synchronous error checking code.
Lachu wrote:Emterpreter makes code slower, but it helps with developing. If you like, I will help you with port.
The Emscripten port is mostly done. Now I focus on integration within the Web platform. I can see at least two areas where I need help:
  • Currently, the SDL2 client window cannot be resized: its size is fixed at startup and never changes afterwards. While it's ~ok for a desktop game, it simply doesn't work on the Web where the user can hit F11 to enlarge the canvas. As a bonus if resizing support gets implemented, desktop users would get a resizable window too!
  • Another area that needs work is support for touch events (eg drag to move the map). As above, any touchscreen user would benefit from progress in this area.
Feel free to ask more questions if you're interested ;)
Louis

Re: [blog] Porting Freeciv to the Web

Posted: Tue Jan 15, 2019 1:04 pm
by Lachu
louis94 wrote:
  • Currently, the SDL2 client window cannot be resized: its size is fixed at startup and never changes afterwards. While it's ~ok for a desktop game, it simply doesn't work on the Web where the user can hit F11 to enlarge the canvas. As a bonus if resizing support gets implemented, desktop users would get a resizable window too!
  • Another area that needs work is support for touch events (eg drag to move the map). As above, any touchscreen user would benefit from progress in this area.
Feel free to ask more questions if you're interested ;)
Louis
Touch events are supported by SDL2 and emscripten. I used this on Android version of Bo7 (my fork of 7kaa). See https://sourceforge.net/p/battleof7/cod ... OMOUSE.cpp . Lines 740 - 760 .

Also I implement setting size of canvas, while game loading (not changing at game works/played yet). See https://sourceforge.net/p/battleof7/cod ... l.cpp#l167 . Lines 88 - 118 .