[AI] Teaching AI instead of hard-coding all

What would you like to see in Freeciv? Do you have a good idea what should be improved or how?
Post Reply
Lachu
Elite
Posts: 472
Joined: Sat May 04, 2013 2:19 pm

[AI] Teaching AI instead of hard-coding all

Post by Lachu »

I think that best way to do AI good competitor is teach it. The problem with Freeciv there can exist many ruleset with many different game styles. Currently, ruleset maker can set flag or role to unit, which suggest AI some behavior. In my opinion, better is teach AI about correct behavior be playing humans against humans via network.

I don't think good option is to use neural network or biological algorithms - let's use probability. In ruleset ruleset maker could save some values by special program(or freeciv server could take a look at game and write values to this file). Default ruleset could not have this file, because AI is the best in default ruleset, so If ruleset doesn't contains statistics, AI won't use it.

Which statistics could be saved? For example chance to achieve some technology by player at some turn, like currency 50 turn - 50%. Instead of that we will save chance to have some amount of technologies in some turn - 10 technologies at 50 turn - 50%. We don't save values for chance at level 0%, so the file wouldn't be too big.

By reading chance to get amount or specific technology, the AI could predict turn other players will have horseback ridding technology and calculates number of turns needed to achieve technology giving pikemen unit.

The file could also contains percentage usage of some unit, while attacking on cities.

The file could also contains percentage chance of wining battle for some unit type in battle with unit of some type(only if both units starts battle with full energy).

By providing probability AI could prepare priorities list, like: discovering technology giving pikeman is possible before enemies gets horsemen only if this projects starts at turn 10, so i add this goal with priority 5, etc.
adamo
Posts: 35
Joined: Sun Nov 16, 2014 6:48 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by adamo »

(sorry for my English)

I wouldn't count on it (human-played-statistics).

Humans can make unpredictable (but yet sensible) moves. AI can't. If AI's move is classified as "unpredictable", it means it is most likely "stupid". What about real-time learning and checking data then? The problem is, this way AI can "learn" best strategies in "real time" (only during a gameplay), which would not be enough. When you turn off the computer (or finish a game), the AI, with achieved experience "dies"; and it might not be suitable for the game, played in very different conditions. Example: AI "learnt", that - in case of the current game - it's better to achieve defensive strategy and start the spaceship as quickly as possible, but that won't give AI nothing, since it already loose. If we hardcode it, it might be a blind alley next time, since the conditions have changed and, in this case, we need offensive strategy. The only thing that lefts, are the... hardcoded routines, applied to all possible factors.

Even though I don't really know hardcoded routines, I can see, they, in most cases, achieve wrong long-term strategies (according to the player steps). The best way to test different AI behaviors would be carefully tweaking AI routines and check if it won with imaginary (current AI) opponent.

IMO, it would be best to simulate AI vs AI duels. We take 6 "current" AIs and 1 AI with a particular tweak. We simulate the game and see if it win or loose. After few simulations we can evaluate if the particular change was "good" or "bad" idea. If it was bad (AI looses in more than 14% cases - because we have 7 players), we forget about that. If good (win in more than 14% cases), we include it. Then we take another tweak. Simulate it few times. Evaluate it. If "good", OK; if "bad", get out. There should be at least dozen of simulations to test the particular tweak. If our computer is fast enough, it's not the problem (I guess there is a way to speed up the gameplay to the maximum).

Of course, the best way would be a REAL human player, playing agaist tweaked AI, and comparing achieved data ("regular" playtesting), but checking AI this way (AI vs. human) takes a lot of (human!) time (I wouldn't be happy to manually test differently tweaked AIs for 100 000 hours). Testing AI vs AI is not that good from the cognitive point of view, but, in the other hand, in practice it can take no time at all; a complete game might theoretically last few seconds, if our machine is strong enough. So it can give valuable output, even if a real human wasn't involved at all in the gameplay (wasn't played real game against AI).


Example simulation 1:
Let's take 7 players: 6 AI players that behaves in "normal" way and 1 "experimental" AI player, that behaves in changed way (more agressive). We simulate 60 gameplays and see if it wins. It looses. Or wins the same as others. Why? Maybe we need to change something more? Like, concentrate on particular goals? Like building more fortresses instead of investing in more defensive units? Or building more walls? Those questions needs yet more simulations.


Things like that are already doable, since we can manipulate three factors by changing their level of: aggression, development and militarism. Maybe this is the key, if we play against a lot of AIs - to give them random factors and see what happens? We might throw differently tweaked AIs in at the deep end in different conditions and who rise on the surface.
Lachu
Elite
Posts: 472
Joined: Sat May 04, 2013 2:19 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by Lachu »

I don't agree AI should learning during normal gameplay, because:
- There can be more than one player playing for some rulesets and if player taking bad moves, AI will be stupid, because it will adjust to stupid player tactic
- Ruleset are written in directory not accessible for normal users

In my opinion, AI should learn, when server was run in special mode. You say AI must learn by playing with AI and maybe you have right.
adamo
Posts: 35
Joined: Sun Nov 16, 2014 6:48 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by adamo »

As I said, the best thing we can do is to change one AI factor and see how deals with unchanged AIs, but every little change would need dozens, if not hundreds of simulations in very different conditions ("modified" AI vs. "regular" AIs). From the cognitive point of view, it's not that good as regular playtesting with a real humans. But it might be useful, though.

But I don't know if it's even possible. We should ask the designers. Can we simulate a game, that'd take few seconds (for a whole game) in very fast CPU for 7 AI's? Can we use two different AIs (changed and unchanged), in one simulation? I guess it's technically doable, but do we have a code for that? If so, we easily compare the output data from 100 games (simulations).

Example2: Do you think AI concentrating on investing in intelligence agents (diplomats, spies) more than usual is a good idea? Does AI have to build more diplomats and spies to gather the information about others, would it help him to win? Let's check it. We change this factor (hardcoded I guess) and start simulating modified AI (more diplomats and spies) versus unchanged AIs (normal number of diplomats and spies). We simulate 100 games in different conditions and check the overall number of wins. Auch, the modified AI tends to loose! What went wrong? Maybe diplomats and spies are to careless and AI should deal with them more carefully? This is the sign, that the movement routine needs to be changed (improved). Ok, we changed it a bit. The output data is even worse! Since the improvement wasn't good, we go back to the previous factors. Instead of changing movement routines, we apply changes to the code, that AI should send a lot of diplomats one time instead of one for every few turns (this way, it's less probable that all of them are killed). It works! Etc, etc. This way (little changes in AI behavior) we can make the AI gameplay style more efficient.

If the AI changes were easy to implement, anybody could simulate that and send the data for best ideas to the game designers so they improved the hardcoded AI. But I doubt it was that easy.

................

Plus, we could randomize behavior of each AI. But it might be a bit risky.
Example: we take 7 civilizations. Every one of them has different playstyle, just like it was in Civ1. Mongols tends to act in more aggressive way, while Babylonians would rather develop cities instead of investing an army, etc. If modified factors were NOT random, for a human playing against AIs, this would be good only if the AI_cheating were enabled (it might be too easy for a player to win a game). But it randomizes a game and implements the element of uncertainty into the game!
Lachu
Elite
Posts: 472
Joined: Sat May 04, 2013 2:19 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by Lachu »

My idea was similar. You talk about giving to the AI factors like: build warrior with priority x, build musketer with priority y and test AI wins. My idea is about: There's 20 percentage chance to conquer a city, when using this unit and performing this action. Game will adjust this data during gameplay, but only if server was ran with special switch. Of course, some thing must be hardcoded, like destroying improvement giving shields will increase chance to conquer a city, but AI test how many times it was used city was conquered. Other things, like priority of science/production, etc. should be checked after gameplay.

Other idea: When Freeciv developers will realize this idea, that's look sensible to give ruleset authors possibility to define many AI's and define leaders with assigned AI's. There's also other good idea: downloading new AI's to existing ruleset from internet.
sveinung
Elite
Posts: 548
Joined: Wed Feb 20, 2013 4:50 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by sveinung »

adamo wrote:Can we simulate a game, that'd take few seconds (for a whole game) in very fast CPU for 7 AI's? Can we use two different AIs (changed and unchanged), in one simulation?
You can run an autogame. Don't expect it to finish in a few seconds.
adamo wrote:If the AI changes were easy to implement, anybody could simulate that and send the data for best ideas to the game designers so they improved the hardcoded AI.
I wrote some hints about how to modify the default AI in an introduction task. Let me know if more is needed. I may have forgotten something, used to complex wording or made a mistake.
Lachu
Elite
Posts: 472
Joined: Sat May 04, 2013 2:19 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by Lachu »

I just recall about project to add possibility to understood equations to Battle of Wesnoth's AI. Maybe there's possibility to add to AI possibility of understanding lua scripts(but only on some basic level) and tool, which will test AI will understood for each of function in lua script. Of course - Freeciv developers must build rules to understood of build-in functions, but maybe is a way to provide custom actions in ruleset. Basically - actions/functions, which contains only list of other actions should be understood able to AI, because AI must understood actions build-in to Freeciv.

Sorry for my English.
Lachu
Elite
Posts: 472
Joined: Sat May 04, 2013 2:19 pm

Re: [AI] Teaching AI instead of hard-coding all

Post by Lachu »

RE UP:
If AI could compute cost of action list, they should be able to decide to perform action or not.
hajo
Posts: 5
Joined: Wed Jul 15, 2015 7:44 pm
Location: DE

Re: [AI] Teaching AI instead of hard-coding all

Post by hajo »

Lachu wrote:I think that best way to do AI good competitor is teach it. 
When playing a game like Civ, there are a lot of informations available,
and a lot of decisions to make.
Then, how do you measure "success", and when ?

Teaching should give feedback like "this decision was good,
because X and Y", but nobody can watch that closely, and tell.

Also, a completely reasonable decision might lead to failure,
e.g. because of bad timing, "good idea, but too little/too late",
or plain bad luck.

And AIs are notoriously bad at recognising "the big picture",
e.g. if the starting location is on a small island.
Such a position requires quite some changes in strategy...
--HaJo
Post Reply