Frogboy Frogboy

Tonight’s AI work

Tonight’s AI work

As some of you know, I’ve been working on a lot of the low level systems of Elemental during the Christmas break.  With the team on vacation, this is a good time for me to get into the bowels of the game itself to make changes without worrying about breaking other people’s changes.

Computer AI is my passion and in recent years (especially since GalCiv II) I haven’t had the pleasure to do nearly as much AI coding as I’d like (with the whole day job of running Stardock and all).  So this is how I’m spending my Christmas break.

Core weaknesses of the Elemental AI

I have a fairly endless list of criticisms with the Elemental AI so I won’t go into more than a few major ones.

The most significant one is that unlike GalCiv, Elemental isn’t able to simulate future turns.  In Galactic Civilizations II, the game would, in the background, play forward a few turns based on its best guess of what other players would do and then react to those future moves as if they were in the present. 

Unfortunately, Elemental is a much more complex game (magic and all) from an AI point of view so it’s reactive. In time, as machines get faster, I hope to eventually create a more general land-based AI that can, like GalCiv, play turns into the future in the background.  But for now, we’ll have to stick to reactive AI with future planning capabilities based on pattern recognition.

Now, before I go into the basic weaknesses here, it’s important to understand that Elemental is not a scripted AI.  That is, I don’t program in to tell it to build this or that in some sequence.  That would be easier to do to be sure but it would, in the long run, result in a game that would eventually become very repetitive.

Instead, my job is to teach the AI how to play the game and have it try its best to “win” the game based on a set of rules and conditions that it is trying to optimize.   The problem with this is that until I get really good at the game, there’s going to be glaring holes. 

In GalCiv, I was pretty good at it so I could teach that AI how to play pretty well. With Elemental, I’m still learning.  When I read on the forums what people are doing I get the distinct feeling I used to get when playing Civilization and thinking I was doing well when I built my first musketmen only to have the Mongols land with tanks. 

So here are the basic problems with the AI that I’m working on:

1. What techs are the ones it should get.  You modders out there can help if you want to.  The tech tree has an AI General section in the XML that I’ve started making more use of to tell the game what techs the AI should value more than others on researching.

2. The AI has a hard time planning ahead on resources. This is a classic issue for any game and it’s typically solved with directives.  Essentially, the AI has a hard time saving up citizens to build things like Command Posts which in turn keeps it from building squadrons of soldiers.  This is a high priority.

3. The AI is not yet good at specializing cities.  It does specialize but some APIs I don’t have easy access to yet.  For instance, the upgrade options when a city levels up have no AI data attached, they’re they might as well be random numbers.

4. General FOW issues that people tend to forget.  I miss the guard tower. The AI was much better at seeing approaching armies with that.

5. How many soldiers should be left to defend a city? How close should a protective army be left?

6. What spells should be used when? There’s no AI data yet connected to spells (that’s a high priority to v1.2) so the AI has to pick things based on general classes. It’s gotten pretty decent at it but it could be way more optimal.

7. Unit design is very challenging in general.

So after, a week and a half of working on it, here’s where it’s at:

Me vs. AI.

TURN 5

ME: Malice

image

Starting position.

In my private build of tonight I made some significant improvements to how resources are distributed. I got very lucky here that I found a lost library so close. So this should give me a significant advantage.

AI 1: Altar

image

They picked a farm to build right away.

AI 2: Umber

image

They too chose to build a farm.  They’re not hard coded to build farms, they just both decided this.

TURN 20:

ME: Malice

image

On turn 20, I’ve got my Beacon of Hope done, farm is built, I’m building a hut and I have a workshop. I’ve trained 1 defender and I have a Pioneer in training.

AI: Altar

 image

So here I have my first problem.  The AI is not constructing anything.

They’ve got a farm, a hut, a study, and a workshop. So they look pretty good.  The city is lightly defended.

AI: Umber is doing better, they’re already level 2 on their city and they’re building their second labor pit.

So let’s go look at the code…

What is Altar…thinking?

image

Altar isn’t building because it’s saving up materials to build their highest end soldier.

Now, the thing is, that’s probably not necessary this early on.  The problem is, the AI isn’t looking at threats very closely, it’s just “preparing for the worst”. A smarter strategy would be to have it scout a little bit better, find out the level of threat. If no threat, build up the economy. If only a minor threat, built a weak soldier and if the threat is significant then build up.  Right now, it’s basically prepping to get “rushed” by either people or monsters.

So I’m going to work on this…

[3 hours later]

Side note, why do saved games take a long time to load on bigger maps?

Answer: All those fancy city improvements use 95% of the load time to regenerate. Going to see what we can do about that in the future.  If you have a quad core (ahem) it’s much faster. But still, if we’re ever going to have sick sized, 64-bit maps, and we are, we’ll have to optimize this.

Now what does the AI do?

image

Better.

Let’s move forward a bit.

image

Learning spells.  I always got for alchemy. Early on, money is an issue so I tend to try to get that right away.

I do think that the amount you get should be based on your intelligence and it should get more expensive each time it is cast by someone of your faction.

Turn 50

On turn 50 let’s see how things are going.

ME: Malice

image

The bear has me a bit nervous. I have a couple of guards in the city. I’m sending out my pioneer (escorted).

AI: Altar

image

They have 3 cities right away. Impressive if they can hold them.

AI: Umber

Here’s an issue I’ve been trying to resolve, the overdefense of a city.

image

No nearby threats and they’ve got a defense of 40. Let’s see why…

It just determined that it needed those units. I could quibble, but I will keep an eye on it.

TURN ~100

image

So Umber thinks they can take me.  Let’s see if I can steam roll them still.

image

I have 3 good cities.

Umber has 4.

image

 

image

And here is when things get problematic.  First, the unit’s weapon is pretty weak.

Second, it’s building individuals.

So we have some questions here:

1. Could the AI be training better units?

2. Is this the best possible design the AI is capable of on turn 100?

3. Could the AI build the war council and thus train multiple units? If not, why not?

So let’s look at those 3 questions…

Umber’s capital..

 

To the question of whether it can build a war council, it does have the tech.

The problem: The War Council requires 35 citizens.  As players know, that’s a lot of citizens and will take several turns of doing nothing empire wide possibly to get to that many free citizens. And that’s one of the big challenges – delayed gratification.  It’s one of the things I’ll be working on that will make a big difference coming up.

If only the AI could cheat, it would be so much easier. Smile

So let’s try steam rolling them with my squads…

About to attack…

Ok, lost that battle.

They are building squads (teams?).

image

But they’re in no condition to go after me yet.

After much AI tweaking…

[5 hours of coding later]

image

Umber is closing in on me with their army but I have a pretty formidable defense.

Now, I could have taken them out earlier but I kept updating the AI. Tweaks here. New API there. Bug fix there.

I haven’t decided the best way to have the AI “save up” in order to build more powerful improvements yet when it requires several turns.  In GalCiv, the AI simulated future turns and thus could just wait. Here, I have to come up with something different. Strategically (from a military point of view) this is the AI’s most significant weakness.

As you can see in the above screenshot, the AI will build groups of units. It just doesn’t do it when it really should.

I also need to get the AI to be far more aggressive about combining up.  It has 6 armies in my territory but it would do well to combine those into 3 major army groups.

One of the big tweaks has been the AI taking a larger view of the battle field. It’s a more expensive calculation but with threading, it doesn’t’ slow the game down.

As a result, the AI is willing to send its forces much further away from home base.

image

[Another 3 hours of programming]

image

The AI is starting to get its act together on forming a more cohesive military strategy.

image

Combat rating of 155 means they’re ready to conquer.

Sadly for me…

image

Umber is fighting two wars at once as they slowly crush Altar.

So after many tweaks, the AI was victorious. But could it win without me giving it lots of second chances to get it right? Not in this game.  The question will be, can it do it better in a fresh game?

We’ll see.

 

## UPDATE ##

image

The AI has gotten better at storing mana for future…use.

177,714 views 90 replies
Reply #51 Top

Great job. The game has improved. Looking forward to the next upgrade. :grin:

Freebird in the birdcage for the moment.

Reply #52 Top

I'd just like to chime in that while the general AI is important, the risk when you start making general spell and building priorities is that the unique nature of each of the factions risks being lost. The suspension of disbelief and story-telling elements are just as important as a "winning" AI. Frankly, I think a skilled human player will be able to beat the AI, regardless of how much tweaking is done. But a good individualized AI that acts uniquely as per the unique society of its faction lets me play with the game to tell a story. Maybe I'll help Pariden bring justice to the world, or help Umber return the Titans to glory. I expect Umber to be following certain research paths and building Titan temples. I don't expect Pariden, say, to ignore magic development and not harvest its shards right away.

Ultimately, if I like Pariden, I'll help it survive. But if it doesn't act like Pariden, I'm just not motivated -- the game loses its color, its story.

Just my two cents as more of a "story" than "strategy" gamer,

-mr. b

Reply #53 Top

I'd just like to chime in that while the general AI is important, the risk when you start making general spell and building priorities is that the unique nature of each of the factions risks being lost.

Not that I disagree with your point, but you're assuming that factions have a unique nature to begin with. They don't. Hopefully this will be addressed in the future.

Reply #54 Top

I think you should just go for the look at future turns like u did wiht gal civ 2, seirously what gamer doesnt have a quad core cpu? comon now there like 60beans online nowaday. The cost of one pc game. (or 2 if u want a lil better one).

If it really is that slow of a community of cpus, maybe make a option of normal , normal (With half assed computer) ,expert , expert (with half assed cpu) .

or u could just make the people taht dont know how to upgrade a computer wait an extra second or two for the turns to end i dotn mind :D

or maybe do a Overall user system rating for all of the elemental users and see if the majority has quad cores or not. maybe slap something on elemental, or even on teh forums taht way only the players involvedd with the game more so get to be the input ? (last part may be pushin it lol)

q9550 2.83 intel core2quad

465 gtx nvidia

8gb ddr2800

Win7x64 Enterprise.

 

Reply #55 Top

Good point above me to , there really is no uniqnes to the races other then the dif between kindom and empire, all them are basically the same as far as i konw minus what there lil startup bonuses are, like 10% tech bonuses, i say we work on the ai now and then people that need a story go play teh campaign, or we work on there individuality after froggy gets a nice structured ai system up and working, weres the fun in a game were u dont have to try when the games set to ridiclous setting, i just rhomper stomper threw with a few good units (single stacked btw). Keep up the good work tho froggy def see that your dedicatedl. Need some game strat tuts, or wlakthroughs let me know im all up on addyz 8) so i dont mind.

Reply #56 Top

Interesting post, thanks. I didn't know the AI in GalCiv2 could predict the future a bit, that sounds very fancy. Good luck with teaching your AI!

Reply #57 Top

Most people realize that it's not as gratifying to play against an AI that "cheats". People want to be challenged based on the effectiveness of their strategy, not simply fight some mindless and artificially inflated horde. That takes the "I" out of "AI".

 

Anyways, great work. It's not easy spending long hours fleshing out details to something that seems simple in concept.

Reply #58 Top

Quoting dopeydave, reply 54
I think you should just go for the look at future turns like u did wiht gal civ 2, seirously what gamer doesnt have a quad core cpu? comon now there like 60beans online nowaday. The cost of one pc game. (or 2 if u want a lil better one).

If it really is that slow of a community of cpus, maybe make a option of normal , normal (With half assed computer) ,expert , expert (with half assed cpu) .

or u could just make the people taht dont know how to upgrade a computer wait an extra second or two for the turns to end i dotn mind

or maybe do a Overall user system rating for all of the elemental users and see if the majority has quad cores or not. maybe slap something on elemental, or even on teh forums taht way only the players involvedd with the game more so get to be the input ? (last part may be pushin it lol)

q9550 2.83 intel core2quad

465 gtx nvidia

8gb ddr2800

Win7x64 Enterprise.

 

 

I'm currently on a dual core laptop.  I'm waiting for solid state drives to come down in price before upgrading laptop, which should be late this year/early next year.  I have to game on laptops.

Reply #59 Top

Quoting troglyte, reply 57
Most people realize that it's not as gratifying to play against an AI that "cheats". People want to be challenged based on the effectiveness of their strategy, not simply fight some mindless and artificially inflated horde. That takes the "I" out of "AI".
I don't find it gratifying to so easily kick an AI to the curb.  In games of complexity, Artificial Intelligence is at a severe disadvantage.  At the moment, a 4x TBS AI cannot stand and fight toe to toe against a human.  I think it natural for the AI to get some handicapping.  It's not a fair fight otherwise.  It is the player who can be said to "cheat", when up against an non-handicapped AI.

I do however find it gratifying to stand victorious against even a dumb AI opponent whom has both a tech and numbers advantage over me.  If the AI is unable to be dynamic with it's tactics, and is efficient in its use of combined arms, then I want it to have a numbers, and perhaps a technological advantage.  With Elementals tactical battles, we humans can easily stomp an AI who even has a slight military advantage over us.  We can use combined arms to a lethal affect.  We can alter our tactics on the fly.  The AI can't match the power of the human mind.  So I don't mind a handicapping of the AI. 

Ideally I'd like to see an AI which needs no external help.  But at this moment in time, I don't know that a competitive 4X TBS AI is realistic.  I think there may be some hardware and software limitations which will deny us this for some time yet to come.  That is not to say that crafty coders won't be able to maximize AI potential.  Clearly there is room for innovations such as Brads multi-threading.  Creative workarounds along with the brute force scripted approach will yield some wiser AI.  But to match a human, I think 4X TBS AI is gonna need some hardware innovation first.  For now, I am cool with a handicapped AI.  The AI isn't cheating, it's simply getting a competitive handicap.   Just as with many sports, a handicap is used to makes things interesting for the both the skilled and novice players.  There is little challenge in the skilled player stomping th4e novice.  Give the novice a handicap, and the contest becomes more interesting.

Reply #60 Top

As far as AI in general, while I would like to see some personality, I also don't want it 'locked in'. If I always know that Umber is aggressive and tries a X type of strategy, that's no more interesting than if all factions did the same strategy. The only surprize is who I meet. I'd rather see randomize personalities/strategies so that if I see Umber I don't know if they're going to be aggresive or defensive or what kind of unit they're going for.


Also, one hitch with personality based AI is that they -aren't- as interesting as something which can be more flexible. Humans can be creative, true, but they can also be very predictable as well. After all, most of us probably build a hut and workshop within the first few turns. If the AI is set up to do the logical thing, it takes away predictability for it to do something outlandish...but that may be equally effective or at least, very interesting for the player to see. Similarly, if the AI is thinking too much, it may just be bland. An example is from the review of GTA4 - the reviewer noted that in MP, players would always do certain things and tend to play in certain ways. The AI on the other hand would provide really interesting situations such as chasing someone up on to a suspended I-beam and having a gunfight there. 


As for the questions...


1. On Challenging or higher, warfare 1 should be one of the first few techs (depending on research speed). Certainly on these difficulties at around 100, Warfare 1 should be being researched, just started, or just done as it is at this point that the starting equipment (assuming no faction bonuses) will significantly drop in effectiveness against mobs. This can be delayed for more risky epansion and such, but only if there isn't immediate danger such as a powerful mob or mob stack (bears with their high HP, highwaymen with their high damage, etc)


5. By turn 100, 4 single-soldier units is likely enough to provide a basic militia (it will deter the mob AI and will at least give pause to a player especially if roaming allied units are near). On cities in danger (whether currently or potentially), 6-8 is enough to deter mob AI and will be a significant obstactle. However, these numbers are not enough to be aggressive with (as in, sending out more units to support field units or perform sorties). This is also the bare minimum - while it may deter a strong stack (especially with field suport), this amount will probably fall under a strong stack (even though it will probably hurt te stack significantly). This also won't survive under a concentrated attack (two or more stacks attacking on the same turn or within a few turns). Groups-of-solders aren't needed yet at this point.


6. Heal and bursting regeneration is helpful within the first 200 hurts. For weaker champions who can't stand toe to toe, they can 'add' hp to other units and let them take on a few more units. However since heal does not scale at all, this will prove less useful later on (6 hp when you have 200 hp and the other guy is doing 20 in 1 attack is unlikely to be very useful).


7. Even a few points in armor and dodge can be very effective in the current set up. the sand golem is functionally invincible for the first 100 turns (a 6/7 creature), if you need a general gauge. Based on 'feel' it takes around a 4 point different to be effective (4 more damage than armor) or at least, seem that way.


Food is -not a high priority; it's a seond or third priority. On a decent start (2 food sources near by), one can usually hold off on building the second food source for a while. But generally, it is the mobs and production time that limit growth, not food.


Population is also -not- a high priority assuming you don't spam relentlessly. While this might not maximize other areas, it does allow for greater overall options as one can always build in different ways depending on the situation rather than, functionally, always needing huts and food to do something.


One thought would be that the AI always tries to save a little for the future; rather than using everything and then cutting back, use up almost everything and 'dip' into the savings when it's desperate

Reply #61 Top

I'll be talking about how modders can do "stuff" to the AI via the XML more in the coming year once Stardock's new Mod Czar starts.

Reply #62 Top

Quoting Frogboy, reply 61
I'll be talking about how modders can do "stuff" to the AI via the XML more in the coming year once Stardock's new Mod Czar starts.

Hmmm, ... it does not bode well, as the modding god has now been reduced the modding czar...

Reply #63 Top
The conversation about the gameplay difference between a dumb/cheating AI and a smart/noncheating one is kind of interesting, and it's the reason I liked the GalCiv 2 AI difficulty model, where the AI difficulty up to a certain point was determined entirely by the AI's intelligence, and past that point, it would play with the same level of intelligence but would also "cheat" in the form of bonuses the player didn't get. Intelligent AI isn't bad in GalCiv, but some people still need more challenge, and there ARE limits on how much challenge an AI can provide.
Reply #64 Top

Hi Santa Frog, are we getting a new EXE in a cute .gift wrap tonight or have we been naughty kids ?

Reply #65 Top

What is a mod czar exacalactly?

Reply #66 Top

Quoting Sarudak, reply 65
What is a mod czar exacalactly?
Stardock is hiring someone to take care of the modding aspects of the game.

Reply #67 Top

I'm going to buck the trend and disagree, on principle, that there is a limit to how much an AI can challenge a player without cheating.  A game operates on a set of rules and constraints (in the case of a TBS game, typically a lot of rules and constraints), which computers are exceptionally good at following.  Things just go south because we impatient players don't want to wait 20 minutes between turns for the computer to fully evaluate every facet of strategy ;)  We just need (significantly) stronger processors to see better AI.

Brad, has anyone tinkered with using GPU processing to boost the processing power of the AI (CUDA, GPGPU, etc)?  It seems to me like a fragmented "auditor/agent" style AI with hundreds or thousands of monitors evaluating a handful of inputs to provide data to a master process would be relatively well-suited to the kind of massively parallel processing that the video card brings to the table.  For ideal applications I see people get 100-300x speed increases over quad core processors (I got 100x for one of my own), so it may be worth taking a look at... if not for Elemental, for a future strategy game.

 

Edit: Of course, AI on the GPU takes cycles away from graphics... but I'd personally prefer smarter to prettier for my strategy games, and I'm sure that's most strategy players.

Reply #68 Top

Edit: Of course, AI on the GPU takes cycles away from graphics... but I'd personally prefer smarter to prettier for my strategy games, and I'm sure that's most strategy players.

The graphics arent that hot to begin with so I say go with it. I'd prefer strategy as well

@Brad: Great post! I love how you include pictures. Lookin forward to future AI matches like you did with GalCiv2  |-)

Reply #69 Top

An AI could easily beat a human if the game was designed for AIs and not for humans. If the game gave advantage to extremely complex calculations, then the computer would beat a human incapable of such. IF damage calculation hade tons of variables interacting, then an AI could predict optimal gear for a unit better than any human, but the game would be no fun.

Reply #70 Top

An example:

 

If each item of gear would slow down the soldier, but also adding armour and attack depending on the item. Items could also act differently depending on the unit wearing them (based on classes like scouts, rangers and such). This could be made insanely complex to give the AI and advantage. The problem is that the whole designing unit thing would be a black box for humans.

Reply #71 Top

Re: AI Saving up

 

In Kohan 2 - there's a value called "saving time limit" where the AI will consider it's current income rate per minute * (saving_time_limit / 60) as how much gold it could have at the end of that limit. If that result is higher than the gold it costs to perform that gold (say researching a unit tech) then the AI will not only save that gold, but ear mark it. So if there's was something else it could do now, it won't UNLESS that goal's cost would not sap too much gold.


Would it be possible to do something like that in Elemental? Something a "population saving limit" where it considers how fast it's getting citizens, how many it would need, and whether or not it can achieve that amount when by the time specified in the populating saving limit threshold?

That would seem to help with the AI not wanting to wait for future big goals like major buildings or larger war parties.

Reply #72 Top

AI population for troops issue:

I think that this could be partially fixed by making population more of a local resource - as in the city has to have enough free population in order to build the structure or train the troops. You are already tracking the local population for cities in the city size/city level bar, so why not have a %population employed? The way the game is set up right now, I can have 1 giant city with thousands of people in it and lots of tiny cities everywhere that do not have enough people locally to staff their improvements using the mega-city's population to run everything. If you shifted more towards a 'population as a local resource' model, you could have the AI decide whether or not it wants to build an improvement or train a group of soldiers in each city it has. The problem with changing to the local resource model is that individual city populations do not grow fast enough to support a 'local population resource' model with the current population costs on structures - perhaps a fix could be that population growth becomes (local prestige)*(city size)? Prestige seems to be small enough anyway unless you go out of your way to build prestige boosters in each city, and if it were seen to be a problem you could cap the number of prestige boosters or have a prestige penalty for higher level cities.

As an example, assuming a level 5 city has 900 people as its pop cap and is using (for my convenience) 15 city tiles on housing (60 people per house), if the average pop use/tile is 20 then the city structures are using only 700 of the local population, leaving 200 available for recruitment. Also, I am fairly certain that an average of 20 people used per tile is ridiculously high since most of the structures that get built are going to be prestige boosters (for an Empire, I think it is 10 pop per monument), tech structures (5 pop each for everyone), and houses (which provide pop).

Side note - if I build a garrison/barracks in a city, why don't I at least get some minor combat troops based on my warfare/conquest tech level for having such a structure in the city? The building requires some population, if I can entrust these population people to train my troops why can't I have them help defend the city? And I cannot think of any reason why I would build a garrison and then fail to garrison it unless it was in a bad location or I did not have the money - which could be in the building upkeep. This could also take some of the pressure off of the AI pop count as it might not see as great a need for troop training if its Barracks gave it something like (5 or 10)*(warfare/conquest tech level) combat rating of 'city patrolmen.'

Reply #73 Top

Whoa, whoa, whoa.  I'd just like to ask, for a moment, why on earth the AI has to "save up population"?  I have never, outside of the first five turns of the game, had to wait on population growth.  Never.  Not once.  It's a complete and total non-issue.  How on earth is the AI having trouble with this given their immense prestige boosts on higher difficulties?  I genuinely do not understand this at all.  I've always felt that population was pretty meaningless as a restriction, as currently designed, given how easy it is to come by.

 

In an attempt to answer my own mystification, I have noticed that the AI tends to avoid building housing.  It ought to be doing roughly the same calculation that I routinely make when considering what to build next.  Specifically: "my prestige in this city is X, a unit of housing takes Y turns to build, I'm considering building something else that take Z turns to finish, and I have A room in my population cap.  If X*(Y+Z) > A, then build housing first."  That avoids putting prestige to waste, which in turn gives both abundant population units and good cash flow from taxes.  Obviously critical buildings are exceptions, but there has to be a standard before there can be exceptions.

Reply #74 Top

I'll be talking about how modders can do "stuff" to the AI via the XML more in the coming year once Stardock's new Mod Czar starts.

Thanks for the heads up.

Reply #75 Top

I've had to do it frequently.

 

That bad, some population structures are rather underpowered- take the tower of sorcery for the empire.

 

It provides 4 arcane knowledge- but requires more people then 4 arcane labs.  (it's a little quicker to get up, but requires lvl 2 city)

 

Pop requirement should be reduced on that building.

 

I run into pop issues frequently, if I don't luck out on food.