CivIV is the best late game AI I've seen as well, but it came nowhere close to holding its own in mid game, let alone late game.  At the highest difficulty, its military could be problem, but only because there is no room for smarts between two stacks of doom.

AI is nowhere near as good as a human being at complex situations, and there's no reason to believe that this will change any time soon.  There has been no revolutionary progress in AI in the last 20-30 years.  We have made plenty of evolutionary advances, due to more processing power, more memory, advances in data structures and algorithms, etc...  But AI can't challenge me in Go, so I do not expect it to challenge me at a fair game in Fallen Enchantress.

Fallen Enchantress can get a lot better, but it will not be because the AI can beat a player fairly.  It will get better when Stardock gets rid of stupid diplomatic actions, fine tunes production/rushing bonuses, and incorporates strategies for specific conditions.

But mostly, AI will get better when you have a more powerful PC, and the AI has more complex algorithms.  What I do is just make up home rules, set up enemy teams, and avoid the most powerful synergies for my side.

Right now I most enjoy a 1-2-6 distribution of sides (player/aligned/opposites) with a custom AI difficulty (5x cash, 3 x production, and 2 x research and the rest Ridiculous) on a Ridiculous large map. I stay away from beast lord, wealthy, stealth, the quest loop, elusive troops, and mass killing magic.  The result works for me.... I win most, I lose some, and the games last about 150 turns, which is enough to keep me interested.

And seriously, you should not bother going to the Victory screen.  One seldom plays chess to a mate... drop a game when it's no longer fun.

Would it be too difficult/too "random" to have the game do a check which says "if human player x% higher faction rating (yes, I now faction rating is weird, but whatever metric is best), then have y% random event happen to weaken human/strengthen AI"....stuff like famine hits, or cities/units xzy revolt and join enemy, or  AI reinforcements arrive from overseas allies, or something...so game gets harder as you start to win, rather than easier?

Would this make game more fun for those who gte bored at the late game, or more annoyingly random?

Of course, to beat the ineviatable response...set it up as an option for the player to choose!!

That's more of a rubberband style which has proven to be nice on paper but a failure in execution. It's arbitrary and not "permanent" since it's a bunch of band-aids on a gushing wound. 

I know that AI takes a lot of processing power, but can large AI factions, or "end game" AI factions delegate control? As in, split them up so each AI city or delegation can react to its particular circumstances? Seems like the AI has a one-size-fits-all brain that does not get any smarter or dumber, and thus, gets dumber as its body gets bigger.

I would be interested to hear from an experienced programmer, or a Stardock employee as to their thoughts on this inherent balance issue with 4X games, and if there's any resources put into addressing it. It is possible that they are simply satisfied with shorter games and replay value. 

Thanks for sharing your story. GalCiv2 attempted that with the Dread Lords, but they never once posed any challege. The only random event that even made me flinch (after reaching critical mass) was the jagged knife, boy do those guys suck.

All the discussion so far has been about the limitations of AI, but don't forget that this problem exists in all-human games as well.  When playing Risk, Shogun, Axis & Allies, etc. with my friends back in the day, it was invariably the case that the game started out difficult as you struggled to consolidate your power, often in the face of multiple nearby opponents, but, once one player or team reached the point of being able to effectively defend their position against all immediate threats, then the game was pretty much over unless everyone else ganged up on them.  Otherwise, they could build up for as long as they like in that defensible position before going out to roflstomp the rest of the players.

Which is not to say that I don't see a significant problem with 4X AI, but it's a more specific problem:  When the time comes to wage war, the AI simply sucks at it.  I'm fairly new to FE and just finished my first game of it last night, but it went pretty much like every 4X does in my experience.  I started out kind of muddling around (like I said, it was my first game of FE and I didn't really know the game's details) and, when I finally made contact with the sole AI player, he had about a 50% lead in faction power.  I continued exploring, building, and researching until he finally declared war on me, by which point his faction power was a little less than twice mine and he started throwing armies of juggernauts at me.  Within five turns or so, he'd lost three cities and 75% of his faction power to my single army of three heroes and some lesser troops (I didn't know about the XP division at that point).  I, on the other hand, hadn't lost so much as a single unit.  The AI later retook one of those cities when I decided to ignore him and take my hero stack off to clear wildlands instead, but that was my only loss to him.

4X AIs consistently base their diplomatic decisions on power ratings which highly overrate the quantity of military forces, while undervaluing the quality of those forces and undervaluing or completely ignoring force mobility (my hero stack is on the wrong side of the world, but I have Cloud Walk and a ton of mana, so it doesn't matter that they're not hanging out on your border) and production capacity (your obsolete troops may have me outnumbered 10-to-1 now, but I just need to stall you for a couple turns and then I'll be producing several new, superior-quality units per turn).  And then, when war does break out, they're typically too timid strategically (take one city, then sit in it for another 20 turns instead of grabbing the two undefended cities right next to it) and utterly incompetent tactically ("Oh, your high-initiative mages and archers just shot down my really-expensive-but-slow juggernaut stack without taking any losses?  I think I'll attack them with another juggernaut stack!") to have a chance against even a moderately-established human player.

I don't remember Star Trek: Birth of the Federation being that good. 

You don't have to remember it being good, as I was telling a story from my experience.

I do not think it is merely a limitation on programming, but more an issue of processing power. To have more complex AI means you need more powerful CPUs to do that. I'm quite certain that if Brad wanted to spend a few hundred man-hours to create a beast of an AI that would take a top rig computer today a few minutes to run simulations on possible outcomes of this and that and then make the best choice based on the human player's situational actions, he could. But I think that'd not be so fun having a game that takes 10 minutes between turns. As a note, all numbers and estimates are conjecture. He may be able to do it faster.. Or slower.

With current technological limitations and lack of pure processing power for situational events (and accounting for potential random occurrences) it is just not practical to create an AI capable of giving humans a real challenge in 4X, aside from bonus stats to their base. While it may be out of the question for the genre right now, you can however get plenty of challenge in chess. If you want to play the AI at it's own game. Deep Blue has nothing on what is out there now.

No you don't understand That sounded good. Did they have some sort of major patch or expansion. I only gotten the vanilla game and it was horrible. If they did put out a hugh patch or expansion I wanted to try to track it down and give it another try.

Well you're not wrong about the combination of those talents being rare, or that someone with those talents could go in so many directions.  But circumstances often dictate where people end up, or don't end up as the case may be.  It's not always about how much money is available for doing some other job.

I would actually love to design game AIs but I'm not qualified to do so.  Besides, nothing is worse for game sales than lots of customers complaining that the AI is 'too evil.'

@Tiokon, while I think you underestimate the challenge in designing and executing an AI in a game as complex and dynamic as FE, you have a general point that is worth discussing.

The person designing the AI needs to be (or have access to) a masterful player of the game.  Otherwise, no matter how keen their AI skills, they face a real struggle to design an AI that plays masterfully.  That however, is not a problem, thanks to player feedback on these forums and things like the "Let's Play" videos, Brad and co. can see top players employing the most successful strategies rather than trying to figure those strategies out for themselves.

On the other hand, since much of the game of FE is focused on handling and defeating the AI's tactics, you could say that no amount of mastery would allow you to make a masterful AI.  Once you release it to the wild and hoards of players have at your AI, strategies will emerge that are specific to defeating your AI no matter how good it is at the game.

Fortunately for everyone involved (except the family's of the programmers) is that the game's AI is not a static thing, but rather improves iteratively with each release.

Right now, I feel like the we're just emerging from the initial post-release bug fixing stage (getting rid of crashes and freezes and "the AI can't see that kind of improvement" or "the AI doesn't understand the significance of the defense stat" bugs).  We're entering the stage where they are balancing the AI so once it has the correct information it can make semi-intelligent decisions.  This would be the part where the mid-game and later AI gets a little more wooly.

I don't think we'll ever get to the point where the AI is a mind-bending challenge for Tuidjy or the other top players here.  They'll always have to handicap themselves in some way to keep themselves interested. 

The challenge really is that the complexity of the game is so great by the mid-to-end game that you're asking the AI to perform some fairly sophisticated deductive reasoning just to get a clear understanding of the world around it.

Putting aside the incredible complexity of getting a machine to understand (especially with in a limited number cycles) something.  There's the added problem of trying to get a machine to anticipate something.  Creativity and imagination are the elements that are most important to the mid and late game, when you're no longer playing against the static map, but now competing directly with dynamic opponents (at least the player is dynamic). 

If deduction is hard for an AI, creativity and imagination are essentially impossible.  And this is usually the phase where the developer substitutes his own creativity instead, (which has its own pitfalls -- like how do you describe a dynamic situation so that the AI will recognize when a specific strategy is called for).  This inevitably falls prey to the above issue that the players quickly learn the hardcoded strategies and adapt their own strategies to work against them. At best it's a losing battle for the developer. 

And it should be.  I don't expect Stardock to be at the bleeding edge of artificial intelligence research and I wouldn't expect them to produce a miraculous breakthrough in intelligence while running a background thread on my commodity desktop machine.

There are ways to tackle the problem of making the end game more interesting without programming consciousness into the AI.  Random events that throw the world into disarray is one.  But giving the AI a simple understanding of diplomacy would add to the game in boundless ways. 

(TL;DR -- here's my suggestions:)

1. Teach the AI to gang up against the top players would be nice.  Take the case where two AI factions both border a faction that is more powerful than either of them, but not as powerful as both of them.  If the two border AIs both declare war, they would be able to crush the third faction and build their own strength accordingly.

2. Teach the AI to gang up against aggressive factions.  A mid-strength empire that has gained its strength through conquest of its neighbors is not to be trusted.  Its in the more powerful factions interest to ensure that the upstart stays in its place.

To implement these ideas (and others) properly you'd need to move beyond the simple +1/-1 relation calculators and allow the AI to formulate diplomatic strategy the way it formulates military strategy and expansion.

That depends on my sovereign!

The next sovereign I play will have several goals in play when taking a city, one of those goals will involve turning certain cities into crippled, abandoned husks.  (The game mechanics have been in place to make this viable, and I need to see how well it works.)

I remember playing that a decade ago. It was prolly the first 4X game I ever played, before I knew what a 4X game was. Has it aged well?

Sweatyboatman I think we are in agreement. I may have miscommunicated my point if it sounds like I think creating a great player level AI is going to happen.  I think it would be well nigh impossible short of artifical sentience.  I agree with all that you said except that I think it is far harder for a programmer to program something they see on Utube if they don't play at that level themselves.  It's one thing to see someone use a strategy but it's another thing to know when to use it and how to bring about the same results.  If it were that easy all average players would play at extremely high levels after a few videos.  For that matter all sports teams would be champions instead of the same few usually being elite and the rest being talented but never seeming to get a clue.  This is true at all levels of competition whether money is involved or not.

"Seeing and copying strategies" CAN work to simulate a good AI to a degree in hidden games when MANY of those strategies are built into the AI and it could be using any of those strategies. Until the human opponent knows what "personality" it's facing it will keep them guessing.  Such "personalities" are still a glorified script but enough variation masks the lie.  Games with better AI's often do this.  Games with weak AI have more limited personalities, making it easy for all but sub par players to get bored with them.  Creating many personalities may take more time than developers often have but it really comes down to how strong of a game they are ok with making.  It is an area where player communities can, and sometimes do, help add a lot to a game when allowed to.

A really sophisticated AI can have stronger mid and/or late game evaluation processes to help it choose and switch to a sub personalitiy that once again can keep the human player guessing.  Current AI's (in some games more than others) do attempt to do this at a very basic level but most developers just don't have the time, ambition, and/or right set of skills to take this to the level any player slightly above average finds satisfying for very long.  As some have mentioned, a few games have gone the extra mile with AI's, just not many.

The true problem is that we are making excuses for something that shouldn't be excused.  "Dev's don't have enough time" is making an excuse.  Solo play is the industry standard.  Solo play requires AI.  The 4x genre isn't suited to multi player on a regular basis.  Perhaps some sort of organized and regulated ladder could help multi player grow but it will never be the main way 4x games are played.

A game won't have any degree of replayability beyond the strength of it's AI no matter how cool the theme or game mechanics are.  If a developer wants to make a game above "average" and become a game industry leader they need a strong AI to help them get there.  Like the teams that don't get into the championship, everything else is just excuses.  It comes down to whether the goal is to be excellent, or average.

I grew up with some guys that develop games for a living at a well known game company.  They have actually moved around to various companies since then as projects finish and as is often the case with an industry in a constant state of growing and dying at the same time.  I used to play games online with them WHILE they were at work.  When I asked if they would get in trouble not not working they replied that actually they needed to go because the company Space Empires games was about to start.  In this particular example two entire games of Space Empires were played by all company members during "work."  If you know that game you know each game can take weeks in a large multi player.  They didn't play a thrid time because a deadline was approaching in a few weeks. (NOTE: The game play wasn't "research." They weren't developing a game related to Space Empires in any way).

They've described the same "development process" at all these companies.  Months of screwing around with little effort put toward the game followed by a short period of "frantic" to make an artsy demo that has little to do with the real game but promises the world.  Then months of relaxed screwing around again with everyone saying they were waiting on someone else to get them something or other they needed, followed by a short period of frantic as final deadlines and release approach.  It used to be that a third period of screwing around would often take place when the developer convinced the publisher/backer that more time was needed to "do it right" and then a final round of frantic once most of the grace period was wasted and the final deadline truely approached.

Some would call it "the artistic process."  Definitely true that creativity can't just be clocked in when someone wants it to be but these guys say that very little effort was put to actual game making as compared to playing other games and doing just about anything other than working on the games during the screw around periods.  This process repeats over and over again and secrets get out.  It's no wonder the big publishers keep shortening the deadlines for their developers to put out a product.  Indie companies are often hungier and harder working, putting out much more creative products but often don't have the resources, manpower, management or complete set of skills needed to make the product they envision.  Brad, Derick, and company fall into the second category no doubt.   I don't think they are anything like these companies I described.  Those companies might have been large but they were barely aiming for average and rarely hit that high.  Brad and company have made some pretty darn good games that show a lot of creativity, effort, and skill.  Better than anything I saw come out of those larger companies.  If they want to get to "great"  though the weak spots would have to be shored up one way or another.  AI appears to be a weak spot that any company that takes it up a notch could make a name for themselves through.  It's not going to happen through the big companies.  Their publisher's only goals are the stock reports and their main investments now are on marketing that promises more than a presidential candidate and delivers less.  My money is still on Stardock and the Indies.