You are not an idiot. Currently the AI looks at one thing when deciding whether to attack you or not: your military power. There are some alterations with their character traits and their ideology, but military power is the main point. And even if you have a rather strong military, as long as there is some civ stronger than you the AI will buy the other civs to declare war on you.
Diplomacy is still pretty much in development and it has been acknowledged by the developers that it needs major improvement. It's sceduled for the 1.5 patch.
