Sounds like you are one of the Advent factions, yes. We'd need to know the name of your titan to be sure its the Rebels.
You can counter the Novalith Cannon by building a Starbase at the planets you're trying to use as a beachhead, then building the "Enduring Devotion" (I think that's the name) upgrade. It prevents the loss of the planet from bombardment, including Novalith cannon attacks. It will still damage the planet and kill the population, but you won't lose control, allowing you to build up factories and things that you really want for this situation. For future reference, this upgrade is usually called by the TEC equivalent upgrade, Auxillary Government (most people start with TEC factions first, so for things all three factions have, we tend to refer to them by the TEC name).
It sounds like your enemies are the TEC Rebels, since they are the only faction that can make peace with the pirates (hence you can't put bounty on them anymore, as they pirates will never attack them). If other players put bounty on that player, it was either before they researched the ability to ally with the pirates, or its from an automated source (Advent players can research an upgrade that puts bounty on players who destroy their trade ships).
Their titan can indeed do a lot of damage (that's all its good for), and against any titan it's preferable to kill it before it gets to level 10 (though your ally who died probably did that one). Best you can do is try to separate it from its defense and large fleet, then attack it with your titan, a bunch of high level capitalships (the Radiance Battleship is good for taking out titans thanks to its detonate antimatter ability. It drains antimatter and damages the titan, and no antimatter means it can't use its special abilities) and a large group of Corvettes and Bombers (the Titan's special abilities cannot damage these units, only its regular weapons can).
The AI doesn't really intentionally try to backstab you. It is possible for an alliance to stop if you let relations with an ally fall to far, but as long as you have a few envoy cruisers at their planets this should never happen (and so long as you're killing ships they're also enemies with, even that shouldn't be necessary). The only exception to this is if you didn't turn on allied victory when the game started. Without this setting only one player can win the game, so when you only have allies left they have to turn on you. I always leave this setting on.
That said, starbasing your allies then betraying them is a totally valid tactic for you, especially if you didn't turn on that allied victory thing and you know its going to happen anyways. Even if it doesn't come to that, it is not unusually for either the AI or other human allies to starbase your planets if its an important choke point. After all, two starbases are much harder to deal with than one.
While its probably good you started with the base game first, you shouldn't give up on mods for later on (full disclosure, I'm a modder). Chances are Armada 3 is dumping because you didn't apply the Large Address Aware (LAA) patch as instructed. Very large total conversion mods like STA 3 and a few others need this to ensure the game can use enough RAM, since they add so much stuff to it.
Glad you're enjoying the game and welcome to the forums.
