I don't know, they just rock up every now and again and stay.  I assumed it was from raids but I don't usually have the highest bounty on me.  Sometimes i'll get a few hostile ones mixed in when they've passed through an advent system on the way to me and they've been taken over so they'll show up as a different colour.

Some screens:

This is one of my worlds

This is that world with them just hanging out doing nothing

This is what the empire bar looks like now

I was thinking today and I might try 'scuttling' one of the planets so it turns neutral and see if they'll leave.

Yeah, the owner faction is pirates and I have the DLC too.

I tried abandoning the planets and it kind of worked, as soon as the planet turned neutral they left and headed back to the pirate base.  Unfortunately as soon as they reached the base they turned around and headed straight back out going somewhere else.  The unfortunate Xon Precept world next to one of mine had nearly 1000 pirates jump into their system which was nice.

I had to abandon the game in the end before I could get any further.  Having thousands of pirate ships on the move just killed performance.  The 60 second countdown to abandon a planet was taking 6 or 7 minutes by the time I quit it.

Think it's a bug, they seem to go out on raids against me even with TAR and don't see their mission as complete until the planet is neutralised.  Shame as it's killed that game.  I think the unusual circumstance of having only 1 TEC rebel left vs 5 advent didn't help as another TEC AI would have probably nuked the base a long time before this became a problem.