I think the threat level needs to be adjusted. Two examples:

Earlier in the game, the SK gifted me a unit that could settle a city. Then several turns later when I used the unit to settle a city, the SK warned me not to expand. I can understand the SK's reaction if I used my own settler, but this is a unit he gave me. In other words, if he gives me  a unit that can settle, he should not be angry when I use it.

Also, I sent a unit out to uncover the fog of war, and after almost every turn, the SK appears and tells me to keep my units on a short leash (My words, not his). Again, I can understand this reaction if I had attacked his units or their fortresses previously, but so far I haven't.