I was just sitting at the mall taking a break from mall walking. A man walked over to me when I wasn’t even looking up. He said "I haven’t seen you here before. " I said I wasn’t really here to talk to people and got up and walked away. He said he would leave, he may even have said sorry. I said I was ready to leave anyway. In retrospect I think he may have just been slightly mentally challenged so I wish I had been a little softer, he did not turn out to be aggressive. Looking for suggestions on something still clear and firm but more polite? This scared me for some reason, I think how fast he walked up and sat down near me.
Editing to add a key detail, he sat down close to me which is probably what made me more nervous than I would have otherwise been. I appreciate all the thoughtful feedback I’ve gotten and interesting discussion.


Women don’t have to be polite in pursuit of their own safety.
I didn’t say we do.
I mean, telling OP she was rude and that she should’ve apologized to the guy is kinda saying we do have to be polite even if we feel uncomfortable.
I don’t think you had ill intent or anything.
OP literally asked for recommendations for being more polite. So that’s what I answered.
Edit: Also, in American English/culture, saying “sorry” isn’t necessarily really apologizing. It’s just an expected social nicety.