It really depends on how they said it and the context of the situation. It can be taken either positively or negatively and that is up to you. To me uniqueness is good thing. It means you aren't afraid to be who you are.
So are you?
It means, "You aren't like me and it bothers me....."
It comes from people who are stupid and can't figure you out.
You are strange.