There are such times, yes. But mostly, I learned not to let myself be bothered by things like that.
Don't worry yourself too much about it. At times, it will feel bad - sure. But having just one true friend is enough to make this feeling go away. That what happened to me when I was your age.