True, however, there are ways to enter a group from the outside. Often we are judged by our looks and personalities. However! When it comes to group judging, that's a whole other universe.
Reason why everyone in most friend groups are always agreeing with one another: They're afraid of judgement. They've found a sense of inner security within this group and they'd do anything to keep it. Meaning, if the majority of the group rejects someone, the disagreeing member will simply go along with the group to maintain the group's favor.
It may not be the case that everyone rejects you, they might just be afraid of being rejected. Not that it justifies their behavior, it can and never will.