I'm afraid the "no matter what I do there is some sort of social formation that I'm not a part of" is true for most people, most of the time. Or at least most somewhat introverted people. It's not really a problem and it doesn't have a solution because it's pretty much all inside each of our heads.

Not that we each have more than one head. I mean, some of us might. Oh god, never mind.