''But an ordinary person who walks in the middle is pretty much invisible'' - I really hope this isn't true. I really don't want this to be true. But I think it might be, to a point - but maybe only online.

Online and in real life the ''extremes'' as you called them leaves strong impressions, of shock OR awe. More balanced interaction give more balanced impressions. And that's where fixed points become a thing.
IRL you have an absolute fixed point - YOU. You are always you, a blend of genes, upbringing and experiences that makes you feel your way through life. That's your fixed point that you use to balance your way, judge people, to the point where balanced people can be allowed a larger amount of impression left on you, as you choose that to happen, the ''extremes'' are more easily predicted and therefore can be (impression-wise) counteracted beforehand.
Online however, its different. Even though the YOU are present, we forgo a few things for fast comms. Human interaction for one, slow location progress for another, and lastly self preservation. Let me explain (and bear in mind I'm no expert, this is just my two cents on it);

Human interaction is so much more than the words spoken. They say about 10% of any conversational interaction between two people are the words themselves. The remaining 90% will be posture, body language, tone of voice, urgency, threat level, etc. That part is all lost when we write letters on a screen (and is also why we so often react with anger or contempt when reading comments - our threat analysis systems always tends to lean towards the ''bad'' to protect us from danger - if you are not aware (or care) of this you will answer aggressively and the fight is on).
Location progress IRL is gruesomely slow compared to online. If you leave the safe room of friends to walk down to the local biker-pub you will be acutely aware that you left a somewhat safe place to spend time to go to a place you deem unsafe. Defences will be up, alertness high. You will expect things to go bad, and therefore have mental defences up. Online, you can leave the My Little Pony Chatroom to go to the Reddit Hate Speech sub in an instant (and all from the comfort of your safe home - further keeping you feeling safe) so when things turn sour, your defences were not up, and you will immediately go into a sort of fight or flight state mentally.

Self preservation is the easy one. IRL you will simply not argue/fight with people the same way you would online for fear of consequences. Not only the 'he will smack me if I talk back'' but how the others in the room will react to you if you start up the argument by falling for one or more of the extremes in the room - you might be considered an extreme yourself for the rest of the evening. Then you also have to think on your feet, we cannot argue IRL backed up by wikipedia, or google searches, and you cannot wait a few minutes to think up a good reply - its all instant. Manners is a thing here too.

These situations vary wildly, and therefore so will your impressions of the people there. The IRL situations will mostly let you block out the ''extremes'', to the point where you don't pay too much attention to them anymore, you continue your conversation with the more balanced people around you instead. You will almost always have a sense of when someone is an ''extreme'' but is not letting it all out (yet) so you have some defences up when it happens.
Online we often concentrate on the extremes, so the more balanced people can possibly disappear to us. IRL most certainly not.