i'm a stoner not a drinker

it's come to my attention, after years of trying to just "get with it", that I am not a drinker. I'm not a drinker because I don't particularly enjoy it, because I don't particularly have the same reactions as most people who like to drink.
Most people: Drink=drunk (happy/silly/whatever)=hungover
Me: Drink=hungover

I don't get the sillies/giggles/whatever, and on the rare occasion that I do, it doesn't last long and I pay for it within the next hour. And this is with a very very limited quantity.

This shouldn't really be a big deal except for 2 things.
1. The world revolves around being drunk. Or, maybe not the world, but anything social and anything fun usually involves "drinking-drinking games-solo cups-bar hopping-etc." When everyone around you is drunk and happy, it's hard not to feel awkward. You're told to drink to feel socially lubriated, to relax, to have fun, to be social. College=Drunk. Partying=Drunk. The whole world is drunk, except for me. And people find that very weird.
I kind of relate it to another thing I've always noticed:
Stand outside for 15 minutes with a cigarette and no one will look twice.
Stand outside doing nothing for 15 minutes and people think you're psycho.

2. Everyone thinks that, when I don't drink, I sit around and judge them for it. I don't. I really, honestly don't. In fact, I don't really mind going out, sipping on a coke while everyone else downs beers. I still have fun. I still like to go to bars, to socialize and have fun.

When it comes down to it, I've discovered I'm a stoner, not a drinker. Yet, a lot of people will judge me for it. The word stoner usually implies something bad- memory loss, the prevalence of "dude" in your vocabulary, apathy, an obsession with Pink Floyd.
Someone who drinks every weekend, or drinks socially, or just enjoys a cold beer every night receives none of the judgment. It doesn't seem fair.