Is tolerance intolerant?

[tackattack]

First, I'm not claiming I'm persecuted, I do sleep well at night. It hasn't always been so in my personal life, but that's irrelevant.
Second, I'm not offended by any point made here. I'm just trying to see the other perspective as valid.
Third, It's not specific to me and I'm not making it about me. I'm trying to make it about western society as a whole and gain some perspective.

To squash the whole minority crap it's pretty simple, the smallest minority is a minority of one.
To which I ask:
1. If I'm allowed to do X and it's not illegal or harmful to anyone
2. I am doing X in a public free place
3. What right do you as an individual to come along and tell me I can't do X because you don't agree with it?
*4.Especially given that you chose to be somewhere X is allowed and don't have to be there

fill in the blanks with sex, drugs, smoking, speaking, giving hugs, etc. You are either for freedoms, or for limiting them.

I'm trying to identify the mechanism (ie intolerance, self-aggrandization, nosy nellies, etc.) by which that happens.
Also, whether there is an alternative to seeking tolerance in a society such as equality, peace, isolation that would be better for society as a whole.