In talking with a female friend a few days back we worked out that there are generally 3 types of Penis Size Anxiety. It is important to note that we worked this out in the perspective and context of heterosexual males, as two of the types involve women and all three are taken from the male perspective -- she being female and me not being gay, we felt it was unfair to make more assumptions than we already had. None of this is scientific, but merely products of meandering experience. Still, they turned out quite nice:
Anxiety with comparison to other Males
Also know as "locker room anxiety" in high school, it is the feeling of inadequacy in examination and comparison of ones size in direct visual respect with one's peers. First-hand *hem* comparison like this is very enlightening and embarrassing since there is no psychological fudge factor, simply visual constants -- you can see how big they are and they can see how small you are.
Anxiety with comparison with other Males by Females
This is the fear that you are smaller than all of you lovers' previous male companions. This is also painful because the woman in question has most likely had intimate and detailed examinations of her previous partners' members; especially in the all-important erect state.
Anxiety with physical contact with Females
Here is where "does size matter" comes in. This is the factorial where men are worried about the size of their equipment in direct relation to sex, and more importantly to their partners pleasure. An entire panel of women could take out a page in the New York Times proclaiming that "SIZE DOESN'T MATTER", and no more than 5% of men would be convinced. Believe it or not, the majority of men want to be good in bed, and they feel that they can't be "as good as they could be" unless they are the perfect size to fit their lover. It is the general (I said "general", not "correct") belief that a small penis will not "stretch" their partner enough so therefore they (their partner) won't enjoy it enough or, heaven forbid, even feel it inside of her. Length, too, plays a factor here, as many men think that the perfect penis will take up the entire length of the vagina to the point of being able to touch the cervix.
The only non-sociological reasons for these feeling we were able to come up with are two possibilities: the first) is that the longer the penis, the farther in the vagina sperm can be deposited and the wider the penis, the better it will be to keep it from leaking out. The second) is envy of the animal kingdom. Humans are, to my knowledge, the only vertebrate mammals that lack a penis bone and so therefore the human penis probably cannot achieve the extreme sizes in comparable animals in the same kingdom as a penis bone allows for much greater lengths that simply using erectile tissue cannot muster.