On a purely psychological level, providing emotional support to someone will mentally wear you out, as will being around people who are expressing negative emotions, as those emotions will tend to affect your own. Conversely, anyone who walks into a roomfull of happy energetic friends usually can't help but perk up a little themselves.
Concerning the metaphysical level: a fairly common belief among several religions and systems of magick is one of the existence of energy that permeates all living things. This energy can be manipulated and exchanged in a variety of ways, and it is likely that there are numerous individuals who do so without realizing it.
Energy exchange can take place in a number of situations and in varying volumes. Sometimes, when providing emotional support to people, your energy and theirs can mingle a little, and you might even give small amounts to them. This is normal and natural, especially if there is physical contact. A person under severe emotional stress may unconciously seek to take in energy because they need it to maintain psychological/emotional/spiritual stability. Generally, if they don't realize what they're doing, they will not take more than they need, and even that amount is too small to be truly dangerous to the donor. Low energy levels get dangerous when they affect one's normal functioning, just like physical or emotional fatigue. Theoretically it should be possible for someone to die as a result of having too much of their energy drained.
Psy-vamps take more energy than they need and/or take it even when they don't need it, and there is one simple reason behind this: the taste and feel of that life energy in sufficient quantities is intoxicating beyond belief. No physical form of sustinence can hold a candle to it, no orgasm can describe how incredible it feels; the closest I can come to describing the taste is to tell you to imagine drinking sunlight, nice calm soaking up of that energy... now imagine something ten times that potent, and taking it in ten times as fast, only now there is a beating heart behind it, and it's so alive and full of colors and nuances and flavors that you can't get from sunlight, flooding into you in a torrent... this is what it is to feed.
But don't stop reading, because you haven't gotten to the important part.
And once that feeding is over, once you come down off that high, the craving sets in. The lust that sits inside of you, coloring everything you see, so that everyone - even the people you love - becomes a target, a potential source of that energy. You are addicted to a powerful drug but there are no limitations on where it can be found or how much you can have, because it's all around you, free for the taking, and your victims will be left only with a touch of fatigue, or a bit of a headache, if that - because you'll only take a little, right? Just a taste?
I think you get the idea.
Briefly - I was offered the opportunity to feed for the first time by a newly-met acquaintence (who said he had large energy reserves, and thus I could not harm him); against my better judgement I accepted. A day later the offer was repeated, as was my acceptence, but this time he fed off of me at the same time. Seems like a nice solution, doesn't it? I feed off you, you feed off me, as long as it's roughly in balance, we both get to experience the ecstacy and no one's harmed for it. The problem with this is that unless your partner is available to you for feeding 24/7 (which makes both of you completely dependent on each other), you will thirst for energy when your partner is not nearby, and if the previous paragraphs haven't convinced you that this is a bad idea, then I don't know what will.
Shortly after this event I came to my senses once I realized that I was contemplating feeding off of my husband... that and some other things jerked me back to reality. At any rate, if you suspect that someone is feeding from you, get the hell away from them and stay away, unless you have the knowledge to stop them. If you can't get away from them, then at the very least don't stand close to them and don't let them touch you (touch facilitates energy exchanges of all kinds). If you are vampiric, I can only tell you that the addiction can be broken, and wish you luck.