"Um, you're welcome, I guess" is a fantastically backhanded variation on the typical
you're welcome response to a "
thank you." It suggests that the "thank you" itself was unnecessary, quite likely because the action which prompted it was so simple that there was literally no effort on the part of the thanked. But it goes a bit farther than that as well; the thanked one could simply reply, "
no problem," or "eh, it was nothing," but by choosing to enter partway into the social convention of responding to the thanker as if thanks were due, and then framing this as though, no, really, they weren't, the thanked one really comments on the thanker: "it's kind of weird, even uncomfortable, that you are thanking me for whatever you are thanking me for, and I really wish you hadn't." It might convey that the "thanks" is perceived as
ingratiating. Example: Oggler asks a hot girl if the seat next to her in the cafeteria is taken. Hot girl shrugs, because it obviously isn't. Oggler plops down: "Hey, thanks so much for letting me sit here!!" Hot girl: "Um, you're welcome, I guess." (This may be further modified by phrasing it as if it were itself a question: "Um, you're welcome? I guess?")
On the other hand, sometimes the phrase is used where no thanks were given, but ought to have been, where the in this case not-thanked one has inadvertently done something to the benefit of the not-thanking one. For example, nagging the other one to drive them to the comic book store where the one who did the driving spots and snatches up the last copy of a rare
comic book which his passenger had wanted to buy.
Driver: "Hey look, I just bought the last copy of Captain Flabbergastic #34!!"
Passenger, who was the one who'd wanted to go to the comic book store in the first place, and who surely would've bought that copy had he driven himself: "Um, you're welcome, I guess."