Less literally, yellow can mean cowardly. This is, of course, an insult. One varation on this is to call the object of your derision a Yellow-Bellied Sapsucker. Which is a kind of bird, actually. I don't think there's anything particularly cowardly about it, though.