To sing a song with gusto and a certain degree of competence, though perhaps not with the style or finesse of a top-notch singer. Might be used to describe the singing of a performer known for being skilled at some other discipline, such as dance or comedy.
Presumably the phrase is related to using one's abdominal muscles to give power to the singing voice.

ex.: "Starring in the The Music Man, Tim Robbins showed that he could really belt out a tune." (fictitious)

var.:"belt one out"