The most well-known phrase for colorfully describing a very small town. Probably comes from the 19th century, maybe from Western America of that period. The most important thing you could have owned back then was probably a horse, so if your town only had one (a community horse, I suppose, public transportation and all that) then you had either an extremely small town (or an extremely poor town).