He is calling the feather macaroni. He is riding a pony. We don't know it's name.
Cut from Wikipedia, 'cause I can't say it any better: The Macaroni wig was an extreme fashion in the 1770s and became contemporary slang for foppishness. The implication of the verse was therefore probably that the Yankees were so unsophisticated that they thought simply sticking a feather in a cap would make them the height of fashion
actually he called his cap macaroni...
Stuck a feather in his cap and Called it Macaroni