"Pay Me My Money Down" has a long tradition as a Georgia Sea Island work/protest song, but it has only rarely been recorded. The first major recording of the song was by The Weavers, with Pete Seeger singing lead, at their landmark Christmas Eve 1955 concert at Carnegie Hall, and subsequently released in 1957 on the Vanguard label. A 1960 album called "Georgia Sea Island Songs" captured Sea Island Singers Joe Armstrong, Jerome Davis, John Davis, Peter Davis, Bessie Jones and Henry Morrison performing a stirring a capella version of the song. The Revels have a compelling a capella version as well. More recently, Dan Zanes recorded the song for a 2002 album "Night Time," as well as for a live DVD; also, Aloysius and Friends performed a rousing version for a 2004 benefit album for South African children affected by HIV/AIDS.

"Pay Me My Money Down" is the only song of "The Seeger Sessions" that Pete Seeger has not released on a Pete Seeger album.