Caldwell was a lifelong resident of the hill country around Coffeeville, Mississippi. He spent most of his adult life working at an industrial plant in Grenada, Mississippi that manufactured heating and cooling equipment. His public performances were limited to stints at parties and local juke joints. Although Caldwell had begun playing the blues as a teenager, his repertoire remained unrecorded until 2002, when he met Fat Possum Records boss Matthew Johnson. Impressed with Caldwell's playing and personal charisma, Thompson set up recording sessions at The Money Shot in Water Valley, Mississippi. Most songs featured just Caldwell's voice and electric guitar, though a few tracks included minimal drums. Midway through the sessions, Caldwell was diagnosed with pancreatic cancer, but he doggedly continued recording. He died in September 2003 at the age of 60.
His sole album, Remember Me, was released posthumously on 24 February 2004, garnering favorable reviews and comparisons to such artists as labelmate Junior Kimbrough, John Lee Hooker, and the early Muddy Waters.