When Elmer Rogers Caldwell was born on 26 September 1889, in Franklin, Tennessee, United States, his father, William Alexander Caldwell, was 57 and his mother, Nancy Rowena Rogers, was 40. He married Ida Izora Cleek on 23 December 1912, in Franklin, Tennessee, United States. They were the parents of at least 1 son and 1 daughter. He lived in Civil District 15, Franklin, Tennessee, United States in 1910 and Stehekin, Chelan, Washington, United States in 1920. He died on 9 November 1974, in Estill Springs, Franklin, Tennessee, United States, at the age of 85, and was buried in Pleasant Grove Cemetery, Pleasant Grove, Franklin, Tennessee, United States.