When Elisha Anderson Allen was born in 1802, in Hephzibah, Richmond, Georgia, United States, his father, Rev Robert Samuel Allen IV, was 27 and his mother, Elizabeth Anderson, was 27. He married Isabelle Jeanette Evans in 1838, in Burke, Georgia, United States. They were the parents of at least 6 sons and 4 daughters. He lived in Burke, Georgia, United States for about 10 years. He died on 1 November 1862, at the age of 60.