Joseph Jacobs was the son of Harriet Jacobs, a slave woman, and Samuel Tredwell Sawyer. Sawyer had promised Harriet that he would arrange for their children to be freed, but she had to pressure him. While Harriet was in hiding from Dr. James Norcom, Joseph's grandmother, Molly Hornblow, a freed woman, raised him and his sister, Louisa. By the time that the children were able to rejoin their mother, Joseph was nearly grown and independent. He and his uncle, John S. Horniblow, went prospecting for gold, first in California and then in Australia. John S. eventually returned home, but Joseph remained in Australia. It is not known what happened to him. His letters stopped, and then someone wrote his mother saying that he was sick and asking her to send money for his passage home. Nothing further was heard from or about him.