Foster was born in Andover, Massachusetts in 1735 the son of Asa and Elizabeth Foster. He graduated from Harvard College in 1756 and went on to study theology. His ordination as a pastor in Canterbury was in 1761, and he married Hanna Badger in that year. He served as pastor in Canterbury until 1779. Hanna died in 1768. With his second wife, Mary Wise Rogers, he had eight children, Hannah, William, James, Sarah, Martha, Abiel, Mary, and Elizabeth.
From 1783 to 1785, Foster was a delegate for New Hampshire to the Continental Congress. On March 3, 1789, he became a member of the First United States Congress as a Representative from New Hampshire until March 3, 1791. He returned to the state legislature in 1791, serving there until 1794 when he was elected again to the U.S. House of Representatives. He served there from March 3, 1795 to March 3, 1803.
Foster died in Canterbury on February 6, 1806 (age 70 years, 182 days). He is interred at the Center Cemetery, Canterbury, New Hampshire.