Mitchell was born in Wethersfield, Hartford County, Connecticut on December 9, 1743 and was the son of James and Rebecca Mitchell. He pursued academic studies; graduated from Yale College in 1763, and served as tutor at Yale from 1766 to 1769 while he studied law at New Haven and with Jared Ingersoll. He was admitted to the bar in 1770 and commenced practice in Newton, Connecticut. He married Hannah Grant and they had eleven children, Donald Grant Mitchell, Stephen Mix Mitchell, Lewis Mitchell, Charles Mitchell, Rebecca Mitchell, Alfred Mitchell, Walter Mitchell, Hannah Grant Mitchell, Harriet Mitchell, Elizabeth Mitchell Chester, and Julia Mitchell.
Mitchell returned to Wethersfield in 1772 and continued the practice of law. He was a member of Connecticut state house of representatives from 1778 to 1784. He was a delegate to the Continental Congress from Connecticut from 1783 to 1788, and a member of Connecticut council of assistants from 1784 to 1792 (with the exception of 1786). He was also associate justice of the county court of Hartford County from 1779 to 1790, and presiding judge from 1790 to 1793. He was a member of the State convention which ratified the Constitution of the United States in 1788.
When Roger Sherman died in 1793, Governor Huntington appointed Mitchell to the United States Senate, where he served from December 2, 1793, to March 3, 1795. He did not seek re-election in 1794, but returned home to accept a seat on the Connecticut Supreme Court beginning in 1795 until he was elevated to Chief Justice in 1807 and served there until his retirement to Wethersfield in 1814 when he became legally disqualified by age. In September 1807, he received the honorary degree of LL. D. from Yale College.
Mitchell was a presidential elector on the Federalist ticket in 1800 and a delegate to Connecticut state constitutional convention, 1818.
Mitchell died in Wethersfield on September 30, 1835 (age 91 years, 295 days). He is interred at Wethersfield Cemetery, Wethersfield, Connecticut. He was the grandfather of author Donald Grant Mitchell.