Isham Pitman made his way here prior to the Revolutionary War. He and his wife Absela farmed the land near Pitman Mill Branch, a tributary of Ashpole Swamp, eventually purchasing a 1775 Land Grant from King Charles II to Daniel Willis, brother of John Willis, founder of Robeson County.

Pitman dammed the branch, making a mill pond and building a grist mill to grind corn and wheat. He built a house nearby.

After the Revolutionary War began Pitman was called to fight for the colonists. Sending his wife to her family home on the Neuse River in Craven County, he turned out all of his livestock to forage the land. When he returned after the war, Pitman began purchasing Land Grants from the state of North Carolina. Eventually his land ownings totaled more than 24,000 acres, all of it in the Ashpole Swamp -- Hog Swamp area surrounding Fairmont. All of the land in downtown Fairmont was Pitman land.

In 1792 Pitman began preaching, organizing Pitman's Meeting House, a Baptist Church. In 1794 he build a building just north of his grist mill where he preached as pastor until his death. About 1820, recognizing that his days were getting short and having no children of his own, he convinced his nephew, Elias Pitman, to move from Horry County, South Carolina to look after Absela and himself until their death. He executed his Last Will and Testament on March 31, 1823, leaving the bulk of his land to Elias. He did leave a tract of land on the west side of Oldfield Swamp to nephews Silas and Hugh Pitman, sons of Joel. This land became part of downtown Fairmont, eventually being owned by J. A. B. Pitman through marriage to Silas' daughter Hannah. He also bequeathed slaves to his wife and other relatives.

Isham Pitman died in June, 1825 and is buried 1/2 mile west of the First Baptist Church off of Church Street.