The community of Pilot Grove was named for a landmark grove of trees. One of its first settlers was John Wesley Vick, who brought his family here about 1875. When the need for a community cemetery became evident, Vick donated one acre of his land for that purpose, provided that it was maintained as a free burial ground. The first adult buried in the Pilot Grove Cemetery was Mary C. Williams in 1880, although it is known that several children of early residents were buried here before that time.
In 1889, J. Wesley Vick and E. J. Vick deeded one acre of land adjoining the graveyard to the Mt. Herman Baptist Church, which had been organized the same year. The congregation built this wood frame sanctuary on the land in 1891. Descendants of the church's charter members continued to worship here until 1948, when the lack of a regular pastor and regular worship services caused the membership to dwindle to five.
Descendants of early Pilot Grove residents have cared for the cemetery and church building over the years. The original cemetery association that formed in 1906 combined with the remaining members of the Mt. Herman Church in 1962 to form the Pilot Grove Church and Cemetery Association.