Mount Zion Baptist Church was organized on July 18, 1869, with fifteen charter members. The Rev. Mitchell Harrison (d. 1906) was the congregation's first pastor and is credited with naming the new church.
The congregation worshiped in a number of places before permanently locating here in 1890. An outdoor brush arbor was built two miles north of Concrete (4. 5 miles northwest), which was later replaced by a log structure south of Concrete. After a storm destroyed the log church a new wooden building was erected, which was disassembled and reconstructed at this site in 1890. The first worship service was held in the new building on October 31, 1890.
The Rev. Mr. Harrison continued to serve as pastor of Mount Zion until his death. He was succeeded by the Rev. S. A. Pleasants, who served until 1923, at which time the Rev. N. E. Henderson became pastor, serving until his death in 1947. The church purchased a schoolhouse for use as a fellowship hall in 1951. The church building was renovated in 1956, with the original twin bell towers being replaced by a new steeple. At the time of its centennial in 1969 Mount Zion Baptist Church had been guided by only four pastors.