The area was first inhabited over 7,000 years ago by Native Americans attracted to a warm mineral spring. The hydrological spring, locally known as the "Original Fountain of Youth", attracted patrons in the 19th century to the spa town, where more than a dozen hotels were built near the spring to accommodate them. Today, the sulfur-scented spring water feeds an adjacent public swimming pool before flowing the short distance to the St. Johns River. The Green Cove Springs area was first developed by George J. F. Clarke in 1816 when he was provided land, under a Spanish land grant, to build a sawmill. Green Cove Springs was established in 1854 as White Sulfur Springs. Renamed in 1866, it became the Clay County seat in 1871.
Agriculture and tourism were the two primary economic activities in the area until the end of the 19th century, when Henry Flagler's railroad began taking tourists further south in Florida. In 1895, the Great Freeze destroyed the area's citrus crops, and tourism all but ended. The 1920s had renewed development, with automobile traffic once again bringing in tourists. The Great Depression of the 1930s marked the end of this period of growth for the city.
The first women's club in the state of Florida was established in Green Cove Springs in 1883. The Village Improvement Association led local efforts to beautify the town, and established its first public library.
The period immediately before and during World War II again brought new growth to Green Cove Springs. On September 11, 1940, the U.S. Navy opened Naval Air Station Lee Field in honor of Ensign Bejamin Lee, who had lost his life in a crash at Killinghome, England, during World War I. In August 1943, the facility was renamed Naval Air Station Green Cove Springs and consisted of four 5,000-foot (1,500 m) asphalt runways. One of the Marine Corps aviators training in the F4U Corsair Operational Training Unit at Lee Field in early 1945 was eventual television personality Ed McMahon. After the war, NAS Green Cove Springs was downgraded in status to a Naval Auxiliary Air Station (NAAS) as part of the greater NAS Jacksonville complex. Thirteen piers were constructed along the west bank of the St. Johns River adjacent to NAAS Green Cove Springs to house a U.S. Navy Atlantic Reserve Fleet, Florida "mothball fleet" of some 500 vessels, primarily destroyers, destroyer escorts, and fleet auxiliaries. In 1960, the Navy decommissioned NAAS Green Cove Springs and the pier facility. Some of the mothballed vessels were transferred to foreign navies, while others were relocated to other Reserve Fleet locations.