Asa Gray (1810-1888) is well known as the "Father of American Botany" and champion of Charles Darwin. He was called to Harvard in 1842 as the new Fisher Professor of Natural History. At that point there was no herbarium, no library, and only a small greenhouse and garden. Whatever cash Gray could spare from his salary went into the cultivation of his library and herbarium, which soon took over his house. In 1864 he offered his collections to Harvard with the stipulation that they build a suitable building to house them. That same year a small brick building was built and the collections were moved. Additional biographical information for Asa Gray may be found in the finding aid of the Asa Gray Papers.

Today the Gray Herbarium Library specializes in botanical history, floras of the Western Hemisphere, and Linnaeana and pre-Linnean sources. It also contains extensive archival collections.

In 1954 the Gray collection was merged with the research materials from the Library of the Arnold Arboretum. With the Arnold’s emphasis on Eastern Hemisphere plants, this merger helped make Harvard's botany collections extremely comprehensive.