Charles Sprague Sargent (1841-1927) was educated at Harvard, served in the military, and traveled Europe for a few years before returning to Boston and taking over the management of the family estate, Holm Lea. Sargent did not have a formal botany education but possesed good botanical instincts. He was called to Harvard in 1872 and soon assumed the Directorship of the Arnold Arboretum. 

In 1863 James Arnold of New Bedford, Massachusetts left over $100,000 to Harvard for "...the promotion of Agricultural, or Horticultural improvements...". This gift was combined with a parcel of land in Jamaica Plain given to the university in 1842 by Benjamin Bussey. Unfortunately with the small stipend of only $3,000 a year, it seemed impossible to turn the land into a flourishing Arboretum.

Sargent, along with Frederick Law Olmsted, undertook a massive job. They worked to convince both the Harvard Corporation and the city of Boston that it would be in Harvard's best interest if the city took the land. The city would then lease the property back to Harvard for 1,000 years, at $1 a year, with an option to renew. In that way the city of Boston would bear the cost of constructing roads and paths and Sargent's funding could go towards the development of the grounds. This was no small undertaking, but finally both parties agreed in December 1881. The Arboretum was now part of the city's "Emerald Necklace" and Olmsted and Sargent began the difficult job of planning and designing the Arboretum.

Sargent served 54 years as Director of the Arboretum. During that time it grew from the original 120 acres to 250 acres. Sargent also continued his own research and writing. He wrote many books including Silva of North America, Trees of North America, and Forest Flora of Japan. He also served as editor for the journal Garden and Forest.  Additional biographical information for Charles Sprague Sargent may be found in the finding aid of the Charles Sprague Sargent Papers at the Arnold Arboretum.

Besides collecting plants and specimens, Sargent also acquired books and journals for the Arboretum library. The collection grew from no books in 1872 to over 40,000 by 1929. Most of these were purchased at Sargent's own expense. By the time of his death Sargent had donated his entire library to the Arboretum as well as a large financial gift for upkeep of the existing collection and the purchase of more materials.

In 1954 many of the library materials of the Arnold Arboretum were moved to Cambridge and merged with the Gray Herbarium Library while all of the books and journals and most of the archival materials related to the living collections remained in Jamaica Plain.

Today, the Arnold Arboretum Library in Cambridge specializes in the identification and classification of Eastern Hemisphere plants, with special emphasis on Southeast Asia. The subjects include systematic botany, floras of the Eastern Hemisphere, literature on woody plants, and books on poisonous plants.