Saint Mary's Seminary was founded in 1848, after Bishop Amadeus Rappe was consecrated the first Bishop of Cleveland. He purchased a residence on East 6th Street and St. Clair Avenue. The adjacent buildings were made into classrooms for Cleveland’s first diocesan seminary. By the second year of its existence, the seminary had 18 students and needed more space. Bishop Rappe purchased a new larger site on Lake and Hamilton Streets and added multiple additions to the building. In 1859, Bishop Rappe began building a new seminary on the same site. It was occupied by the Philosophy and Theology departments, while the older buildings became the classics department, Saint Mary’s College.

Then in 1924, the Ordinary, Bishop Schrembes, broke ground for a new seminary building on Ansel Road. In the new building, Bishop Schrembs intended to include the Philosophy Program, which prior to that time had been in Cincinnati. In 1929, the Philosophy and Theology Departments were returned to the new Saint Mary, which then became Cleveland’s major Seminary with both a Philosophy Department and a Theologate. In 1954, Archbishop Edward F. Hoban transferred the Philosophy Program from Saint Mary to the newly established Borromeo Seminary of Ohio in Wickliffe, Ohio. Since then, Saint Mary Seminary and Graduate School of Theology has been exclusively a graduate theologate. From 1962 to 1970 The Seminary was an affiliate of Washington D.C.’s Catholic University of America.