Klein was born in Libava, Russian Empire. He studied at Berlin and Marburg, where he received his Ph.D. in 1922. A student of Nicolai Hartmann, Martin Heidegger, and Edmund Husserl, he later taught at St. John's College in Annapolis, Maryland from 1937 until his death. He served as dean from 1949 to 1958.
Klein was affectionately known as Jasha (pronounced "Yasha"). He was one of the world's preeminent interpreters of Plato and the Platonic tradition. As one of many Jewish scholars who were no longer safe in Europe, he fled the Nazis. He was a friend of fellow émigré and German-American philosopher Leo Strauss. Of Klein's first book Greek Mathematical Thought and the Origin of Algebra, Strauss said:
The work is much more than a historical study. But even if we take it as a purely historical work, there is not, in my opinion, a contemporary work in the history of philosophy or science or in "the history of ideas" generally speaking which in intrinsic worth comes within hailing distance of it.
The central thesis of his work Greek Mathematical Thought and the Origin of Algebra is that the modern concept of mathematics is based on the symbolic interpretation of the Greek concept of number (arithmos).
Klein died in 1978 in Annapolis, Maryland.