Flesch was born in Moson (now part of Mosonmagyaróvár) in Hungary in 1873. He began playing the violin at seven years of age. At 10 he was taken to Vienna to study with Jakob Grün. At 17 he left for Paris, and joined the Paris Conservatoire, studying with Martin Pierre Marsick. He settled in Berlin, and in 1934 in London.
He was known for his solo performances in a very wide range of repertoire (from Baroque music to contemporary), gaining fame as a chamber music performer. He also taught in Bucharest (1897-1902), Amsterdam (1903-08), Philadelphia (1924-28) and Berlin (Hochschule fuer Musik, 1929-34). He published a number of instructional books, including Die Kunst des Violin-Spiels (The Art of Violin Playing, 1923) in which he advocated for the violinist as artist rather than merely virtuoso. Among his pupils were Charles Barkel, Edwin Bélanger, Bronislaw Gimpel, Ivry Gitlis, Szymon Goldberg, Ida Haendel, Josef Hassid, Adolf Leschinski, Alma Moodie, Ginette Neveu, Yfrah Neaman, Ricardo Odnoposoff, Eric Rosenblith, Max Rostal, Henryk Szeryng, Henri Temianka, Roman Totenberg and Josef Wolfsthal, all of whom achieved considerable fame as both performers and pedagogues. He said his favorite pupil was the Australian Alma Moodie, who achieved great fame in the 1920s and 1930s but made no recordings and is little known today. In his memoirs he said, "there was above all Henry [i.e., Henri] Temianka, who did great credit to the [Curtis] Institute: both musically and technically, he possessed a model collection of talents." See: List of music students by teacher: C to F#Carl Flesch.
One of Flesch's few recordings is a highly distinguished interpretation of Bach's great D minor Double Violin Concerto (Columbia) in which he played second violin to the great Joseph Szigeti, with Walter Goehr conducting an anonymous London string orchestra in the late 1930s.
He was consulted (as was Oskar Adler) by Louis Krasner over technical difficulties in the Violin Concerto of Alban Berg, which Krasner was to premiere. Carl Flesch's Scale System is a staple of violin pedagogy.