Krauss was born in Vienna to Clementine Krauss, then a 15-year-old dancer in the Vienna Imperial Opera Ballet, later a leading actress and operetta singer, niece of the prominent nineteenth-century operatic soprano Gabrielle Krauss. His natural father, Chevalier Hector Baltazzi (1851-1916), came from a family of wealthy Phanariot bankers resident in Vienna. Baltazzi's older sister Helene was married to Baron Albin Vetsera and was the mother of Baroness Mary Vetsera, who was accordingly Clemens Krauss' first cousin.
Krauss sang in the Hofkapelle (Imperial Choir) as a Vienna Choir Boy. He graduated from the Vienna Conservatory in 1912, after studying composition with Hermann Graedener and theory with Richard Heuberger there. He was then appointed chorus master in the Brno Theater (1912-1913), where he made his conducting debut in 1913. The famous Romanian soprano Viorica Ursuleac, who often sang under him, became his second wife.
Krauss made the rounds of regional centers, conducting in Riga (1913-1914), Nuremberg (1915) and Stettin (1916-1921) (formerly part of Pomerania in Germany; now part of Poland). The latter appointment gave him ample opportunity to travel to Berlin to hear Arthur Nikisch conduct the Berlin Philharmonic, a major influence. He then returned to Austria as director of the opera and symphony concerts in Graz. In 1922, he joined the conducting staff of the Vienna State Opera and teacher of the conducting class in the Vienna Singakademie. He conducted the Vienna Tonkünstler concerts from 1923 to 1927, and was Intendant of the opera in Frankfurt and director of the Museum concerts there from 1924 to 1929.
He guest-conducted in the United States in 1929, with the Philadelphia Orchestra and New York Philharmonic. Also in 1929, he was appointed director of the Vienna State Opera. Its orchestra, which was part of the independent concert entity known as the Vienna Philharmonic, appointed him its music director in 1930. He was a regular conductor at the Salzburg Festival from 1926 to 1934, where in 1930 he conducted Alban Berg's avant-garde atonal opera Wozzeck.
Krauss gave up his Vienna positions in 1933-34 to direct the Berlin State Opera in 1935 after Erich Kleiber had resigned in protest against Nazi policies. In 1933, he took over the preparations for the premieres of Richard Strauss's opera Arabella after the departure of conductor Fritz Busch (another non-Jewish anti-Nazi). Krauss's own position on Nazism was unclear, although he did enjoy a close relationship with Nazi official Alfred Frauenfeld and it has been claimed that he sought Nazi Party membership in 1933. However, throughout the 1930s he and his wife were involved in helping Jews escape from Germany. After befriending British novelist Ida Cook and her sister Mary, both opera fans, he gave cover to their smuggling operation (started after Viorica asked the Cooks to assist a Jewish friend). Munich Opera House shows were arranged around the times and cities that the Cooks needed to make contact with escapees.
In 1937, he was appointed Intendant of the National Theatre Munich following Hans Knappertsbusch's resignation. He became a close friend of Strauss, and even wrote the libretto for his opera Capriccio which he premiered in Munich in 1942. He also conducted the premieres of Strauss's operas Friedenstag and Die Liebe der Danae. During the early 1940s, he taught at the Mozarteum University of Salzburg where among his pupils was composer Roman Toi.
After the Munich opera house had been destroyed by Allied bombing, Krauss returned to conduct the Vienna Philharmonic in 1944-45 until it ceased activities shortly before the end of World War II. After the war, Allied officials investigated his career and forbade him from appearing in public until 1947, when it emerged that he had helped Jews escape from the Third Reich. Krauss then resumed conducting many of the Vienna Philharmonic's concerts, including its famous annual New Year's Day pops concerts featuring Johann and Josef Strauss waltzes, overtures and polkas, many of which were recorded for Decca along with other studio recordings of mostly Johann, Josef and Richard Strauss.
He conducted opera at Covent Garden in London starting in 1951, and at the 1953 Bayreuth Festival (including an impressive Wagner Ring cycle now available on CD, starring Astrid Varnay as Brunnhilde). He also recorded a highly regarded Parsifal at Bayreuth, starring Martha Mödl as Kundry, in 1953, at the height of Mödl's brief prime.
Krauss made relatively few recordings; but his amiable, spirited 1950 Decca rendition of Johann Strauss II's Die Fledermaus, with the Vienna Philharmonic and State Opera star soloists (not including any of the dialogue; only the second complete recording after a pre-World War I acoustical 78 set made in Berlin), is still regarded by many as the best, most authentic and most "gemütlich." His 1953 live performance of Richard Wagner's Ring Cycle from Bayreuth is highly regarded. A performance with the Vienna Symphony of Beethoven's Choral Fantasy, reissued on more than one inexpensive label since its original appearance on a Vox LP, is also one of the few recordings featuring pianist Friedrich Wührer. All three have been reissued on compact disc.