Graevius was originally intended for the law, but made the acquaintance of Johann Friedrich Gronovius during a casual visit to Deventer, under whose influence he abandoned jurisprudence for philology. He completed his studies under Daniel Heinsius at Leiden, and among others under the Protestant theologian David Blondel at Amsterdam.
During his residence in Amsterdam, under Blondel's influence he abandoned Lutheranism and joined the Reformed Church; and in 1656 he was called by the Elector of Brandenburg to the chair of rhetoric in the University of Duisburg. Two years afterwards, on the recommendation of Gronovius, he was chosen to succeed that scholar at Deventer; in 1662 he moved to the University of Utrecht, where he occupied first the chair of rhetoric, and in addition, from 1667 until his death, that of history and politics.
Graevius enjoyed a very high reputation as a teacher, and his lecture-room was crowded by pupils, many of them of distinguished rank, from all parts of the world. He was visited by Lorenzo Magalotti and honoured with special recognition by Louis XIV, and was a particular favourite of William III of England, who made him historiographer royal.
His two most important works are the Thesaurus antiquitatum Romanarum (1694-1699, in 12 volumes), and the Thesaurus antiquitatum et historiarum Italiae published after his death, and continued by the elder Pieter Burmann (1704-1725), although these have not always been looked upon favourably. His editions of the classics, although they marked a distinct advance in scholarship, are now for the most part superseded. They include Hesiod (1667), Lucian, Pseudosophista (1668), Justin, Historiae Philippicae (1669), Suetonius (1672), Catullus, Tibullus et Propertius (1680), and several of the works of Cicero, which are considered his best.
He also edited many of the writings of contemporary scholars. His library, rich in antiquarian classical books, was bought after his death by Johann Wilhelm, Elector Palatine (reigned 1690-1716); part of it was later transferred to Heidelberg University Library
Graevius died in Utrecht in 1703.