Phillips was the son of James Phillips, printer and bookseller in London. He became interested in mineralogy and geology, and was one of the founders of the Geological Society of London (1807). His Outlines of Mineralogy and Geology (1815) and Elementary Introduction to the Knowledge of Mineralogy (1816) became standard textbooks.
His digest of English geology, A selection of Facts from the Best Authorities, arranged so as to form an Outline of the Geology of England and Wales (1818), formed the foundation of the larger work undertaken by Phillips in conjunction with William Conybeare, of which only the first part was published, entitled Outlines of the Geology of England and Wales (1822). This book had a major influence on the development of geology in Britain. In this work Phillips reprinted his description of the chalk cliffs of Dover and other parts of East Kent. Phillips was a member of the Religious Society of Friends. He was elected a Fellow of the Royal Society in 1827.