Born in New York City, Cannon started at the New York Daily News when he was 17. He later wrote for the New York Post, New York Journal-American and King Features Syndicate. He was a war correspondent for Stars and Stripes during World War II. He also wrote a column for Newsday during the 1950s.
A long-time boxing writer, Cannon once wrote that "boxing is the red light district of sports". When Sugar Ray Robinson was making his famous comeback, Cannon told him not to return to the ring. He wrote about this several times in his column. He famously said of Joe Louis that "he is a credit to his race, the human race" and was one of the first sportswriters to see the importance of the black athlete. A contemporary of Ernest Hemingway, he much admired Hemingway's writing, and the admiration was mutual.
On frequent occasions, when Cannon had no particular sports news to report, he would still manage to fill his daily column space by starting off with the phrase "Nobody asked me, but..." and then filling the rest of the column with his random opinions on any and every subject outside of the sports world. This gambit has been eagerly seized upon by newspaper columnists ever since, not only on the sports page but in every other section. Columnists who "borrow" this device will typically lead off with some lip-service tribute to its originator, such as "In the words of the immortal Jimmy Cannon: Nobody asked me, but..." and then they're off.
Cannon is mentioned in Ian Fleming's James Bond novel, Diamonds Are Forever, in which Fleming describes Cannon's prose as "muscular" and "craftsmanlike". Felix Leiter hands Bond a cutting on the horse-racing in Saratoga Springs from the New York Post, telling him, "This Jimmy Cannon is their sports columnist. Good writer. Knows what he's talking about."
Cannon is described as "an attractive young man with wide, straight eyes and a rather thin-lipped smile."