While introducing Elvis Costello‘s 27th and final appearance on his show, Late Show host David Letterman noted that the English singer/songwriter had always gone out of his way to make each of his performances special. Friday night was no exception as Costello, with assistance from the vocal group the Quills, sang a medley of compatriot Nick Lowe‘s  tune “When I Write the Book” and Costello’s own “Everyday I Write the Book.”

Costello’s medley was essentially a rendition of Lowe’s Rockpile tune with a brief verse and chorus of his own song inserted as a bridge. Accompanied by his acoustic guitar and the Quills’ backing vocals, Costello’s performance was genuine and more than a bit wistful—a fitting farewell to his friend, Letterman.

Costello actually guest hosted an episode of Late Show, filling in for an ill Letterman back in 2003; he was also the musical guest, playing another Lowe-penned tune, “(What’s So Funny ‘Bout) Peace, Love and Understanding.” Watch Elvis Costello sing “When I Write the Book” and “Everyday I Write the Book” below: