Crispin Hellion Glover has been played on NTS in shows including Black Impulse, featured first on 31 March 2018. Songs played include These Boots Are Made For Walking.

In 1989, during a hiatus from films, Glover released an album called "The Big Problem ≠ the Solution. The Solution = Let It Be " through Restless Records, produced by Barnes & Barnes (of "Fish Heads" fame). The album features original songs like "Clowny Clown Clown", warped covers of Lee Hazlewood's "These Boots Are Made for Walking'" and Charles Manson's "Never Say 'Never' to Always" (sung in falsetto), and readings from his art books Rat Catching and Oak Mot (Glover modified old books with expired copyrights by adding or deleting pictures, text, and drawings). Sample pages from these books are featured in the album's liner notes. The music itself is similar to outsider music, with seemingly absurd, dream-like lyrics.

The back cover of the album is a collage of figures relating to each track on the album, with a puzzle: "All words and lyrics point to THE BIG PROBLEM. The solution lay within the title; LET IT BE. Crispin Hellion Glover wants to know what you think these nine things all have in common." He included his home phone number with copies of the album, encouraging listeners to phone when they had "solved" his puzzle. Glover later commented that he was surprised how many people figured it out.

In 2003, he recorded a cover version of the Michael Jackson classic song "Ben" to coincide with the release of the film Willard. In the eccentric music video for the song, which is included on the Willard DVD, he sings to a rat named Ben. In the commentary for the film, he remarks that he has recorded a second album.