In 1992, Pipe, Ron Liberti (vocals), Mike Kenlan (guitar), Dave Alworth (bass), and former Superchunk drummer Chuck Garrison, released the Ball Peen EP (Sonic Bubblegum). Kenlan then left to concentrate on Small (later Small 23, which also included Garrison.

Clifton Lee Mann of the Bad Checks came on to play guitar temporarily until a permanent guitarist could be found. Mann and Liberti worked well together, and Mann stayed. Pipe recorded their first single "You're Soaking in It" in 1993 followed by their full-length debut, Six Days to Bellus (Jesus Christ) in 1994.

Pipe then toured with Bad Brains and soon began work on their second full-length, International Cement (Jesus Christ), released in 1995 along with the "Raceway Park" single on Amish Records. Bassist Alworth quit the band after a tour with Polvo and Archers of Loaf. Greg Adams replaced Alworth, and Pipe recorded their third album, Slowboy, released on Merge in 1997. Merge then re-released Pipe's two earlier albums and several singles. An unfortunate series of events (Garrison's broken hand early during a tour and an injury to Mann's hand and eventual departure) led to the dissolution of Pipe.

split 7" w/ Small :: 7" (Matt Label) 1991
Ball Peen EP :: (Sonic Bubblegum) 1992
"Your Soaking In It" :: 7" (Amish) 1993
"Ashtray" b/w "Warsaw" :: 7" (Merge) 1993
"Human Gutterball" b/w "Figure 8" :: 7" (Merge) 1994
6 Days till Bellus :: (Jesus Christ) 1994
International Cement :: (Jesus Christ) 1995
"Raceway Park" :: 7" (Amish) 1995
Slowboy :: CD/LP (Merge) 1997