In 1992, Pipe, Ron Liberti (vocals), Mike Kenlan (guitar), Dave Alworth (bass), and former Superchunk drummer Chuck Garrison, released the Ball Peen EP (Sonic Bubblegum). Kenlan then left to concentrate on Small (later Small 23, which also included Garrison.

Clifton Lee Mann of the Bad Checks came on to play guitar temporarily until a permanent guitarist could be found. Mann and Liberti worked well together, and Mann stayed. Pipe recorded their first single "You're Soaking in It" in 1993 followed by their full-length debut, Six Days to Bellus (Jesus Christ) in 1994.

Pipe then toured with Bad Brains and soon began work on their second full-length, International Cement (Jesus Christ), released in 1995 along with the "Raceway Park" single on Amish Records. Bassist Alworth quit the band after a tour with Polvo and Archers of Loaf. Greg Adams replaced Alworth, and Pipe recorded their third album, Slowboy, released on Merge in 1997. Merge then re-released Pipe's two earlier albums and several singles. An unfortunate series of events (Garrison's broken hand early during a tour and an injury to Mann's hand and eventual departure) led to the dissolution of Pipe.

Submariner split w/ Small [7"] (Matt Label) 1991
6 Days Til Bellus [cd] (Merge) 1991
Ball Peen [cd-ep] (Sonic Bubblegum) 1992
You're Soaking In It [7"] (Merge) 1992
6 Days Til Bellus / International Cement [dbl-lp] (Merge) 1992
"Ashtray" b/w "Warsaw" [7"] (Merge) 1993
split 7" w/ Rubbermaid [7"] (Merge) 1993
International Cement [cd] (Merge) 1993
You're Soaking In It / Trouble [7"] (Amish) 1994
"Ashtray" on Rows of Teeth comp. [cd] (Merge) 1994
"Human Gutterball" b/w "Figure 8" [7"] (Merge) 1994
Six Days Till Bellus [cd/lp] (Jesus Christ) 1995
International Cement [cd/lp] (Jesus Christ) 1996
Raceway Pk. split w/ Rubbermaid [7"] (Amish) 1996
Slowboy [cd/lp] (Merge) 1997
"Chula" on Bridges of Dissonant County Vol. 1 comp. [cd] (?) 1999
"Sheer Heart Attack" on Oh, Merge comp. [cd] (Merge) 1999