Stolen Babies formed from a 12+ member high school performance troupe named the Fratellis; the band takes its name from one of the skits performed by the group during this period (written by Dominique Persi, who handled the theatrical and performance-art aspect of the group). Stolen Babies released their first Demo CD in 2002 through their own label, No Comment Records.

Among the band's many musical influences are groups such as Oingo Boingo, Mr. Bungle, and Fishbone (with whom Gil Sharone has performed). Stolen Babies are known for their on-stage theatrics and energetic performances, filled with props and Gorey-esque backdrops, all made by the various members. Except for the earliest demo, each album has featured artwork by indie comic artist Crab Scrambly.
There Be Squabbles Ahead, the band's debut full-length album (featuring remixed versions of "A Year of Judges" and "Push Button" from the 2004 demo), was released early in 2006 through the band's website, but was re-released on October 3 following the band's signing with The End Records. This album was produced by Dan Rathbun, member of the similarly "avant garde" experimental rock group Sleepytime Gorilla Museum, who also provides instrumentation on track 7 (tuba) as well as various background vocals on tracks 3, 7, and 12. Carla Kihlstedt (from Sleepytime Gorilla Museum) also contributed violin on tracks 7 and 11. The re-release also featured a music video for the song "Push Button" directed by Erik Tillmans. This album also marked the departure of Davin Givhan, the band's former guitarist. Guitarist George Earth, formerly of Switchblade Symphony, took Givhan's place during the international tour that followed.
They supported Lacuna Coil along side within temptation and the gathering on their The Hottest Chicks in Metal Tour 2007 featuring female-fronted metal bands.