TWDY was formed by guitarists Chris King and Jeremy Galindo, bassist Raymond Brown and drummer Andrew Miller in San Marcos, Texas, in 2005. They had met through mutual friends, and played together in various different bands throughout high school, before the line-up was finalised by around 2002. Early iterations of the band experimented with vocals, sung by Galindo, but after recording some tracks they decided the results were "awful" and didn't fit in with the rest of their music. The band then tried writing different tracks, one of which was instrumental. Chris King said in an interview with BBC Northern Ireland, "we were … writing different kinds of songs and we wrote one [instrumental] song, and we were like, that works, lets go for it!" The band took it's name from an early song that they were originally going to call "TWDY", but this was rejected for being too pretentious. The band found the name "hilarious". When asked whether he wished the band had chosen a different name, King said to Rock Sound, "It's supposed to be a little bit obnoxious. There's something about people automatically hating you before you play that's kind of endearing. It gets to the point where it's over-the-top obnoxious. It's an attention-grabber and people will check it out I guess. Even if they hate it."

The band self-recorded and self-produced Young Mountain in 2005. This was intended to be a demo, and was sold as a CD-R after their concerts. The record was well received by critics, including being named one of the best instrumental releases of the year by The Silent Ballet. The band caught the attention of Magic Bullet Records, who re-packaged the record and re-released it in 2006. After recording their eponymous debut studio album, TWDY, in February 2007, the band's tour schedule saw them play with such bands as 65daysofstatic. After the release of the album in January 2008, the band parted company with bassist Raymond Brown, who left in order to focus on his medical career, replacing him with Donovan Jones.

The band released a split EP, titled Field Studies, with their labelmates Lymbyc Systym, in January 2009, although it was made available through mail order from the previous October. Field Studies includes two songs by TWDY: "Brutalism and the Worship of the Machine" and "Freedom Blade." The band then set out on a series of tours, including an appearance at the ATP festival in April, playing alongside bands such as Devo and The Jesus Lizard. In October of the same year the band performed at Damnation Festival, where they shared a stage with Jesu. In November 2009 the band replaced Andrew Miller with Alex Bhore, drummer for Texas group The Rocketboys, citing musical differences.

In May 2010 the band released an exclusive track entitled "Their Celebrations" as part of PEACE, a compilation in aid of Amnesty International,[23] after a request from the charity. The band subsequently released a two-song 12″ EP entitled "Moving on the Edges of Things" in August 2010, before embarking on a September US tour, supporting the metal group Deftones. This was followed by the release of a two-track 7″ single, Communal Blood, in December of the same year, although it was available at their live shows from May. This is the bands first single and is taken from their forthcoming second full length, Tunnel Blanket. This has been recorded and mastered by John Congleton, and is due for release on 9th May on Monotreme Records in Europe and on Suicide Squeeze Records outside of Europe.

A Place To Bury Strangers have often been called "the loudest band in New York". This may very well be the case, but unlike much so-called "loud" rock and roll that's out there, APTBS is not loud simply for the sake of it. The sonically overdriven sound they've accomplished is no clumsy accident, but a carefully cultivated and well-maintained entity all its own, fostered by an unbridled passion that's clearly evident in every live show they play and each recording they make. A Place To Bury Strangers does not so much play songs as allow them to pour out. They are songs about longing, heartbreak and confusion played extremely well and at a passionately loud volume.