Yeah, it only plays a door sound if switching between "outside" (so any maps like cities and routes) and "inside" (houses, caves, towers etc).
Due to its open ceiling, the Burned Tower is declared as outside map, so when you pass through the doors, your don't switch between two different types of maps.
However, if you go to the lower level of the Burned Tower, it plays the sound, because that map is closed on top, so it's a change from outside -> inside.