The two-storey castle is the mightiest building in Brežice, built in the middle of the 16th century in the site of the castle which was burnt down in 1515, at the time of the great peasant revolt. In the early 17th century and late 18th century, the aristocratic family Attems turned the castle into a residence.