We do not know much about the beginnings of the castle. It is first mentioned in the 1352-1357 period. Its design and construction is very interesting. The castle was built on a steep and rocky hill at the end of a large ridge. The moat is 13 metres wide; the castle itself was created with use of a natural rock cleft. There is nothing like the traditional main tower: the main defence element was the peripheral bullwark. The palace and the cellars carved in the rock beneath are still preserved. The water reservoir in the opposite corner is 14 metres deep. The Chudý Hrádek is considered one of the most interesting ruins in the region of Českolipsko.

The access path from the Dolský Mill is rather steep, though.