In 1315 King Henry of Habsburg granted the other half of the castle to the burgrave Henry of Parcines in fiefdom, with the condition of acquiring the remaining part as well. Enrico accepted and settled in his half of the castle, also changing his surname to von Annenberg.
In 1327 Egno di Màzia granted him his half of the castle initially for 8 years, but later Enrico became its sole owner. Then began a restructuring of the casatorre that by now bore the signs of the time.
In 1364 Henry von Annenberg died and his four children shared the inheritance. The castle went to Anton von Annenberg who created a library of manuscripts and incunabula. Between the end of the fifteenth and the beginning of the sixteenth centuries, the von Annenbergs underwent major renovations, also following the damage due to the Swabian war. They enlarged the castle by building a residential building and added quadrangular walls with circular turrets at the corners, which was later taken as a model for that of Castel Coldrano.
In 1695 after the extinction of the von Annenbergs, the castle passed into the hands of the Counts of Mohr who already owned the castle of Laces and the castle Montani di Sopra and di Sotto.
The family of the counts of Mohr also became extinct and a large part of their heritage was dispersed, including the library of incunabula, while the castle passed to the lords von Haussmann who kept the property until 1813.
However, by the end of the 17th century it had already begun the decline of the manor, which got worse when the von Haussmanns sold it to peasants. The castle soon became a ruin with collapsed roofs and floors. In 1896 the cavalry captain Martin Stocker bought the castle and began its rehabilitation, which continued in 1910 with the new owner Captain Schörger, who restored the building by raising it a floor and making it habitable again.
In 1923 the property passed to Paolo Drigo who ceded it in 1928 to the Bensa accounts of Genoa who continued the renovation work. In 1972 the castle was sold to the veterinarian Karl-Heinz Politzar, to whose death in 2010 it passed to the Fuchs family, current owner.