Prem Castle belongs to the older castles of Slovenia. Official sources first mention it indirectly as early as in 1213, while the building itself dates back to the 12th century.
According to legend, Prem Castle was to have been created in the location of an old Roman fortress; however, no archaeological remains that would confirm or refute this legend have been discovered so far. The first owners of the castle were the Lords of Duino. The nobleman Udo de Prem was one of the most powerful vassals of the Patriarchate of Aquileia and gained many estates in the Kras and in the north Kvarner region.
The Duino family line ended in 1399; the estates in their property were given to the management of the Walsee family. With the increasing influence and power of the Hapsburgs in the 15th century, the Walsees were forced to sell their most important estates of Devin, Štivan, Senožeče, and Prem. In subsequent centuries the castle saw a number of owners, among which the most important ones was the Porcia family from Friuli. They owned the castle until the beginning of the 20th century. In 1927, it was bought as a ruin by the Trieste doctor, Bruno Zucculini, who set up his summer residence in the castle. Today the castle is renovated in the style that had been introduced by its last owner; in the interior, visitors can see the castle rooms, containing different museum collections.