Little remains of the initial structure of the fortress, which has undergone countless subsequent renovations.
On 7 July 1382, Gonçalo Vasques de Azevedo, commander of the fortress on the border between the Tagus and the Guadiana, brought captains, alcaides and commanders of other minor border fortresses together here to defend the Alentejo against a Castilian invasion. They included the young Nuno Álvares Pereira. In July 1383, six thousand men left Vila Viçosa for the border at Elvas.
Following the death of King Fernando, Vila Viçosa took the side of Castile under its governor, Vasco Porcalho, albeit against the wishes of the townspeople. Vila Viçosa would only return to Portuguese rule after the Battle of Aljubarrota, when King João I gave it to the Warden of the Kingdom, Nuno Álvares Pereira.
Dom Fernando, who received the title of Count of Arraiolos from his grandfather, Nuno Álvares Pereira, and became 2nd Duke of Bragança upon the early death of his eldest brother, Dom Afonso, the Count of Ourém, established himself at Vila Viçosa. Ever since then, this was the chosen home of the House of Bragança, whose nobles lived in the castle fortress until around 1502, when construction work began on a palace at Reguengo, which is now the Ducal Palace of Vila Viçosa.
The exclusively military function assumed by the castle from the 16th century on required extensive alterations. Teodósio I had it laid out as an artillery castle, the only one of its kind in Portugal, based on a plan by Benedetto da Ravenna, the architect behind Mazagan Fortress in North Africa.
Following the Restoration of Independence and in order to withstand potential attacks, the Castle was adapted by Cosmander to provide the kind of defences required in that age, by adding a polygonal, star-shaped system and reinforcing the citadel walls. It was fortified once again by Schomberg in 1662 for its last major military conflict, which saw the Portuguese triumph over the Castilian army led by the Marquis of Caracena at the Battle of Montes Claros in June 1665.
Up until the late 19th century, the palace housed an infantry regiment.
The walls were restored in the 1930s and 40s at the behest of the General Directorate of National Buildings and Monuments, while the preservation of the ramparts and fortress of Vila Viçosa Castle were funded by the Fundação da Casa de Bragança.
The walled enclosure contains the Igreja Matriz, the main church that holds an image of Our Lady of the Immaculate Conception of Vila Viçosa.