The garden is well worth a visit, regardless of the time year. The daffodils around Easter cover the policies with various shades of yellow, replaced by the vibrant reds, pinks and whites of the rhododendrons and azaleas that the West Highlands are famous for. Other highlights are the heathers, roses and varied trees that are spectacular later in the summer and autumn.

The garden covers sixteen acres, of which, around two acres are formal lawns and flowerbeds, the remainder being park and woodland. Extending to 180 hectares they form one of the most important designed landscapes in Scotland.

The climate in Argyll, with its yearly average rainfall of 230cms (90 inches), is ideally suited to Rhododendrons and Azaleas, which flower in the gardens from April until June. Conifers also grow well in the poor acidic soil of a high rainfall area, as can be seen by the fine specimens such as Cedrus Deodars, Sequoiadendron Wellingtonia, Cryptomeria Japonica and Taxus Baccata.

The borders on each side of the central path, beyond the lawns, are known as the 'Flag-Borders' - the paths having been laid out in the shape of Scotland's National flag, the St. Andrew's Cross. These borders, outstanding in the spring with beautiful Prunus 'Ukon' and Prunus subhirtella, are underplanted with an interesting mixture of Rhododendrons, Eucrypyias, various shrubs and herbaceous plants, giving interest all year round.

The ‘Policies' (a Scots term for the improved grounds that surround a country house) date back to the 1600s.

At Inveraray there are three important avenues: the Lime Avenue radiating to the South West of the Castle, the Town Avenue and the Glen Shira Avenue. Other highlights are the ‘Watch Tower' at the top of the hill to the North of the Castle, the Doocot (Dovecote) which can be seen from the Avenue leading from the North West of the castle car park and Garden or Frew's Bridge.

Sections of the River Aray were canalised and cascades set in place in the 1700s to enhance the sound of flowing water and to provide the grounds with various points of interest.

As was the custom elsewhere during the latter part of the 19th century, distinguished people who visited the Castle were asked to plant trees. These included Queen Victoria, David Livingstone, William Gladstone, the Earl of Shaftesbury and others.