Inveraray is located on the shores of Loch Fyne, where it links with Loch Shira. The village, as it stands today, traces its origins back to the mid 1700s, when the 3rd Duke Of Argyll demolished an earlier small fishing village to accommodate his newly rebuilt castle (Inverary Castle). The original village of Inveraray was made a burgh of barony in 1472, and subsequently made into a royal burgh in 1648. Due to its location, lying at a key point in the limited road network, and with a good harbour, Inverary had become the centre of the county by the 18th century. The 3rd Duke of Argyll inherited the castle when he succeeded to his title in 1743, but found the castle to be in ruins and uninhabitable. His response was to rebuild the castle and move the entire town further to the south.

Inveraray Castle is the reason many visitors arrive in the village each year, it is as impressive inside as it is outside, has a huge collection of historical weaponry and many fine antiques and furniture. Inverary is also a very attractive village, built originally along two streets, Front Street and Main Street, which are lined with large white-harled buildings. In the centre is Church Square, which accommodates Glenaray and Inveraray Parish Church, built between 1792 and 1802 by Robert Mylne.

Those with historical interests may also enjoy visiting Inveraray Jail, which also dates back to the redevelopement implemented by the 3rd Duke. It is open daily throughout the year and comprises of a Georgian courthouse and prison blocks. The jail closed in the 1930s but in recent times has re-opened as a visitor centre and museum. In the museum prison conditions from medieval times up to the 1800s are described using interactive displays and live performances by actors.

The Inveraray Maritime museum is located at the town pier and is comprised of two moored vessels: 'The Arctic Penguin', a triple-masted schooner built in Dublin in 1911 and the steamer' The Vital Spark'. This is also open to the public all year.