Albany in WA & Robe in SA would come close to Portland VIC for the title of having the coolest summer temps in the mainland outside of the Alps.
Of course all of those places front the Southern Ocean/Great Australian Bight, where most of the cool air comes from.