In fairness, I've never been there either. There are various parts of Britain which are subject to playful disdain... Wales, anything within 50 miles of Manchester, Liverpool or Birmingham, the West Coast of Scotland, everything North of Stirling, everything West of Bristol... and absolutely everything within the M25. That's the abridged version anyway.
No, really I think the actors were just unaccustomed to such long location shoots - and because of the nature of the show, most of the leg work is falling onto the two principles who will be in every episode... which is of course, why they did some Doctor lite episodes - although, while reading, apparently this was actually something that went on back in the old days too.
Anyway, you should have a good feel for Cardiff at least. Given that most of the filming is done there - a lot of London is actually Cardiff and of course, Torchwood actually doesn't pretend otherwise... whatever happened to Torchwood? Back in "Spring 2009", apparently.