That's interesting. We lived in Somerset for a good while before leaving the UK, and people (more so from from the southern counties), would often say that they could hear a bit of West Country in my accent, which they thought was vaguely 'midlands'. When I speak with Brummies, I notice how brumm they sound at first, then their accent becomes more 'normal' as conversation proceeds, as my ear tunes back to how it originally learnt to listen. I sometimes get "I didn't realise you were from Birmingham" as well, so my accent is probably reverting just as in your comment.
I have also noticed that when people try to imitate a Brummie accent for a sentence or two, particularly if I say I am from Birmingham and they feel a need to 'bond', it is more like Black Country that comes out.
Andrew.