Once used language that was a bit too colloquial in a formal paper and had it handed back to me. It was a linguistics paper, and the "errors" were marked by a linguist who knew that colloquial English word choice and syntax was every bit as "grammatical" as formal English. However, he pointed out, it's still a fact that you will be judged on how you speak and that this judgment is both widespread and important. So I fixed the paper. He was right.
Mu personal judgment was that it didn't matter. My personal judgment, in the grand scheme of things, counts for so little as to be best described as "utterly unimportant."
What's considered affected and not is a purely social matter and depends on a lot of factors such as age and geography. If you're a 25-year-old Whovian you're probably going to pick up some Britishisms and its not necessarily affected any more than having an East Ender pick up Americanisms from US tv shows or a Parisian using English words would be. If you're a 40-something professional who wants to show how urbane you are by saying "brilliant," or like my aunt who insisted on using certain low-frequency words because they were trendy several social levels above her, then (in my humble judgment) they're affected.
Then again, among the 40-something's social set such words may be simply "de rigueur" and not affected because, well, everybody's "normal" in their own judgment. It's always others that are somehow high-falutin' or slumming.
Of course, it depends on the word. Some can be considered affected. Some can't be.
I've had to repeatedly catch myself before I say something's a "cock up" in my classroom simply because I've watched so much Red Dwarf and have occasional Red Dwarf marathons. (My son, never out of the US and who has never lived in an area without rhotic vowels, lacks rhotic vowels entirely. Too much British TV in our household.)