I think it would depend on the purpose of the visit. If just a quick errand, then probably no bag, but much more likely to take the bag if it was a social visit and you were going in for a drink and conversation.
On a social visit yes, I would definitely take my bag.