My H would accuse me of that also, but just like Vinlad says "it's his problem" I would relax about it, next time H brings it up say something like "I'm sorry you feel that way" and leave at that, end of conversation. Don't explain, don't try to make H see something he cannot see or will not see. YOU know it's not true, so that is the important thing..IMHO.
