If you h isn't home by a certain hour, go on about your business and do whatever you need to do. Don't hold dinner, put it away and find something to do. He's a grown man and knows what he needs to be doing and if he's not following through, then it's his loss.
As for the people coming to your home, I wouldn't like it, but they are trying to warn you of what he's been doing and what the consequences may be. I don't think they are crazy at all, just trying to let you know. The crazy one is your h and look what he's brought down around you, i.e., people coming to your home to tell you about the affair, etc. I'm glad you told him about the visits and you should have advise him that you do not want people coming to your home to tell you about his behavior on the job.
As for divorcing you, he doesn't know which end is up. I do think that he's going to be facing his consequences for his actions at work sooner than anyone thinks. So, prepare yourself in case he gets fired.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.
