First, why do you care how other people feel about you? If they think badly of you, then they don't KNOW you and don't matter.
I would tell him you will have dinner ready at a certain time and he can decide for himself if he wants to be there. A boundary is: I will make dinner. You are welcome to have some. I will not be responsible for waiting for you or keeping it ready for you without prior notice that you will be late or not coming.
Then, enjoy your dinner with your D.
Another boundary is: I have people I want to spend time with. I will help you with your laundry and billing but I will expect something in return. I, as well as you, need time for myself. You decide.
You can still be pleasant and cordial but you have to respect yourself before you can expect others to. Especially, someone who is acting so selfishly.
Just my opinion.
M:63 H:53 S:41, SS:28, SS:25, SD:23 M:15 T:16
Bomb:12/17/11, "I think we should go our separate ways." H moves to his mother's house, 4/1/12 12/21/12: H moves back home, piecing