I would imagine that he is in hiding while your mum is here, he must realistically have no idea what to do or say or how to behave while she is around. I called H weeks ago and him mum who I always got on really well with answered and my heart sunk I didn't know how to act or talk. So quickly mumbled a message for him said thanks and got off the line. She also was very weird not talking and chatty like she had been, I guess she like me didn't know how to react either.
I can see him running a mile from the dinner invite - I know I would - I would make up so many urgent things I had to do just so I wouldn't be in what I feel would be uncomfortable situation.
Besides you mum doesn't think it a good idea - and if it happened by chance that you managed to get him to dinner it could all collapse really fast - just with interactions between H and mum.
I wouldn't do, but then you know them both better than anyone.