Not wanting to do the birthday, yes a lot depends on whether his reply was before or after your finance meeting, Lost Rabbits observations are correct. However whatever the truth, he’s said he doesn’t want a fuss, give him what he wants and don’t make one – a simple card sent to his home should be enough. He may decide that he wants to organize something with the kids on his own, that’s fine, just wish them a happy time, it’s not up to you to suggest it though.
As for the finances, surely what you should be considering is not what he wants but what is best for you? Think about that first, you need to protect yourself and the kids financially. Contact your solicitor and find out where you stand (if you haven’t already). If you decide to go the solicitor route then pass on their details to your H again, his solicitor can then deal with yours (costly though). Reiterate once again to your H that this is not your choice and whilst you don’t agree you will not stand in his way.