This is just my opinion but I think often times men say things like "whatever you want is fine by me" to be agreeable and easy going. And there is nothing wrong with that as long as it is not all the time.
If I said to my H... I am craving pizza and he responded with "well, I am not craving pizza but if you are that is fine by me". IMO that is being flexible. There has to be a limit though. When I did try and broach the subject with my H about how I never felt like he never contributed to any decisions he would get angry and tell me that the reason he always said he didn't care what we did or where we went is he didn't care as long as we were together. That sounds nice but it is also nice to not have to make ALL the decisions (and this goes both ways of course).
I think sometimes one spouse thinks they are being easy going and agreeable by saying "anything is fine by me" without realizing the amount of pressure it puts on the other spouse when in fact all the "easy going" spouse is doing is trying to make the other happy.
Looking back I can see why my H became so defensive when I told him how I didn't like how I had to make all the decisions. In his mind he was being agreeable, easy going and flexible so why is that so bad? It's not "bad" per say it just made me feel like I was part of a one man team. I suppose I did resent him in a way and I started doing things without his input. Not super major things but I eventually got tired of "I don't care, whatever you want" so just took some matters in my own hands.
Funny how things become so clarified once you really step back.