It's commendable that you do not wish your BiL or SiL to pick sides, asking him to clarify what he said to you, I do not feel is asking him to pick a side.
We could all suppose and postulate here, but only he knows what he meant.
Experience is a brutal teacher, but you learn. My God, do you learn. - C.S. Lewis
Life is usually all about how you handle Plan B. - Jack3Beans
Listen without defending; Speak without offending - FaithinAK