The side you are after is "fideltiy"... You aren't asking them to enter a war with your husband. You jsut want them to agree that infidelity is harming the family and you want them to invite him to stop.
Side with the marriage rather than with its covert disolution by an interloper... plain and simple