Umm...when is your down time? As coworkers you had better set a schedule. This is such a big part of your life I think you may need a boundary or two. If you were working elsewhere ask yourself if you would be good with this. If so, great, but if not then you have to figure it out.