There is no reason to take them down unless they make you feel uncomfortable. This is your office space and you should be able to do whatever you want in that particular space. Personally, I would leave them up.
The "office" is in my (the marital) bedroom, which is where our daughter gets ready in the morning and evening. So, it's not a separate room, and is something she sees everyday.
