My H had to rescue his wedding pictures to save for the kids from the garbage can when his XW threw them out. I definitely think he did the right thing. He also rescued treasured baby toys of his kids, etc... Good thing he saw the trash that day.
I still have stuff from my wedding with XH even though we had no children, but it is definitely put away. Dunno what to think about that really, sometimes it bothers me to have old stuff around. But it seems important to preserve that for my own sake.
The stuff that was definitely XH's I returned to him -- like your XH's sweater, for instance. That is his to preserve or not.
Definitely dump the mundane reminders. New curtains, change the furniture, etc...
I really think the WASs get the better end of the deal. They leave, take a few things that are important, and the LBSs get stuck with the lifetime of stuff. Largely dumping it and starting fresh is the way to go.
If it is something you wouldn't be comfortable sharing with a new love, it's probably a problem...
If it is something that keeps the past in the present, lose it or pack it away. It's weird what these things are. Some old furniture of mine doesn't reek of the past, it is simply furniture. Other pieces, I'm better off dumping.
I guess, briefly, don't keep things in your life that bring you down. Some people need to toss, some don't and can merely pack things away. It probably depends what it is.
But no thing is worth saving that causes constant emotional stabs.
Also, there is stuff and there is stuff. The stuff that wouldn't be on your short list of things to save in a fire should probably go. For instance, in my mind Christmas ornaments aren't that important. You'll never use them all again. They would be intrusive in any future R. Let your children pick a few to make their OWN, drop the rest at an ebay shop.
That's it! Take the old stuff to an ebay shop, use the proceeds for new dreams :-)