Instead of removing objects, you can archive the object clicking the purple button with a ghost on it and label “make a ghost”.

You can see those objects again, clicking the similar button in the top right panel.