Instead of clicking the Remove All Unused Media From Project button, go into the bin you want to clean out and select just the files you want to remove. Then, right-click one of the selected files and choose Remove From Project from the context menu. If I understand you correctly, that does what you want and only cleans out the selected files. Do this in Details view so you can easily see the files with a 0 use count. It's not quite as automatic as just hitting the remove all button, but I think it does what you're after.
Lightning Bolt? I went and looked and wondered what lightning bolt.
But then I tried placing the mouse pointer over each and every one of the buttons and read what they do from the pop-up help, and I found the correct button. It is the leftmost one and looks like a list with a red x at the bottom.
Anyway, thanks, as I really had no idea this option existed.