Only if all fuel inventory slots are filtered it does mean "nothing else". It's not about removing things from the list afterwards, it's more about controlled addition.
I guess right now you have something like this:
if setting says to add fuel
add all fuel (and spoiled items)
instead you'll need
if entity has at least one unfiltered fuel slot
add all fuel (and spoiled items) from the allowed fuel categories
else
add all fuel (and spoiled items) from the filters
I'm not certain whether similar logic should apply if the fuel inventory slots are not filtered and instead have fuel in them. Because fuel rarely changes, and if it does it's likely a mixed belt of input and you don't filter inserters for that. I really like the sorting idea, for that scenario I'd definitely put all the additional fuels at the very bottom of the list.