As of Version 1.0.3 upgrading a belt before a splitter will also upgrade any belts before the other side of the splitter.
It often makes sense to me that the other input of a splitter could be a "side-belt", though not always
Image link for clarity: https://imgur.com/a/dLJoUby
- Green light signifies where the belt was upgraded from using the tool

Behavior where expected is current:
- Upgrading a belt downstream of a splitter will propagate to both inputs of the splitter
- Upgrading a splitter will propagate to both inputs and the output

Behavior where expected is different from current:
- Expected
- Upgrading a belt upstream of a splitter will propagate to the output of a splitter but not the other upstream input
- Current
- Upgrading a belt upstream of a splitter will propagate to the output of a splitter and the other upstream input

Maybe both behaviors could be accessed much like side-belts currently do?
- Left-Click = Expected
- Shift + Left-Click = Current

I'm going to look into making this behavior a mod setting - so everyone can make it behave how they want, as there seems to be 2 trains of thought on splitters.

Also looking into having a feature to only upgrade "upstream" and "downstream" belts from where you click - stay tuned.