You might have to try testing again with the latest latest version of RSC (v2.3.1), that fixes the rescaling calculations for the Outer Rim + RSCompat case, at the very least
Is there any chance that this factor can be applied based on radial distance from the sun?
I don't think that is something for RSC to do, since those should be optional fields that can be defined by other mods. But you mean, the farther out from the sun, the higher their redrawn_connections_length_multiplier should be? I can play around with that and see, though this is likely to cause longer routes in general (especially with something like Asteroid Belt, which pushes a lot of stuff even further out)
That being said, I'm not sure I want to make any changes to these values just yet, not until the planet mod authors or Outer Rim dev have a chance to look at this from their ends (anything I do might override their changes)
EDIT: I forget which version I added this, but there is some code in place to change the route length multiplier, but it's disabled (commented out the require line in data-updates). Should be safe to re-enable for testing, but I'm not sure it would help in the Outer Rim scenario (due to the weird parenting issues involved)
It basically sets the route multiplier to 1 for any planets with a shorter distance to the sun than nauvis, scaling up to 1.25 for any planets with a shorter distance to the sun than fulgora, and then scaling up again to 2 for any planets with a shorter distance to the sun than the solar system edge; skipping any planets that already had this value defined, or any planets that lie beyond the SSEdge (those should probably be handled manually on a case-by-case basis). SSEdge itself gets a value of 4.85 (which should keep the aquilo-ssedge distance to 100kkm, as it was before)
But for now, I'm going to mark the main reported issue is as Not a Bug, in case Outer Rim fixes it from their end