The LTN works great. It's getting the correct building materials to the provider station's buffer chests that's the trouble.
https://factorioprints.com/view/-MNZWdWosuqr3vtaC2hD
The solution I settled on was to use the Network ID. The construction network is 4096 and all construction sites set that ID. I configured the combinators to monitor 4096+2048=6144. The providers have a constant network ID of 2048 and they add that again when it has determined it has sufficient inventory for a train to be called, thus matching the requesters and completing the circuit.
Now each provider can see what is being requested and the total being readied at all providers yet only enable the train when it wants. This prevents trains coming to take away only a few items (because my thresholds are just 1).
I have some clean up to do with non-construction sites that didn't set a network ID but I can mostly ignore those with some additional constants to cancel-out those specific requests. Only brick is a problem since it's the only construction item that is also requested in bulk.