The critters you put in the dig site walk out to random spots in the harvesting area. This means in order to have any decent mining speed you need to have all available ore close to the dig site. Counter-intuitively this means the optimal placement is to put the dig site on the ore facing outwards to cover as little of the ore as possible, rather than turning it towards the ore area.
Either the harvesting area should be much smaller so that the critters harvest closer to the dig site, or the AI on the critters should prefer ore closer to the dig site.

Hello, I am just another player of Py mods. Interesting observation. I agree. It does seem odd to have a drawback to maximizing coverage. Maybe another possible solution for a future update is to have the critters auto adjust their walking speed so it always takes the same amount of time to get to their mining spot. One thing that helps for now (if you weren't already aware) is that the workers are affected by walking tiles! Oddly walking tiles affect the mining workers and not the caravans, and it seems weird to pave over ore, so I would also like to see a better (and more consistent) solution.

Have you tried fences?