All my agri towers eventually have their input slots clogged up by stacks of 5 spoilage each or so.
I would appear that inserters refuse to extract from input slots as a rule.
I'm not sure how to handle this problem, nor how it could be addressed if seeds are meant to spoil.

Edit:
I just realized that the tower starts planting spoilage which eventually frees up the slots, is that right?

I do however have one tower that's deadlocked because my seed+fuel inserter prefers to insert seeds before the fuel, which it can't because all slots are filled by spoilage, even manually clearing the inserter hand just causes it to pick up more seeds.

You can use logic between the tower and inserter, tell the inserter not to move until spoilage and the seed is <1.

I suppose that will eventually drain any accumulated spoilage from the input from the tower "planting" it.

In case you're interested, I was curious if I could hack together a fix to this, not really meant to actually be used.
Since I believe inserters are hardcoded not never grab from input inventories:

-- Not sure this is actually correct...
---@param inserter LuaEntity
local function get_inserter_stack_size(inserter)
    local prototype = inserter.prototype
    local stack_size = prototype.inserter_stack_size_bonus + 1
    if prototype.uses_inserter_stack_size_bonus then
        if prototype.bulk then
            stack_size = stack_size + inserter.force.bulk_inserter_capacity_bonus
        else
            stack_size = stack_size + inserter.force.inserter_stack_size_bonus
        end
    end
    local override = inserter.inserter_stack_size_override
    if override > 0 and override < stack_size then
        stack_size = override
    end
    return stack_size
end

-- Fix for Foliax spoiled seeds accumulating in agri tower input slots
-- Periodically scan for any inserters extracting from any agri tower, and move spoilage from input slots to inserter when appropriate
-- this simulates the inserter actually grabbing from the tower input slots (in a pretty dodgy way)
-- if used for real, we'd likely want to cache the towers + the inserters to avoid scanning, but this isn't that easy
script.on_nth_tick(60, function(event)
    local surface = game.surfaces["foliax"]
    if surface == nil then return end

    for _,tower in pairs(surface.find_entities_filtered{name="foliax-burner-agricultural-tower", surface=surface}) do
        local area = {
            { tower.position.x-3.5, tower.position.y-3.5 },
            { tower.position.x+3.5, tower.position.y+3.5 },
        }
        for _,inserter in pairs(surface.find_entities_filtered{area=area, type="inserter"}) do
            --rendering.draw_circle{target=inserter, radius=0.5, surface=surface, color={0,1,0}, time_to_live=60}
            -- only inserters extracting from tower, currently does not alternate between inserters like game!
            if inserter.pickup_target == tower then
                local input = tower.get_inventory(defines.inventory.agricultural_tower_input)
                if input.get_item_count("spoilage") > 0 then
                    -- inserter idle and hand empty
                    if inserter.status == defines.entity_status.waiting_for_source_items and
                       inserter.held_stack.count == 0 then
                        local stack_size = get_inserter_stack_size(inserter)
                        -- move one spoilage stack into inserter
                        -- this is probably not the correct way to do this... ignores any spoil bar, and only does normal quality
                        -- use transfer_stack?
                        local count = input.remove({name="spoilage", count=stack_size})
                        inserter.held_stack.set_stack({name="spoilage", count=count})
                    end
                end
            end
        end
    end
end)

Simpler version:

local surface = game.surfaces["foliax"]
if surface == nil then return end

local towers = surface.find_entities_filtered{
    surface=surface,
    name="foliax-burner-agricultural-tower"
}
for _,tower in pairs(towers) do
    local input = tower.get_inventory(defines.inventory.agricultural_tower_input)
    if input.get_item_count("spoilage") > 0 then
        local area = { { tower.position.x-3.5, tower.position.y-3.5 },
                       { tower.position.x+3.5, tower.position.y+3.5 } }
        for _,inserter in pairs(surface.find_entities_filtered{area=area, type="inserter"}) do
            -- only inserters extracting from tower, currently does not alternate between inserters like game!
            if inserter.pickup_target == tower and
               inserter.status == defines.entity_status.waiting_for_source_items and
               inserter.held_stack.count == 0 then
                -- Seems like transfer_stack handles inserter stack size automatically
                -- this should also correctly transfer the spoil bar etc.
                local stack = input.find_item_stack("spoilage")
                if stack then
                    inserter.held_stack.transfer_stack(stack)
                end
            end
        end
    end
end

If you want to create a mod with runtime script, go for it. I believe its easier to just use combinators.