Offshore oil can be extremely rare on maps without large oceans. It only generates when the default oil autoplacer tries to make oil in a 16x16 chunk that is entirely water. Also, chunks generate when pollution reaches them, not just when you explore them, so you might not have reached any actual new chunks yet.
To completely regenerate all the oil deposits in the game, including in chunks that have already generated, run the /regenerate-oil command. This resets all deposits to their starting richness too.