I was once told never to put the filler directly on the metal but maybe thats for patching. it was a moisture issue I believe and that was a while ago. With how close you are It might not be a big deal since im guessing there wont be too much of it left on the vehicle.
People have run experiments proving that filler over metal has better adhesion, than over epoxy primer. Also, if you epoxy primer, then use filler, when you sand the filler, you get into bare metal again while feathering, before your second application of filler. So it can be a exercise of chasing your tail.
If I suspect any pinholes in my welds(which is rare with TIG), or it's in heavy weathered areas like the bottoms of doors or fenders, I epoxy front and back before the filler. That seals the metal.
Having done a lot of repairs on cars restored in the '70's and '80's, when you remove the filler, you 'may' see surface rust under the filler. The story goes that the heat from the cure of the Bondo causes condensation behind it. In many thick repairs, you may even find uncured filler, or moisture penetration from bad welds.
Back in the day, they used to put it on thick, as in 1" sometimes, and fast at that.
With the level of metal finishing and the light years in technology that body filler has come, you really shouldn't be putting it on thicker than 1/8". Also, the filler AND metal must be acclimated at certain working temps. It's improper to fire up a wood burner in a 30° shop and two hours later start body work. The metal is still cold. That's why you get deformation and the body work "changes" after a couple of summers out in the sun.
There is a lot of controversy on filler methods. It's better than primer, faster than primer, and cheaper than primer and most of it ends up in the floor