There is an energy associated with the action of the charge on itself. In classical physics this energy explodes to infinity if the size of the charge shrinks to zero.
Feynman, as a young student, played with the idea that elementary charges do not act act on themselves, and that fields in between the charges were not real. This was an action-at-a-distance model. They fly in Feyman's ointment was that Lorentz had shown that the action of the charge on itself explains radiation resistance.