Hello.
This isn't going to be a problem with the starter itself. In order for the starter to do anything, it has to be getting power from the starter solenoid. When the key is not in the start position, the starter isn't supposed to be getting any power. If it is getting power, then some other component is responsible for that. Probably either the starter solenoid, or the ignition switch. 