Is this an auto or manual?
If it sat for a long time then the fuel filter and/or injectors may be clogged. If you spray some carb cleaner in the air intake and crank it over and it starts for a second or two then you know that it's a fuel issue. Since your anti-theft light is on your PATS(passive anti-theft system) may be cutting fuel injector ground pulse to the injectors. In general, when the anti-theft light stays on continuously solid it usually means that there's a fault in the PATS module(in the instrument cluster in this model/year 1999-04) and there's a code registered in the PCM. Those codes will start with a B and will be considered "body" codes. Get a scanner that can read body codes and post the code or codes that you get here.
Talked about here:
(197) 2002 GT - Theft light on continuously but car runs | Ford Mustang Forum (allfordmustangs.com)
The other potential here is that the PATS module requires a certain amount of voltage at startup and if it doesn't get it then it'll either store a code and keep the light on or it'll assume the system has been tampered with and to prevent potential theft it'll shut down the starting system and/or it'll shut down the fuel injector pulse depending on the year.
Your battery should be checked to make sure that it holds a good charge after sitting for many hours. It should have no less than 12.5v. If there's an internal short or dead cell in the battery then it'll result in less voltage for the vehicle's modules. You should also check all your grounds as well. Loosen the bolts that fasten them and retighten them.
To understand it you have a basic fuel or ignition problem you need to put a noid light or test light on a fuel injector connector to see if it pulses when you crank it over. If it doesn't pulse then your PCM isn't sending any ground signal to the injectors. The PCM fires the injectors by completing the ground circuit to each one. There should always be constant 12v power to them when the ignition is in the on position. When the engine is cranking over or running the PCM switches the individual grounds on for a split second to fire each injector.
The key getting stuck in the ignition may be from the shifter lockout switch under the shifter if this is an automatic. It could also perhaps be from a bad brake light switch if you have no brake lights. The ignition cylinder may need some lock cylinder lube sprayed in it or the ignition switch in the column could also be a problem.
This is talked about here:
(197) 2000 GT automatic parking issue | Ford Mustang Forum (allfordmustangs.com)
The battery in the key fob is only there to lock/unlock the doors. It has nothing to do with the ignition. The PATS transceiver ring energizes the coil in the key when it's put into or near the ignition cylinder so the key requires no power in it to start the car.
First thing to do is buy or borrow a good capable scanner to read body codes. I would also buy a test light in order to do the fuel injector electrical test. A multimeter is needed as well to test your system voltage.