YES,YES,YES,YES,and YES!!!!!Read through this thread, it pretty much covers the basics and will get you started. http://www.allfordmustangs.com/foru...mods-do-first-1999-2004-gts-final-thread.html
If it were my car, the items on my to-do list at this time would be....
1) Off road exhaust midpipe. Doesn't really matter power wise whether you go with an "H" or "X" configuration. And don't throw out your factory catted midpipe, these have a way of coming in handy in the future.
2) Rearend gears. At the risk of sounding ignorant, just get 4.10's or 4.30's and call it a day. This will be about the best entry level mod you can do.
3) Short throw shifter if your car has manual transmission.
4) Suspension stuff, like subframe connectors and lower control arms. These two items alone will help you get power to the ground a little better and make your car's handling feel much more confident.
5) PI Swap. Basically this is swapping your current heads and intake with that from a 99-04 GT. This is going to be a mandatory operation if you are truly on a quest for power.