No information regarding mods. Is this a stock fuel pump?
Confirm that the fuel pump is compatible with 99-04 model year. It must be able to works with a FPDM and variable voltage.
Confirm the battery terminals are clean and tight. Confirm the frame grounds around the radiator core support are clean and tight. Confirm the ground in the trunk is clean and tight.
Have the alternator tested for excessive AC ripple (bad diode). Almost any auto parts store will test on the car for free.
If this were my car, I would CONFIRM the voltage drop at either the fuel pump fuse or the trunk mounted IFS switch. Once the voltage drop has been confirmed, then additional trouble shooting will be needed to figure out WHY the voltage is dropping.
My gut tells me this is a battery, alternator, battery cable problem and the charging system voltage is dropping to the entire car.
Bottom line. Today's cars simply will NOT run right without a strong battery and charging system. Every electrical problem should start with a review of the battery and charging system. Cut corners here at your own risk!
2003 GT Convertible (sold & missed)
2000 GT coupe (Craigslist project. Fixed. Now my DD). Windsor to Romeo swap.