If it is what I am thinking of, there is a grey "lever" on that connector that you flip up (or was it down...) . If not, then removing the screws worked.... I had a hell of a time before I found that lever when I did my V6 stang.
I don't quite feel like replacing my door speakers and subs for my shaker 500. I had JL's in my 06 V6 and they are so sweet. I have the head, amps but need the speakers as they went with my 06. That was STUPID.