Ok, for the exhaust problem, I would suggest spraying a VERY generous amount of WD-40 or similar on all the bolts holding it down, letting it sit a couple of minutes, and then trying to take them off slowly.
For the other problems, I have had those. It sounds like you idle, fuel mixture, and choke are off.
Lets start with the choke. When its cold, take the air cleaner assembly off, and see what position the choke plate is in. It should be almost closed. If not that means your choke is off. To fix this, while it is cold, loosen the 3 screws holding that black knob on and turn it counterclockwise so it closes the choke. Also make sure that the one idle screw isn't contacting the cam that is in between the thermostatic spring housing and carburetor body. The screw will be on the drivers side of the car, with the head facing down at the manifold, and it will have a spring on it, just like the mixture screw on the passenger side. If the screw is contacting, then you will need to unscrew it a little bit, you can either do this with your hand, (well I could), or with a spanner.
Fixing the choke is pretty much the first step to fixing it, but I think you really do need to replace the manifold gasket. That would let excess air into the cylinder, and probably cause poor firing. I hope this helps some.
I recomend getting a 66 shop manual if you haven't got one. I am kind of new here too, and for all my questions, at least one person told me to get the shop manual. I did, and trust me it helps alot.
1966 Mustang Sprint 200 "Oliver"... factory fog lamps, factory rally pac, off the road for restoration.
2005 Mustang 4.0 litre... Airaid CAI, Honeycomb taillamp panel, more mods to follow.