You can replace the t-stat housing as a preventative maintenance item, but since the leak is coming from behind the newly-installed water pump, I'd say take the pump off and re-install it. (Get a good look at the block while the pump is off, too.)
Make sure that all the old gasket material is removed and clean all threads. You want clean, flat mating surfaces and evenly torqued bolts. If any of the threads are dirty, it will screw with your torque readings during assembly and some of them will be too loose.
1995 Mustang GT (5.0), daily driver
1995 GMC C2500 (5.7), alternate daily driver