First, check the fluid level for sure. If that is ok then it sounds like you have a either a sticky shift solenoid or a leaky 2-3 shift accumulator, both of which are easy to replace and relatively cheap.
Because you say it is jerking on the 3-2 downshift along with late/harsh 2-3 upshifts, I would lean toward it being a sticky solenoid, more specifically it being SSB.
In order to change the shift solenoids SSA and SSB you have to drop the pan and valve body. While you are there you might as well change the 2-3 shift accumulator, it is cheap and easy to change. The shift solenoids SSA and SSB are sold in pairs so you will be changing both at the same time.
2013 GT Brembo Black, 3:73, 6-Speed, GT500's, Urethan Shift Bushing, 8 Ball Shifter, J&M LCA's & PHB, Eibach Pro springs, SR Performance F/R Adj Sway bars, KONI Orange Struts & Shocks, SCT S4/XF4 tuner w/BAMA tunes.