Way back in the good ol' days when I had my YJ on the road, I installed a 2.5" lift with 31's. I built a trackbar braket similar to the one RE lifts have. I removed the rear TB and built sway-bar discos that were 2.5" longer. I didn't like the ride as it was a bit bumpy, and it had a bit of bumpsteer as well. For some reason I noticed my bumpsteer was no longer there one day and after investigating, I found that the bolt that held the braket to the diff had come out for some unknown reason. I decided to remove the tracbar as it wasn't doing anything and the bumpsteer was gone. I didn't notice any bad drivability issues without it either. After a few wheeling trips and disconnecting and reconnectinng the sway bar, I got lazy and neglected to reconnect. What a differenc in ride. It was way smooter. It did however have a bit of noticeable body roll without it, but nothing scary.
My advise is to ditch both TBs and the SB, and just drive accordingly. It's a Jeep, not a sports car, and it will never ride like a Caddy.
Just my $0.02