Honestly for an all out rig I would do a 4 door Wrangler. That will probably be my next build and more of an expedition style with good on street manners. The frame makes things easier, the design of the body makes it less prone to getting wacked and easier to fix if it does. Tons of aftermarket stuff so its cheaper.
If I had to build a rig on $4000-parts only, I'd probably a TJ, its the cheapest on aftermarket parts and lots of used stuff.
$4000 including the rig, I'd do an XJ hands down. You can pick a decent running beater up for CHEAP, hack the fenders, budget boost, 33s, locker, winch, good to go. Drivetrain is near bulletproof under 33s, other than the D35 if you really like the skinny pedal.
Would I ever build another ZJ? Doubtful, although I do have a 5.9 that I daily drive that would make a cool expedition rig.