Do I need to dig out all the really old dirt in my brick patio planters?

I just moved into my Mom's townhouse, which has a patio with some nice but so-far neglected brick planter boxes. The old dirt has been sitting in there for who knows how long. She bought the place 5 years ago, so the dirt is at least that old. Questions: would I need/want to completley dig out all the old dirt and replace it with fresh/new? Should I add a layer of say pea gravel for drainage at the bottom? I'm fairly sure there is bare dirt and not patio concrete at the bottom of these things. It would be a pretty substanial amount of dirt for one average person to dig up & replace by hand; 2 cubic yards worth. Could I get away with digging half the old dirt up instead? Should I mulch as well? Live in Zone 8/South Cenral Texas. Advice please :)