Cats naturally like to dig holes to soil in so they can cover it and hide the scent. Maybe you can raise your flower pots on plant stands or hang them from the ceiling. Most cats do not like the scent of citrus, so try spraying some citrus scented air fresheners around your plant pots to deter the cats from going near them. Maybe provide some live catnip plants for your cat that he is allowed to have, and he may become more interested in the catnip plant and leave the other plants alone.
Pine cones or stone (not pebbles something with irregular edges) around the base ot the plant keeps my cats out. They only get into the plants where the dirt is exposed, so try planting smaller plants at the base of the plant or place smaller pots of plants to compliment the one already in the pot. If they can get in they will so barricade them out.
Send your cat to Mars. That'll stop that sort of behaviour.
Put crinkle paper in there. It makes noise when they step on it and there are many different colors of it. You can get it at the craft store. I use it and it works great
Put a few Giant Cacti around the house plants...they'll learn your lesson..
Scold him/ her loudly and if you have a closed backyard, tiled laundry room, etc., stick him/ her in there.