Apart from my required physics classes, should I take higher level physics classes or math classes?

The answer that I have gotten from a lot my mentors (I have the same aspirations as you do of becoming a theoretical physicist) has always been to take more math, if only for the reason that you will always have time to study graduate-level physics in physics graduate school; most likely, however, you will not be taking math classes in graduate school, and therefore should learn as much of it as you can as an undergraduate.

Math Always pick math. Physics is applied math and you will always need math to understand physics

All of the other answers to this question recommend you take more math. That is very reasonable and you will certainly need as much math as you can study. However, there are a couple of good reasons to take the grad level physics courses instead. First, they may be more interesting than the math. Second, if you can eliminate the required graduate physics courses you will be way ahead once you are in grad school (no physics grad student wants to take the basic grad courses in mechanics, E and M, subatomic, etc. - it's more about the seminars and thesis work.). Ultimately, you should ask yourself whether you can learn the math on your own, or some of the physics. It may be that, pushed a bit by the physics courses, you can learn the math from the excellent texts and treatises available.

If you're going to study particle physics then learning about lie groups and lie algebras will help you there.  Having an deep understanding of linear algebra will help you with quantum mechanics.  A course on discrete groups or number theory will be mostly a waste of time for physicists.  Understanding real analysis is useful in many fields.  Knowing how to do a mathematical proof is very useful, although you'll go insane when you get to quantum field theory because it is not yet understood well enough to know whether it is even consistent.