Figuring out what "System Data" is and how to reduce it on Android 4.0.4

It is really unclear what you mean by memory. I'm assuming you mean solid memory chips like an sdcard and not RAM. Basically you're phone has a removable chip and a built in chip. The built in chip will have slightly faster processing speeds that won't be noticeable unless you are measuring due to literally being closer to the processor and other complicated factors. That's why the OS is always on that chip. If you're concerned about space purchase a larger chip for your internal removable chip. Many things could show your phone down that are not related to the physical amount of data. Number of processes open (don't run all apps at once). The speed of the processer. The operating system itself. Graphics add ons. The system data is generally important stuff. It's hard to differentiate. You may want to consider backing up data and doing a factory reset.

My phone is a Verizon DNA Droid (a version of the http://en.wikipedia.org/wiki/HTC_Butterfly) and this memory is listed as "Other". It's also 4.57 GB because the phone maintains a backup copy. That makes it possible to do Over-The-Air firmware updates, which has happened three times since I got the phone. (One of those was an upgrade to Android 4.4.2.)