How to Convert a C++ Structure into C# Structure?

I want to convert file into some other format (structure) before storing inside HDFS. Can I manage it using Hadoop or inside hadoop? (PS: File is around 500 MB and file should be present at one location to convert it to other format.)

• I have huge amount of data and i would like to perform batch analysis using Hadoop. Only problem is raw data file size is greater or equals to 500 MB and one file should be present at one place to perform analysis. Although i have a conversion method. After converting the file into this format i can perform batch analysis on each batch (128 Mb blocks or 64 MB blocks). There is no need that the file should be present at one place. What i want to know is where should i do this conversion?

• Answer:

  So you have a file that is not in HDFS in some format, and you want to store it in HDFS but in a different format. If the file is not available in any form that Hadoop can "see", like in HBase or something, then you can't use things like MapReduce to do the work. If you really mean you just have one file, it's not worth using Hadoop at all. I assume there are many. You could copy it into HDFS, and then use some process like MapReduce to convert it. This is the usual Hadoop answer. Store the raw data in its original form, then transform. If you have an existing process that can convert the file on the command line, you can use something like this to transform as you copy: your-converter < your-raw-data.bin | hdfs dfs -put - /your/converted/file.bin