Find solution
What web scraping tool is the best to extract data?

Which ETL tools support analysis of pickle'd Python blobs in a database?

  • I currently do some ETL research for  my company. Using python as web framework language, lots of data is stored as blob type, which is the result of pickle function. Now I want to extract data from pickle field. So with which ETL tool I can easily extract data from python pickle?

  • Answer:

    In general, unpickling an object requires that you have the necessary class around in order to instantiate it. This isn't a problem if the original object was a built-in type, but if it was a custom class you may have to do some digging into the original application's code. In case this isn't already clear, an ETL tool is not really going to do you any good. However, once you have unpickled the objects, you can write them out to another format, such as a CSV file, which would be amenable to an ETL tool.

Michael R. Bernstein at Quora Visit the source

Was this solution helpful to you?

Other answers

Try this, it extracts the operations defined in the pickle without loading any external references. For me, the arguments to the operations were of value and I did not have all the referenced modules and classes around to load it the standard way. import pickletools with open(pickle_file_path, 'r') as f: items = pickletools.genops(f) data = [(op, key, val) for op, key, val in items]

Grischa Meyer

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.