What out-of-core dimension reduction algorithms, available in python, should I use to help visualize big data?
-
My Challenge: to process Cluster Million of documents in unsupervised algorithm. And visualize them . Entering each cluster , I want to Visualize Centroids and documents. My Weapons: scikit-learn , and d3 for HTML5 visualization, and I use HashingVectorizer + Minibatch Kmeans to solve memory problem for clustering, as explained here http://scikit-learn.org/dev/auto_examples/applications/plot_out_of_core_classification.html. RandomizedPCA is senseless to run on Chunked data. Is there any out-of-core Dimension Reduction Algorithm in python that can be applied on scipy sparse matrices ?. I am looking for a Python based solution - so not interested in incorporating external data structures, such as Hadoop.
-
Answer:
Would any (not "out of core" implemented) techniques help? (I'm interested in it myself)
Dan Ofer at Quora Visit the source
Related Q & A:
- How can I use real time social data from Datasift and perform real time analytics on it?Best solution by Quora
- How do I use WordNet in Python?Best solution by Stack Overflow
- I can't see what people are typing in the chat rooms ever since I was booted? Help.Best solution by answers.yahoo.com
- What country should I do my foreign exchange in and what program should I use?Best solution by answers.yahoo.com
- What products can I use if I have facial eczema?Best solution by everydayhealth.com
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.