What is the best way to use Apache Solr for relational JOIN?
-
I have: * above 10 millions of documents * documents are indexed in Apache Solr * each document has ID which is related to his author * authors are in H2 database in memory - performance, many calls per second Then: * I make a search in Solr for specified query * get list of documents and based on authorID I make a call for author to H2 database I want to have: * I want to search also by authors names during whole search process How can I achieve that? What will be the best solution to search also for authors? I think about other model in Solr index to store authors, but this solution requires to use JOINs during search process, which will be slow and performance of whole application will be terrible. The problem is that this data is updated in real-time and it must be efficient. Authors could have a lot of related docments and could be updated as well, so put this as a field of documents is not the solution. Do you think that ElasticSearch will be better option to index such a large amount of data?
-
Answer:
Why not run a single-use script that does the joins on the SOLR index to replace the authorID with the author name, or add the field if you need both? Updating 10,000,000+ documents will take a while, but it's certainly doable. Then you can have new documents indexed with both fields as well, doing the JOIN one at a time. In general, SOLR works best with completely denormalized data. EDIT: Okay, if it's more than just authors' names, you could still denormalize the data, repeating your documents for each of the related author documents. However, that's an ugly solution, and SOLR 4 does have "JOIN" capability. You'd first need to at least have a pipeline where author data is indexed into SOLR in real-time. Then yes, I'd use ElasticSearch to easily handle the complex SOLR queries involving JOIN-like operations.
Scott Danzig at Quora Visit the source
Related Q & A:
- What is the best way to distribute an audio/video feed from a computer to TVs over existing indoor coax cable?Best solution by Audio-Video Production
- What is the best way to clean LEGO bricks?Best solution by bricks.stackexchange.com
- What is the best way to make UI for an Isometric game in Java?Best solution by Game Development
- What is the best way to calculate a date difference?Best solution by Stack Overflow
- What is the best way to count lines in file?Best solution by Stack Overflow
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.