How should I design my HBASE schema for this time series scenario?
-
We have a data stream that contains: time stamp, object ID, data The needed processing is: for each new entry, fetch all entries with the same object ID, and do something with all data The 1st option is to use a separate queuing service. In this case, the HBASE schema can include the object id as a simple key, as all queries are based on the object ID. The main drawback is the need to maintain an additional infrastructure. another option is to use a complex key of the form object ID.time stamp I would also add a 'processed' boolean flag in the value, to indicate whether this record was already processed. the queries by user id should remain fast. however, i'm not sure that querying by time range will also be fast in this case. Any other suggestions or refinements?
-
Answer:
Refer schema of opentsdb which has successfully solved the problem you mentioned above. You might need to modify its schema slightly and add processed flag as well. I hope this architecture solves your problem.
Jatinder Singh at Quora Visit the source
Related Q & A:
- How do I change the widget clock to central time?Best solution by Yahoo! Answers
- How can I design a workout program for myself?Best solution by Yahoo! Answers
- How should I design my Science Fair Board?Best solution by Yahoo! Answers
- How do I write a resume for a part-time job?Best solution by Yahoo! Answers
- How can I delete my all mails at a time?Best solution by Yahoo! Answers
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.