How can I find trends and patterns in all of my data? I have a DMP and tons of customer data but don't know how to use it.

Like and , I read this as a possible job posting. Being an idea man, let's assume that you know what business objectives you want to achieve using the data. Now, how can you find a data scientist to answer the business questions using your data? Here are three possible ways to find such individuals: a. You can browse through the "Data Mining", "Machine Learning" and "Data Science" categories in Quora and send messages to people writing answers you like. b. You can search for these skills in LinkedIn (Just type the skill terms, and LI should suggest searching for the skill). c. Use freelancer search sites: please see http://epiclaunch.com/employees-for-your-startup/ This blog post suggests , and for finding experts. I believe that in your case hiring an expert should give you quicker results.

Forget about the data for a second and think about some really clear objectives you would like to achieve. The marketing piece is one of them, that's a great start but get super specific, what exactly do you want to do. Do you want to find people like your existing customers? Do you want to find out if your customers share certain traits? Do you want to find out which customers are likely to return vs one-time customers? Do you want to find out which customers will pay vs which might default? These are all really good questions that you can answer with your data. As for what you can achieve, this is a classic "hire or learn" situation. My guess is that as a business owner it's worth it to you to hire someone else to work on this rather than to spend the hours it would take to do meaningful analysis and be able to vet model performance appropriately unless you already have some good background on the topic. There are great books and online classes and educational programs on the topic, but you need to figure out if you want to invest the time and money there or invest the money in finding a data scientist to help you. For the record, if you have all of the above data in a fairly clean digital format and you have a good amount of it then yes, you can absolutely build predictive models that should achieve progress on the above projects.

There isn't going to be some magic fairy that'll come and tell you these are the patterns that exist in your data. Your question is not very specific, it is almost like, "I have a lot of stock data, how do I find patterns in this?" There are plenty of techniques, that can help identify and investigate the data. But none can promise to find the pattern for you. You are an idea man, you can definitely come up with a lot of common sense hypothesis. For example, you have a feeling that the poor people are more likely to respond to loan ads. Now, even though this may seem very logical to you, data science will help you to validate this. Thus you will end up relying more on scientific data to make decisions rather than your creativity. I cannot answer more, because I am not familiar with your business or with the tools you have mentioned. I have a good understanding of ML though. To give you an idea, there are largely 2 broad categories of prediction tasks here. 1) Classification: Give me all the information you have, and I will tell you whether to make decision 1 or decision 2 (eg. customer will default on loan or not) 2) Regression: Give me all information you have, and I will predict a continuous target function.(eg what is going to be the stock price of XYZ tomorrow) Your pattern analysis largely will fit in one of the above two categories. Think of what sort of information you are trying to extract from your data, and what sort of information you think you may find useful  to make a business decision. Please comment if you would need more assistance from me.

Reading your description I understand that we deal with large datasets. The format of the data varies so we have to deal with numerical and categorical data. If you dont know what you are looking for, the path you could follow is called unsupervised learning. Under this umbrella there are a lot of techniques to extract the most important features out of your data (feature extraction). Then, it is clustering. Again, a bunch of techniques to choose from. What clustering does is to find 'similar' in a way, structures in the data. Let me give you an example. Having a customer dataset with each row being an individual customer and the columns being the attributes of those eg charecteristics of each one that you have collected. Feature extraction will reduce those attributes by compressing or summarizing, if you like, information in less attributes. Then clustering can efficiently reveal natural groupings in your data. Since your question is vague, my answer is vague as well. Still, the approach should work.

You can try some existing solutions to help you with initial research. The ones I can think of right away are: http://www.tableausoftware.com/. http://www.contextrelevant.com/. (Not affiliated, but heard of them since they have presence in Greater Seattle Area). Then, depending on how technical you'd like to get, http://www.cloudera.com/content/cloudera/en/home.html offers quite a few possibilities to start working with technologies as complex as Hadoop without investing much time into setting them up. And, yes, think about hiring consultants. Initial data research is a task which is almost ideal to outsource to consultants. It would not take much time and money on one hand, and they would live to impress you with their discoveries on another. (Here I can help with connections.) In fact, if I were you and would have data, budget and no time/desire to look into the data myself, I would even go as far as hiring multiple different consulting companies independently and picking the one that finds more interesting facts in the data :-) Good luck! Dima