How does one go from making college-level projects to making massive industry-grade products?
-
Summary of the question: How to go about gaining advanced technical know-how in Computer Science, to be able to make high-quality software like Google,Amazon, Quora, Wikipedia etc - software that are used by people worldwide, that are reliable , secure ,efficient and can scale easily. Sub-question(s): 1. How to go from being an intermediate-level software engineer to an expert? 2. Where does one gain knowledge about all extremely-advanced concepts that are used in major industry software like Amazon,Google,Facebook,Quora etc? (Optimization, scalability, reliability, large user base etc) 3. I rely for knowledge on established work like tutorials and books , instead of research papers and publications. Is that the problem? 4. Is knowing advanced math that important to make great software? I use existing or popular algorithms to do my work, but I don't have the strong mathematical background to create those algorithms. Sometimes I don't even understand the math behind popular machine learning algorithms. Successful developers like Sergey Brin, Mark Zuckerberg, Adam D'Angelo, Bill Gates are all known to be extremely good at math. 5. Am I missing out on some big knowledge channel? Background, leading to the above questions( lengthy, but will be useful in understanding the problem and answering it): I'm a Computer Science graduate who is now in the middle of a Master's programme at an internationally reputed university. Throughout my CS education, I've worked on various college projects that help you underst and the basic concepts of each subject. But all of these end up being crude and naive, something that no one would use. They're not industry-grade in the sense that they're not as reliable, scalable and efficient. And the thing is, no school actually teaches you how to convert these "projects" to real high-quality products. Schools do a great job of laying a good foundation in some subject, and they also go in to introducing advanced concepts. But they never get into the full details. And that's because they do not have the resources to do so. Eg: They don't have Google's massive data centers, so they cannot really teach you how to work with big data the way Google does. I've made music players, compilers , OSes, cloud services, android applications but none of them are VLC, gcc, Android OS, Amazon/Google Cloud etc. And I don't have any idea how to make them of such high quality. I'll illustrate this with some more examples. NOTE: These are just specific examples, my question is of a more general nature. a. If I am making a software that uses a database, how do I ensure the best data security for my users' sensitive data? How do Google, Facebook, Amazon ensure sensitive data security so reliably? We actually trust them, and they deliver. How can I bring that to my products? b. If I make a social networking site, what should I do to properly support a billion users? c. If I make a language translation software, how do I make it perform really fast and with high accuracy? d. If I write software for a missile guidance system, how to make it extremely optimal? And these are things you cannot just google and find "tutorials" for. You've to make it yourself. But how? So far in my education, I've worked on projects as an engineer, but not as a scientist. That is, I learn "how to do this and that" from the internet and then implement those in my programs. I'm good at putting different things together like an engineer, but I don't have research-oriented thinking.
-
Answer:
There's only one answer: practice. Oh, sure, there's a ton of info out there on the web - lots of great blogs about scalability, tons of technical papers, places like Quora and StackOverflow to learn from, but at the end of the day, the only way you're going to learn is to actually build stuff, have it blow up, and build it again... In theory, not too hard. In practice.... A bit more difficult. How do you get an app with a million downloads? How do you know if your SaaS package built to scale to 100 million users actually will? And the biggie, how do you get someone to pay for all that? I don't have any good answers for that. I was lucky. I was one of a very limited number of people at the beginnings of the TCP/IP era who had experience in hardware design, programming both shared and distributed memory multiprocessors, UNIX, real time and embedded systems, and a handful of things that put me on the "oh hey, call this guy" list. I was very VERY lucky. I parlayed my personal interests into interesting work, and leveraged that to do more interesting work, etc. There was a saying from http://en.wikipedia.org/wiki/Tracy_Kidder's book http://en.wikipedia.org/wiki/The_Soul_of_a_New_Machine where one of the key players was talking about the industry. He said "It's like pinball. You play, you win, you get to play again." My best advice: start playing. Get a job where you can learn something you don't know, using something you do know. Join open source projects which offer the same opportunity. Sure, you're working when you're at work, and you're working in your free time, but hey, you're building a profile, building skills, learning stuff you can only learn by doing. And sooner or later, your name will be on someone's list. Good luck.
Stan Hanks at Quora Visit the source
Other answers
Incrementally move to massive scale and/or complexity with progressively larger problem scope and personal responsibility. Iteratively approach industry-leading quality by not repeating mistakes you've encountered before (yours and other peoples') on new projects. As suggests this is much more accessible (people actively recruit you) once you have enough initial experience. While reading relevant papers is a good way to start projects, the hard issues (like how to validate what you built, design so that problems can be debugged, make systems resilient to unavoidable implementation errors, apply appropriate software process) are practical ones solved by engineers who write relatively little about what they did. You'll go farther sooner standing on the shoulders of such people farther along the path than independently re-inventing the wheel. Seek them out as co-workers and don't expect to find them at mature companies where cruft and hiring practices get in the way of practicing their art or geographic locations which don't have healthy ecosystems building complex software. At some point progression will require completely new projects which don't yet have the constraints implied by intellectual property already owned by an organization. There are a lot more opportunities to do that in the San Francisco Bay Area than elsewhere. We have 3X more venture capital deals than second place contenders like New York City, New England, and LA. The advantage is 10-20:1 over third-tier cities like Austin TX, Boulder CO, Portland OR, Raleigh NC, and Seattle WA. That comes at the cost of affordable houses and culture although many people are happier optimizing what they do for work on which they spend more waking hours than anything else. Paradoxically location matters more than it did before ubiquitous high-speed internet. Decades ago hardware constraints limited software scope, the orders of magnitude separating simple from complex, and the accompanying diversity of thought needed to solve different problems. Now that's no longer the case and people benefit more from living in larger ecosystems where they're more likely to have experts to work with in and around their areas of specialization.
Drew Eckhardt
Get a job. Work maintenance on existing projects. Get practice. Network. Get involved in more ambitious, "make something better" projects. Get practice. Network. Get noticed. Get involved in a brand new development as project lead. Get practice. Network. Lather. Rinse. Repeat. Take over the world. Profit!
Christopher Pow
How do you become a top industry-grade software engineer? Same way you get to Carnegie Hall: "practice, practice, practice." You can get practice in a couple of ways: by getting a job with a company that's doing interesting stuff, or by finding an open-source project that's doing interesting stuff and becoming a contributor. In either case, expect to put in your time "paying your dues" by doing maintenance work, fixing bugs, documenting, and so forth, before you're trusted to work on more central parts of the application in question. These days, most of the more-advanced things you can learn about will be found online. Find the news sites that cover the technology you're interested in, but, beyond that, find out who is working in the areas you want to learn about, and learn all you can about them. Follow them on social media, and see if they have regular blogs where they talk about what they're working on. (Many do.) Engage them to a certain degree, just don't "stalk" them. Mostly, listen to what they have to say. Advanced math isn't as important as you might think. Some graphics work relies on matrix algebra, but the math for most software that people use (outside of big supercomputer simulations and the like) doesn't really get beyond basic algebra. You can also start some of your own projects as open-source and put them up on GitHub or the like. Part of what this does is it helps build up a "portfolio" of sorts, which you may be able to leverage to get a better job than you might otherwise have done, as potential employers can see your work. Keep an open mind, keep learning outside the classroom, and keep in practice. That's the best way to get ahead. Hat tip: for the "unofficial" A2A. Edit: Read 's comment below, too. I like what he has to say.
Eric Bowersox
TL;DR All the above mentioned companies have one underlying concept at their heart, or at least had one when they started. They all focused on solving a problem or filling up a gap. Before Google, even the biggest corporations failed to create a search engine which made some sense. Come Google and boom! Google grew up from there. Google never thought about how they would support a billion users. I believe that is a recipe for disaster, because then you are an ant looking at scaling Mt Everest and would die out before the summit is even nigh enough. I am already 7 years out of college and so wish that the current MOOC world was so open at that time. As I write this, everything you require to be a great programmer+entrepreneur, solving internet scale problems to how to sell those solutions as products, are available as courses on platforms like Coursera, Edx etc. Make the most of them. Again, try to solve a problem. It would show you the way if you persist with it.
Aditya Gupta
If you want to know the real answer to your question do not read this answer... I told you stop reading!!.... I am serious stop reading or you will become for ever unemployable by any one in Silicon Valley () EDIT: Any blog links may or may not be working the owner of the blog somehow took a dislike to this answer and deleted them. Again proving reading it bad for your health. This is true of the linked questions. Ok you asked for it here it is: For my general bio and such see one thing you will see is I ****SUCK**** at math (and apparently what passes for as the non-CNF language called English) and think it has no real place in CS. I am going to disagree with almost everyone's answer.... the answer *IS NOT* practice it is learning how work smarter and when you have the all the fancy support systems of a university and/or company your not forced to work smarter and not harder (yes the company/school give lip service but if the cost of saving you an hr was greater then the cost of paying you for the hr guess what your doing tonight... since your cost to the school is near zero [except the power your presence consumes] it has no incentive to give you assignments to keep you busy). Step 1: STOP DOING COLLEGE ASSIGNMENTS Either do the assignment professionally or not at all. Like in real life it is up to you a) prove you a professional not the school or the company and b) there is no such thing as college projects these are work projects (if you need to motivate your self pay your self $1/hr or something) Now back to the general answer. So the best way to *REALLY* learn how to do it right is then do it for real (not what some employer calls real, but I mean fucking for real) namely do projects that you must do or you starve. Do this 2 or 3 times and you will be sr. and do it 5 or 6 times and you will be world class. Do it 9-10 times (like I have) and you're an insane masochist. Step 2: DON'T WORK FOR SOMEONE ELSE The only way is to do it right or starve and the quickest way to starve is start your own company. Your first company should be throw away and you should stay to relatively simple commercial jobs in it (you will not loose too much face here instead of going for the golden ring and failing). Here goes the first secret of the industry: None of the people you listed in the details as developers are developers they are BUSINESS PEOPLE (the evil suits). Now that you have the environment that you can really learn in lets take the next steps: Step 3: FORGET ALL THE CS YOU HAVE LEARNED Every invention I have made (8 total) has because I refused to accept what CS said about some thing. For example an idea I had a few years ago when I returned to get my BA in CS after 20 years in the industry was when I realized that my theory of computation book was full of it on a key problem in decidability. Forth coming product so no more details. Don't worry if anyone else thinks your wrong or right as long your code works and proves the point who cares what those who do not have the time or brain power to understand it can't. Such this harassing troll I am dealing with right now who thinks he understands better then I do when I invented the thing in the first place. Step 4: YOUR GUT IS RIGHT NOT SOMEONE ELSE Even if you are wrong you will learn so much more then you worked for someone else it is worth it (even if your poor clients don't agree). Step 5: ALWAYS DO PROJECTS BEYOND YOUR CURRENT SKILLS Part of what you need to learn is your limits and this will teach them to you. Step 6: READ READ READ Now that you have killed a couple smalll dragons learning the right way above it is now time to go back to school and learn the right way. Read the classics of software engineering: The mythical man-month Goto considered harmful The gang of four Recursive Make Considered Harmful The pragmatic programmer Step 7: MAKE A FRAMEWORK You will fail but attempting to make one will teach what you need for being an architect. Step 8: GIVE UP YOUR COMPANY By now your going to have all kinds of horrible unmaintainable legacy code. It was wrtten by such an idiot that it is unsavable. Step 9: COME UP WITH THE NEXT KILLER APP AND MAKE IT The rest is up to you If you want a mentor we are looking for help in making our secure open source IaaS/PaaS (no other IaaS/PaaS can make even 1/10th the security claims we must make under the law and we surpass that by a mile). I promise to be mercless on your code. The plus though is since it is commercial open source you can sell the results of some or most your work doing steps 1 and 2
Aryeh Friedman
Let me thank you for asking a very specific and fabulous question. It is evident many are in search of this answer. I have resource which is tailor-made for you. It has been specifically created/ compiled to cater to software development aspect of building highly scalable software products. http://www.aosabook.org/en/ It contains three books consisting of personal experiences about technical challenges faced by the chief technical developers of some large scalable software product many of which are still in use today, ranging from databases to web applications. All challenges are covered with strong focus on technical aspect, with all of aspects written by main software developers themselves thus covering all experiences in the order experienced, thus exposing us to their trail of thought patterns. This is what I believe you are looking for. While all other answers are very useful indeed, you already have enough experience under your belt, but you wish to proceed to next level. All above experiences shared in those books will definitely give you ideas on how those smart programmers worked out their problems, but you will have to get yourself into situation where you can experience this firsthand. Unless you are ready to reinvent the wheel again and yet you may not have opportunity to test your product in market to its limit. Sorry buddy, but you have to trust yourself. Take the dive. You already should know your strengths & limitations by now, with your pet projects but go change jobs if you are unhappy with your job trajectory.
Sourabh Lonikar
While making collage level projects, people usually don't follow best practices (at least I didn't). So if you are planning to develop some real world applications, SOFTWARE BEST PRACTICES is the first and foremost thing you should be looking into. Apart from that: Choose technology and framework after doing some research. Keep this in mind ONE SHOE DOESN'T FIT ALL. Don't even think of deploying your application unless you've done thorough TESTING. Choose your HOSTING PROVIDER wisely. Create a separate TEST ENVIRONMENT to test your application. Following are the technologies that you should look for: Angular JS: Best front end framework(MVC framework). RESTful web services. MongoDB: NoSQL Database. I hope this information is helpful for you.
Anonymous
Related Q & A:
- How do I save a large number of incoming e-mails to my computer in one go?Best solution by Yahoo! Answers
- How does one go about moving across country?Best solution by Yahoo! Answers
- How do I go from Yahoo! Classic to the newer one?Best solution by Yahoo! Answers
- How would one go about ridding him/herself of sinus pressure?Best solution by Yahoo! Answers
- How do I go about college transferring?Best solution by universitylanguage.com
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.