What does a production engineer do?

What is it like to be a Production Engineer at Facebook?

• How different a role is it from a software engineering role? In terms of compensation, how would you compare the two? What are the prospects of getting a software engineering role in another company after you leave this role at Facebook? Is it really close to Software Engineering as the job description claims to be? Will I be making a mistake taking this role over a software engineering role at another company, considering that this is my first job?

• Answer:

  First, a disclaimer: I interned as a PE last summer, and enjoyed it enough that I'm coming back after graduation. Also, I'm in no way an official spokesperson of anything but my own opinions. It would be nice if someone with more job experience also threw in their two cents. (Cough, the people following this question.) Since I enjoyed the role, it would be easy for me to give an unqualified "yes, you should" but I will try to give you a slightly better answer. Here's what I did on a day to day basis: live with large infrastructure. That may sound a bit trite, but PE encompasses a much wider variety of things than a traditional software engineering role. Unlike a traditional SWE, it is usually not a Production Engineer's job to take lists of features and convert them to lines of code. (Though I did quite a bit of that this summer, given my software engineering background.) If you're looking for the safety of specifications or enjoy spending long hours piecing together monolithic blocks of code, this might not be what you're looking for. This does not mean you aren't constantly writing code. It is always a PE's job to be thinking about how built and to-be-built systems work in, well, production. That means understanding the nuts and bolts of the services you 'own'. When something goes down, a PE is often the best person on a team to understand why, even if they didn't write the code in the first place. If you like poking around systems and fixing things on the fly, you will enjoy this. If this sounds terrifying, you won't be happy. If it sounds scary, and you still want to do it, you're a Production Engineer. If you're not chugging out features, then what are you doing? The need for a role like PE at Facebook or SRE at Google arises out of an operations problem: how do you run some software on some machine to provide some service? On the scale of one or a few machines, this job involves an admin doing lots of manual setup and maintenance running a bunch of commands. On the scale of many data centers holding thousands of machines serving a billion people, the problem of operating software becomes an interesting puzzle of its own. You can't solve it in the piecemeal fashion that works just fine on one computer. PEs write the code that wrangles these systems. Even in the few short months of my internship, I gained a ton from being thrown into a PE roll. The job is an excellent education in how software services work, and sometimes don't work, at scale. There are only a handful of places in the world where you can see software operate at that magnitude, and Facebook is one of them. In my personal opinion, the operations experience you can gain as a PE is an excellent way for a more traditional SWE to round out their skill set. If you don't know now how to go about actually operating a real piece of software in a datacenter, you will by your 6 month mark. About pay: Salary at FB is, of course, based on performance. The same scales are used across all engineering divisions. You should talk to your recruiter, because they will be able to give you a much better answer. (Also, expect it to be honest. I never got the sense that FB recruitment would say something un-candid to persuade someone.) On making a mistake: I've tried to give you a more roundabout answer that describes what I saw the PEs around me doing during my time at Facebook. If it sounds interesting to you, then I would say you're not making a mistake. Also, it's pretty easy to move around in this industry both inside of a company like FB and outside of it. Like any job, this one has it's ups and downs. Some might say the operations work is a downer. I say if you're doing too much ops work, it's about time you wrote some software to do it for you. I didn't expect to write this much, so I'm going to stop now.