Company BlogCultureData ScienceHyperoptHyperparameter TuningMachine LearningMLflowMLlib

Brickster Spotlight: Meet Greg From Intern to Senior Software Engineer

At Databricks, we’re committed to learning and development at every level, so it’s important to our teams that we recruit and develop our next generation of Databricks leaders. Our interns are encouraged to live out one of our core values, “be an owner” and they play an integral role in developing our platform during their time here. Learn more about Greg Owen, who started as an intern in 2014, and is now a Sr. Software Engineer at Databricks, and why he’s excited to be a part of our growth five years later!

Spark + AI Summit 2017

Tell us a little bit about yourself.

I was studying at Princeton and ran into Patrick Wendell at an alumni event, where we started chatting about the research I was doing (he happened to have done work with the same professor). A couple weeks later, Patrick reached out to me, asking if I’d be interested in interning at a company that he was founding. I said yes and 5 years later, I’m still here at Databricks!

Greg with the team during his internship (middle front row)

Today, I’m an engineer on the Serverless Team and I work at the intersection of the Databricks platform and the Apache Sparkproject. We build features that cross the interface between our webapp and the clusters that run customer code. Because our team touches so many components of the stack, I get to see all of the different initiatives in the company and community, whether they’re in open source or on the product side.

Why did you choose to intern at Databricks? And why did you return as a full-time employee?

2018 Napa Company Retreat

At Databricks, I get to tackle interesting projects that are always a step beyond what I know I can handle. This was true when I was an intern and has continued to be true over the four years that I’ve worked here full-time, even as the company and my role within it have grown. Without a doubt, the strongest driver of my growth (and the company’s) has been my coworkers; they’re experts in their fields who are happy to explain their work and are excited to see me improve as an engineer and leader in the company.

Before my internship at Databricks, I was sure that I was going to grad school after graduating. After my experience as an intern that summer, though, I decided to take my return offer instead. The thing that changed my mind was the culture of teaching at Databricks: all of my coworkers were eager to teach me how their field worked, even if they had a PhD and 10 years of experience. They would explain introductory concepts from the ground up, the tradeoffs between different options, and the reasoning behind their decisions. I wouldn’t have to miss out on the learning experiences of getting a PhD, and I’d also get practical experience building things that real users cared about. Even after 5 years and a 10x increase in the size of our engineering org, Databricks is still that way today. We’re committed to building a team of humble engineers dedicated to learning and growth for the whole team. That’s why I joined full time and that’s what keeps me here today (and now I get to be the one answering questions sometimes).

One of our core values at Databricks is to be an owner. What is your most memorable experience at Databricks when you owned it?

“Ownership” is a two-way street: companies can say they want employees to act like owners, but that won’t do anything unless the company actually gives those employees the space to be owners. I’ve found that Databricks really gets this right: when employees push, the company moves. At a high level, I’ve seen many examples where leadership has acted on concerns that individual contributors bring up. On a more day-to-day level, engineers can influence the product direction when we notice that we can improve the customer experience, reduce bugs, or move forward faster.

From an engineering perspective, I’m really excited about leading the Credential Passthrough project. It’s an interesting technical challenge in which I get to think adversarially, “If I were attacking this feature, how would I break it?” It’s also something that our customers care a lot about – I’ve been on multiple calls with customers who are excited about this feature, and our usage is growing exponentially week over week.

Outside of purely technical work, Databricks is still small enough that individual engineers can wear multiple hats and have an impact on our company culture. A few years ago, I asked to take a sprint off from my normal engineering work and revamp our engineering onboarding, which was struggling to keep up as we added more engineers and increased the scope of what engineers had to know. I automated our setup and restructured our documentation to make it easier to follow – by the end of that sprint, the process had gone from one week to a single day, which has been a huge help as we’ve continued to grow.

What has been the biggest challenge you’ve faced, and what is a lesson you learned from it?

Making sure that I am getting enough feedback (and the right kind of feedback) on the things I want to grow on. One of the things I noticed after graduating was that I was getting a lot less information about how well I was doing. Instead of getting a grade on every problem set, I had to take the time to actively seek feedback on the work I was doing. It’s really easy to get sucked into deadlines, but taking the time to run experiments, reflect on the results, and iterate to make different decisions is crucial to getting better.

Databricks has grown tremendously in the last few years. How do you see the future of Databricks evolving and what are you most excited to see us accomplish?

One of the most exciting things about being at a company that’s growing as rapidly as Databricks is that I’ve gotten to see at least two versions of most of our systems. The experience of seeing how a system evolves over time – designing it, building it, seeing how it breaks in the real world, and then designing the fix – is how you really get into the craft of engineering. This is what I’m most excited to see Databricks continue to accomplish. Getting that full-cycle feedback as requirements and scales change, learning what works and what doesn’t, is something that’s much harder to get in academia (you generally don’t have any users who really care about your project) or larger company settings (a lot of the decisions have already been made or are buried beneath established layers of abstraction). It’s a unique opportunity to be able to participate in and learn from multiple iterations of our product.

What advice would you give to incoming interns and new grads at Databricks?

Always make sure you understand why you are doing something, at both a product and technical level. Any sufficiently complex project will eventually run into some ambiguity, and the only way to resolve that ambiguity is to understand the impact that your work is going to have on the end user. From an engineering perspective, you can’t evaluate whether you’ve done the right thing unless you understand why you’re making your changes and what the expected outcome is. You get better by trying something, seeing how the result compares against your expectations and goals, and then deciding what to try differently the next time.

On a more personal note, the first company you work for will set the expectations and habits you’ll have going into your future jobs. Make sure that it’s a place that has strong engineering talent that will expose you to good ideas and good engineering habits. And if you find a place with good coworkers, take advantage of it! Try to talk to as many people you can, especially outside of your team. The more people you talk to, the more different perspectives and approaches you’ll learn. Building that network will also give you a chance to increase the number of strong engineers who you can work with later on in your career.

Want to work with Greg? Check out our Careers Page.

Databricks Blog

Source link

Leave a Reply

Your email address will not be published. Required fields are marked *