Spotify needed a data role that did not exist yet, so it invented one
Spotify once posted a job for a person nobody had named yet: someone who understood the business and also wrote code like a software engineer. Victoria Perez Mola, a systems engineer who spent years between payroll queries, ERP migrations and data warehouses, now does that job in Berlin. For anyone stuck between analyst and engineer, the open question is which skills actually carry over.
Learn what analytics engineers do, which tools they use and how to start practicing SQL today
Victoria Perez Mola explains the analytics engineer role, which sits between the data engineer and the data analyst, bringing software engineering practices to analytics and business thinking to data engineering. She traces her own path from systems engineering studies and ERP consulting in Argentina to SQL development and analytics engineering in Berlin, and lists the skills and tools a newcomer should learn first.
“But basically what the analytics engineer is going to do is going to introduce the good software engineering practices to the efforts of the data analyst and the data scientists as well.”
“I didn't want it to be 100% technical. I wanted to be something in between.”
What you will learn
Distinguish the analytics engineer from the data engineer and the data analyst by responsibilities and focus.
Map the tools an analytics engineer uses at each step: storage, loading, modeling and presentation.
Recognise how mixed technical and business experience, such as ERP and reporting work, transfers into analytics engineering.
Plan a learning path covering database concepts, data modeling approaches, SQL, pipelines, reporting and coding best practices.
Practice SQL with a local Postgres or MySQL database, open datasets and HackerRank exercises.
Key takeaways from the speaker
data
career
Details
Level
Beginner
Format
Talk
Technical depth
Introductory technical
For
Student, Early career, Career changer
Length
18:19
Published
June 2021
Questions this talk answers
What is an analytics engineer?
A role between the data engineer and the data analyst that brings software engineering practices to analytics work and a business mindset to data engineering. It provides clean, transformed data so analysts can focus on analysis.
Where did the analytics engineer role come from?
As the speaker cited, Spotify realised it was missing someone who knew the business and software engineering practices, posted an opening for an expert in building analytical data layers, and hired its first analytics engineer.
Which tools does an analytics engineer need to know?
Cloud data warehouses such as Snowflake, BigQuery or Redshift, loading tools such as Fivetran or Stitch, modeling tools such as dbt or Dataform, and BI tools such as Looker, Mode or Tableau.
What is the best way to prepare and practice SQL?
Download a database such as Postgres or MySQL, get a dataset from an open website and start querying, and use sites like HackerRank whose exercises resemble real-life cases.
Do analytics engineers need Python or R?
Victoria does all her data modeling in SQL and uses neither, though she notes other analytics engineers, for example at Airbnb, use Python and Jupyter, so she recommends Python as well.
About this talk
Have you ever heard about Analytics Engineer?
Data roles keep appearing on daily basis and makes it hard to keep track of them. At the end of this talk you should be able to identify what are the tasks and responsibilities of the Analytics Engineer... and maybe even want to become one.
How Spotify invented the analytics engineer (60 s)
Why she chose a role in between (47 s)
How to practice SQL for data roles (54 s)
Speaker 1: Welcome, everyone. I hope you're enjoying the conference. There are certainly a lot of very interesting talks, and I appreciate that you are attending to a little bit of an introduction about myself. I am an analytics engineer, as the title says, and what I'm going to talk today is how I end up with this title. Basically, because this is a fairly new title, I would say there's not much also online. I really like it, and I want to share it, and I want more people to know about it and what are analytics engineers doing. You can also be one if you want. I'm not going to introduce myself a little bit more because we're going to go more into details into my career.
Speaker 2: What is exactly the analytics engineer role? Speaker 2: So a few years ago, in a Spotify, they realized that they had, they were missing a person. Basically, they were missing someone Speaker 2: That could get like to know the business, but also the software engineer and practices and the technical stuff, because they were seeing that the analysts, they're very deep into business knowledge.
Speaker 2: But they lack of the computer science background, basically. And the data engineers, they have it, but they're very deep into the technical stuff. So they needed something in between. They needed like a connection. And this is what they posted. They didn't know what they needed, basically, but they needed something. And this is what they put in their in their show opening. Basically, we are looking for an expert in building out analytical data layers to showing the band and help drive and data first culture across the ad supported business in Spotify.
Speaker 2: And they invented a title kind of like data specialist, something like that. But basically, this is how they end up hiring their first analytics engineer for the company. Speaker 2: So as I said, basically, they created this role where lies a little bit in between the data engineer and the data analyst. So if we think about the roles in the data team, most of us will already know what a data engineer does and what a data analyst does as well. And we know how the connection is and how they deal with the data.
Speaker 2: So data engineers are going to be preparing and maintaining the infrastructure that the data team is going to be using. So very deep technical oriented and data analyst, they're going to be using that data to answer questions to solve problems. So they are going to transform that data basically to translate it to English to answer the questions that the business has. So they need to have a lot of business knowledge as well or domain knowledge.
Speaker 2: So here is where the analytics engineer comes in. The line in between the data engineer and the data analyst is and the analytics engineer in between is a little bit blurry. So they're going to be analytics engineers that are going to be more on the data engineer side and analytics engineers that are going to be more on the data analyst side.
Speaker 2: But basically what the analytics engineer is going to do is going to introduce the good software engineering practices to the efforts of the data analyst and the data scientists as well. Speaker 2: And it's going to bring an analytical and business kind of mindset to the efforts of the data engineering. Speaker 2: If we go a little bit more deep into the responsibilities. So we see here the data engineer is going to be taking care of the platform. So they're going to build custom data integrations.
Speaker 2: They're going to manage the pipelines. They're going to develop and deploy the machine and learning endpoints. Speaker 2: And they're going to also take care of the data warehouse where the data analyst is going to do deep inside works. They're going to be answered questions about like, how's the net revenue like? Speaker 2: What happened when a user does this? What are our users doing? They're going to work with business stakeholders. They're going to be working with the business people that know nothing about data and tech.
Speaker 2: And they're going to be going to be building dashboards doing forecasting reporting. So the analytics engineer, it's going to provide clean transform data for this analysis for the analyst. So the analyst doesn't have to do the technical work basically date and they can do what they know best, which is analyze the data. Speaker 2: They're going to apply software engineering best practices to the code. So version control.
Speaker 2: CICD testing. And they're going to know they're going to know best how to write good code basically. So they're going to maintain this code using the best and applying the best practices. Speaker 2: Also maintain documentation and definitions and they're going to train the business users on how to use all these tools that they are maintaining.
Speaker 2: And I'm talking a lot about tools. So let's go a little bit deeper into that. So since the analytics engineer goes a little bit from the data engineer to the data analyst, there are a bunch of steps that the data goes through that the analytics engineer is going to be involved with. Speaker 2: So the first one is data storing. And here we're going to talk about cloud data warehouses like Snowflake, BigQuery or Redshift. So the analytics engineer should be familiarized with this.
Speaker 2: Tools as well. We're talking about data loading. So here we can talk about ETL tools like five trying at leap or stitch. Speaker 2: Then comes the data modeling and they are now very cool tools like DBT or data form that kind of like a started with the analytics engineer movement. And at the end, there need to know also about data presentations. And here we are talking about BI tools like looker.
Speaker 2: Mode or tableau mode or tableau. Yes. So there's a question that says as the analytics engineer does the data preparation part for the data analyst. Yes, they do. Um, so the, let's say, um, I'm going to give like a more clear example. So let's say that now the data Speaker 2: business needs a new KPI depending on a new kind of like data point. So the analytics engineer is going to take care of, um, making that data available. So of course there's going to be a data engineer work that it's going to maybe like get the backend events or something like that. But then the analytics engineer is going to take care of, um, Speaker 2: making that data available. So of course there's going to be a data engineer work that's going to maybe like get the backend events or something like that. But basically what I would do is I would try to build the pipeline in the ETL tools.
Speaker 2: Um, I'm going to do the data modeling using, in my case, I use DBT a lot and, and then I'm going, so these models are going to be available for the data analyst and they're the ones that are going to know with what the business is. Speaker 2: or squand, right? And I'm going to make this data available in the VA tool as well. But at the end, the data analyst, they can build on top of whatever I did. I clean the data, the data that I clean and I made available for them.
Speaker 2: So how did I get myself to the analytics engineer role? So this is my geographic journey. So as I said before, I come from Argentina, Speaker 2: I come from Argentina and in there I started systems engineer and started working there as well. And at the moment I'm now living and working in Berlin in Germany.
Speaker 2: So let's go a little bit more into details and I'm going to be telling exactly how the knowledge that I acquired over the years. They, um, I use them as an analytics engineers today. So as I said, I started studying systems engineers in GSL. Speaker 2: And I finished in 2015. Um, overall. So I learned a lot about computer science, about managing IT projects as well.
Speaker 2: And in there, there were several subjects that were about how to write code, how to use the right patterns. So things like the drive concept, for example, how to write clean code, basically, how to make it maintainable. Speaker 2: And, um, and the, um, I'm, I'll get to the question on the end. That one. Um, so, and then I'll, there were also several subjects about, uh, databases, data warehouse, and information data information management.
Speaker 2: So all of this are the subjects. I would say they helped me the most to get where I am today, because I learned the very good practices of a software engineer and also learn all the basics for data management as well. Speaker 2: In there. I also started to work as a technical and functional implementer for an ERP company.
Speaker 2: So my work there was a little bit mixed. I wasn't fully technical, but I also wasn't fully on the business side. So I needed to learn a lot about payroll. Speaker 2: I was a specialized in payroll mostly, but I need to learn a lot about accounting about taxes as well. Speaker 2: And then I needed to use my technical knowledge to write the queries that people would use to calculate payroll to calculate taxes as well.
Speaker 2: So, and there I started to work with databases, um, for the first time, aside of like the studies, and I started with SQL Server and relational databases. Speaker 2: So it served, it, it helped me a lot to understand how business users things, how the accountant things, for example, in this case, and how I could apply my technical knowledge to help them do that work.
Speaker 2: After I graduated, I started working as a consultant. I started working as an ERP consultant. So I continue with this mixed profile. So I never was going for the technical and never was going fully on the administration business kind of side. Speaker 2: Uh, again, I was using a lot of my knowledge and still forming myself, learning more accounting, more about taxes.
Speaker 2: And come combining those to help people to use their systems, to build reports, to understand what the data looked like when there was an issue. Speaker 2: So I had to write a lot of SQL, a lot of procedures. I had to write even ETL, um, to bring the data from others. Speaker 2: Systems to the ERP system, for example. And I also learned a lot about management about project management as well, because I had to do this was a very small company.
Speaker 2: So it helped me a lot to be one woman show basically in the projects. Speaker 2: And then in 2018, I decided to go to Europe and travel and I ended up living in Berlin. Speaker 2: And that's when I started working as an IT systems specialist. And this job, I thought it was perfect for me, but because again, I was going to be working with an ERP system in the finance department, helping accountants to do their jobs, basically. So doing the reporting, integrating the ERP without the tools.
Speaker 2: Uh, but then soon I realized that I was a bit over with ERP systems because they're all different. They're all changed. They're all very customized. Speaker 2: So at the end, it's always learning again and again, how they work. So that's when I move and I moved to SQL developer because I wanted to be for the first time a little bit more technical.
Speaker 2: And that's where I started working for the first time as well with data warehouses. I knew the theory. I hadn't started, but that's when I started building a data warehouse for real. It was an SQL server. Speaker 2: And I had to really, really get my game with dimensions and, um, and fact tables and what are data parts and how to get the data from other places.
Speaker 2: But at the same time in the company, they were doing an ERP implementation. So I started to work a little bit on the project. And soon enough, they moved me to a complete new team. Speaker 2: To implement and do like the whole data migration of the ERP. So once again, I couldn't get out from the ERP side.
Speaker 2: Uh, and I still had like this mixed profile once again, even though I tried to go a little bit technical and then it was close to me that this is what I wanted. I didn't want it to be 100% technical. I wanted to be something in between. Speaker 2: And I wanted to be technical and not, and not on the ERP side, right? So that's when I moved to analytics engineer.
Speaker 2: And I really enjoy it because I get to know a lot of like the domain knowledge. I got to know a lot about the business, but I don't have to deal with the business stakeholders all the time because I have the analysts and the data scientists doing that for me. Speaker 2: And they're also the dates what I can choose be there with my music and just working in my code and being deeply technical at the same time.
Speaker 2: And now there are a lot of questions that I'm going to answer in this part of the of the talk. So for example, what is the best way to prepare and practice SQL? Speaker 2: Can I know what the professional shop qualifications we should have to be an analytics engineer? What skills and tools do you think are important for becoming analytics engineer except data preparation and data exploration techniques?
Speaker 2: And if I use Python and R in my work. So these are the things that I feel are the most important ones to become an analytics engineers. So first of all, databases concepts. Speaker 2: Learn a lot about what are databases, how they work, what are data warehouses, data modeling. So read about relational models, C-shaped date, for example, read about Kimball, read about Inmon, read about data bulbs.
Speaker 2: You don't need to know the very detail, but you at least need to see what is out there. And also the more, you know, the more ideas you're going to get, right? Speaker 2: Learn about SQL SQL is everything for me. So this is, this is, I don't personally, I don't use Python or R, but I do know about other analytics engineers. So for example, in Airbnb, I know they use Python and they use Jupyter as well. So yes, I would recommend Python as well.
Speaker 2: But mainly SQL. So I do the whole data modeling in SQL and I've been working with SQL for the past years. So you should know from basic queries to window functions. Speaker 2: How, what is the best way to prepare and practice SQL? Download anything like Postgres, for example, MySQL, create a database or get a database from somewhere. There are a bunch of open websites and start querying.
Speaker 2: There's also websites like HackerRank. I really, really like HackerRank, for example, and you can go there and you can write queries and those are similar to real life cases. So that is very good experience as well. Speaker 2: Read about data pipelines, ETL, ELT. What are those things? What is the data pipeline? Again, you don't need to know like at the very detail. There are also a lot of tools, but you need to understand the concepts.
Speaker 2: Learn about reporting. So have a general knowledge on how to present the data using data visualization tools. Speaker 2: And at the end, also, it's very important to know the good practices. So the software engineering best practices for code development. What does the good code looks like? What should you do to get like good code and maintain code?
Speaker 2: So I hope you enjoy it. I hope you understand a little bit more what my job is about. You can find me in GitHub and in there I have a repository where you can find these slides. Speaker 2: And in the slides, not only I put the links on the tools I linked before, but there's also this last slide where I put the link to a lot of articles that I think are very interesting, especially the one, for example, at Spotify, where I took the code at the beginning.
Speaker 2: And at the end, you're going to find this analytics readings. This is a repository that has a lot of links to a lot of useful resources to learn about all the concepts that I mentioned before. Speaker 2: Do you have any more questions? I think we are also good with time. So if you have any more questions, I can say for a few more minutes.
Speaker 2: Anyways, I appreciate you attending to this talk and I hope you continue enjoying the conference. Speaker 2: Yes, it's data engineer is similar to analytics engineer, but the data engineer is going to be more technical focus. So they're going to care more about infrastructure basically. Speaker 2: Great. Yeah, you can also, I also left my LinkedIn here. So you can also reach out to me there. You can connect. If you have questions or anything, I'm always willing to talk.
Speaker 2: And discuss and also share resources that I have if you need any, if you want to get like deep your knowledge and something else. Thank you. Bye. Speaker 2: Bye.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
What does an analytics engineer do, and how do you become one?
Victoria Perez Mola explains the data role that sits between engineering and analysis, and the mixed career path that led her to it.
In short
An analytics engineer sits between the data engineer and the data analyst: they turn raw data into clean, well-modeled, tested datasets so analysts can focus on answering business questions. Victoria Perez Mola, an analytics engineer based in Berlin, shows that the role suits people who want to stay technical while understanding the business, and that SQL, database concepts and software engineering habits are the core skills to build.
Key takeaways
Distinguish the analytics engineer from the data engineer and the data analyst by responsibilities and focus.
Map the tools an analytics engineer uses at each step: storage, loading, modeling and presentation.
Recognise how mixed technical and business experience, such as ERP and reporting work, transfers into analytics engineering.
Plan a learning path covering database concepts, data modeling approaches, SQL, pipelines, reporting and coding best practices.
Practice SQL with a local Postgres or MySQL database, open datasets and HackerRank exercises.
A role born from a gap at Spotify
Victoria opens with the origin story of her job title. As she tells it, Spotify realised a few years ago that something was missing in its data team. Analysts knew the business deeply but lacked a computer science background, while data engineers had the technical depth but were far from the business questions. Spotify did not quite know what to call the person it needed, so it wrote a job opening for an expert in building analytical data layers and helping drive a data-first culture, and that is how it hired its first analytics engineer.
She then places the role on a spectrum. Data engineers build and maintain the infrastructure; analysts use data to answer questions, or as she puts it, "So they are going to transform that data basically to translate it to English to answer the questions that the business has." (2:33) The analytics engineer lives in the blurry line between them, and individual analytics engineers can lean toward either side.
What the job involves day to day
The core of the role, in Victoria's words: "But basically what the analytics engineer is going to do is going to introduce the good software engineering practices to the efforts of the data analyst and the data scientists as well." (3:23) In practice that means providing clean, transformed data for analysis, applying version control, CI/CD and testing to analytics code, maintaining documentation and metric definitions, and training business users on the tools the team maintains.
She walks through the stack an analytics engineer touches as data moves along: cloud data warehouses such as Snowflake, BigQuery or Redshift for storage; ETL tools such as Fivetran or Stitch for loading; modeling tools such as dbt or Dataform, which she credits with starting the analytics engineer movement; and BI tools such as Looker, Mode or Tableau for presentation.
Answering an audience question, she gives a concrete example. When the business needs a new KPI based on a new data point, a data engineer may first capture the backend events. Victoria then builds the pipeline in the ETL tool, models the data in dbt and makes it available in the BI tool, so analysts can build on top of clean data rather than wrangling it themselves.
A career built in the middle
Victoria's own path shows how a mixed profile prepares someone for the role. She studied systems engineering in Argentina, finishing in 2015, where courses on clean, maintainable code and on databases, data warehousing and information management gave her the foundations she still uses. In parallel she worked as a technical and functional implementer for an ERP company, specialising in payroll and learning accounting and taxes while writing the SQL Server queries that calculated them.
After graduating she became an ERP consultant, writing SQL, procedures and even ETL to bring data into the ERP from other systems. Working in a small company, she also learned project management: "So it helped me a lot to be one woman show basically in the projects." (10:58)
In 2018 she moved to Europe and settled in Berlin, first as an IT systems specialist in a finance department, then as a SQL developer building a real data warehouse with dimensions and fact tables. Pulled back into an ERP migration project, she realised what she actually wanted: "I didn't want it to be 100% technical. I wanted to be something in between." (12:41) Analytics engineering gave her that balance, with business knowledge from analysts and data scientists and long stretches of deep technical work on her code.
The skills to learn first
Victoria closes with a learning list built from audience questions. Start with database concepts: how databases and data warehouses work, and data modeling approaches such as relational models, Kimball, Inmon and Data Vault. "You don't need to know the very detail, but you at least need to see what is out there." (14:25)
Then SQL, which she calls central to her work: "Learn about SQL SQL is everything for me." (14:25) She does all her modeling in SQL and does not use Python or R herself, though she notes analytics engineers at Airbnb use Python and Jupyter, so she recommends Python too. To practice, she suggests installing Postgres or MySQL, loading an open dataset and querying it, and working through HackerRank exercises that resemble real cases. Beyond SQL, learn the concepts behind data pipelines, ETL and ELT, get a general grasp of reporting and data visualization, and study software engineering best practices for writing and maintaining good code.
Her slides, tool links and an analytics readings repository of further resources are available on her GitHub, and she invites viewers to connect with her on LinkedIn.
Watch the talk
"How I became an Analytics Engineer" by Victoria Perez Mola, 18 minutes: https://www.womentech.net/video/how-i-became-analytics-engineer
About the speaker
Victoria Perez Mola is an analytics engineer based in Berlin, Germany; born in Argentina and a graduate in Systems Engineering, she works as an Analytics Engineer at Tier. She has over 5 years of experience working in IT with reporting and databases and is passionate about using technology to help people make their daily tasks easier.
Questions this talk answers
What is an analytics engineer?
Where did the analytics engineer role come from?
Which tools does an analytics engineer need to know?
What is the best way to prepare and practice SQL?
Do analytics engineers need Python or R?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
Kimball dimensional modeling
Inmon data warehousing
Data Vault
ETL and ELT
CI/CD
Version control
DRY principle
Resources mentioned
HackerRank
PostgreSQL
MySQL
Analytics readings repository (GitHub)
Speaker's GitHub repository with slides and links
Spotify article on its first analytics engineer
Cite this talk
Victoria Perez Mola. "How I became an Analytics Engineer." WomenTech Network, 2021. https://www.womentech.net/video/how-i-became-analytics-engineer