Career Path for Data Science - How to be a Data Scientist?
Isha Shah28:142022
Even product managers still ask data scientists whether to trust data or instinct
Even the product managers who work with data scientists every day keep asking the same thing: should I trust the data, or my instinct? Senior TikTok data scientist Isha Shah, who started out as a data analyst, and colleague Claire Zhang call it one of tech's most ill-defined job titles. So what does the work really involve, and what do data science interview rounds actually test for anyone trying to break in?
Get the eight core data science responsibilities and a round-by-round interview prep plan
Isha Shah and Claire Zhang, data scientists at TikTok, explain what data scientists actually do, from data pipelines and A/B tests to causal inference, and the value that work brings a business. Isha then compares four data roles and walks through the data science interview loop, with the topics and prep resources for each round.
“Should I trust data? Or should I follow my instinct?”
“Okay, so usually when we have this kind of a very vague question, so this is a time we try to understand what is the goal of this question, and transform this question to a mathematical question, to something we can quantify and help them with.”
What you will learn
Distinguish data analyst, data scientist, data engineer and machine learning engineer roles by their responsibilities and skill sets.
Explain the eight core responsibilities of a data scientist and the business value each one creates.
Structure answers to product metrics interview questions across investigating metrics, measuring success, feature changes and trade-offs.
Design an A/B test from hypothesis through sample size, statistical power, alpha and the launch decision.
Prepare for statistics, machine learning and SQL interview rounds using resources such as StatQuest, LeetCode and Tableau.
Key takeaways from the speaker
The presenters are from Ex-Meta, Ex-LinkedIn and currently at TikTok. They'll be sharing their own journey of getting into data science.
Attendees should feel confident about the role of Data Science in a tech industry and how to succeed in their careers.
Attendees will learn various roles in Data and what value it provides to the business
This session will provide them with the resources, and skills to become a Data Scientist
Details
Level
Beginner
Format
Workshop
Technical depth
Introductory technical
For
Student, Early career, Career changer
Length
28:14
Published
July 2022
Questions this talk answers
What does a data scientist actually do?
Claire Zhang groups the work into eight responsibilities: data pipeline applications, analytics frameworks, interactive dashboards, A/B testing, data research, machine learning, programming and causal inference.
How is a data scientist different from a data analyst, data engineer or ML engineer?
Analysts gather and clean data and share insights using SQL, dashboards and storytelling, while data scientists add mathematics, statistics, experiment design and scripting. Data engineers build and maintain pipelines and ETL, and ML engineers build and deploy models into production with a software engineering background.
What rounds are in a typical data science interview?
Isha Shah describes four onsite rounds: a product or analytical framework round, statistics, SQL or Python, and a behavioral round, with some companies adding a machine learning round.
How should you answer product metrics interview questions?
Identify which of four types it is (investigating metrics, measuring success, feature changes or metrics trade-offs), clarify the question, gather context, form hypotheses, define clear metrics and use A/B tests to measure impact.
What is the hardest part of being a data scientist?
For Isha, it is convincing product managers to follow the data when metrics do not support a feature they feel pressure to launch. For Claire, working on bullying and harassment, it is facing firsthand escalations, which she turns into motivation.
About this talk
In this workshop, we would like to talk about career paths for data science that includes - what is the role of Data Scientists?, Value of Data Science for a Business, Data Science Career Prospects, Skills required to be a Data Scientist and resources/projects for people starting their journey in Data Science.
Turning vague questions into math and dashboards (83 s)
The hardest part of a data scientist's job (64 s)
Good afternoon, everyone. Thank you for joining this session today. My name is Isha Shah. I'll be presenting with my colleague Claire here on Career Path for Data Science, How to Become a Data Scientist. We are very excited to share our journey as we started our journey as like data science. And I hope people who want to start their career in data science, or are looking to transition from data analyst to data science track can benefit from this session. So let's start with the introduction. My name is Isha Shah. I studied my master's in information system from University of Illinois, Chicago. And my first job was a data analyst. So I started as a data analyst for a couple of years. After that, I switched to data science track. And currently, I'm a senior data scientist at TikTok.
My current focus is on TikTok Live. And our team focuses on building like new features for our content creators, to make sure that we keep our platform safe. So that's about me. I'll pass it on to Claire. Thanks, Ishia. Hi, everyone. My name is Claire. It's so excited to be here today with everyone. And same as Ishia, I work for TikTok Trust and Safety. And my day to day job is basically how to protect and find out this kind of bad content, especially regarding harassment and bullying online to better protect our user to making sure they're having great time on our platform. And before TikTok, I work for LinkedIn. And then, but in LinkedIn, I actually work in a completely different domain. I work as a go to marketplace. So I'm pretty much like moving from the front end and try to come to the back end to protect more users.
Okay. So this is pretty much about me. And so for today's agenda, actually, we're going to divide everything into three sections. So the first one will be a quick introduction of data. Data scientist. And the second one will be the main part, like how to become a data scientist. And the second one will be the main part, like how to become a data scientist.
Also, the third one is we're going to share everyone some projects and also useful resource for your reference. And for the first one, as I said, who is a data scientist? Also, we try to help people understand what is the value of the data scientist? Okay. Okay. I have to be 100% honest with everyone that data science, especially data scientists, it's actually one of the most influential things.
or even ill-defined job titles in this industry. Because you can see a scientist, a very fancy word in the title. So basically, people think, okay, great, these people seem smart and probably like very, very top talent people, same as these other scientists in the lab. Okay, so especially to most of the people who are not familiar with, data scientists, this is more than a buzzword. And even to people who are familiar or even working with data scientists, for example, me in my daily work, no matter is from my PM or from my stakeholder, actually, I was always asked the same question like, Claire, should I trust you? Should I trust data? Or should I follow my instinct? Okay, so basically, these are two types of questions that we always find people have, no matter you're familiar or unfamiliar with data science.
Okay. So today we try to give a several, like, different perspectives to people what data scientists are doing. And by sharing you guys what we do to help you guys better understand what we are. Yeah. So question came to what else we do? Okay, so I generally divided is job responsibility into eight sections. And the first one is actually about data pipeline applications. But okay, forget about these fancy words. Okay, listen to me. So let's take a very easy, like metaphor. Okay, let's think we have like a lot of data, huge of data. And we can divide your data into like small, small cubic. For example, there's one cubic of our user. There's one cubic of our There's one cubic about location, there's one cubic about behavior, blah, blah, blah, blah. So you will have many, many, many cubics. And try to think if you are a popular platform, like Tiktok, you're going to have like, a large scale, a seriously large scale tons of data every day. In this case, if you just want to find warm, warm, tiny use cases of data, do you actually need to go through all these cubics to find your data? That's kind of impossible, right?
All right. So data scientists are actually smart. So that's why we do this kind of a second layer on top of these small cubics, by different combinations of these cubics, to making sure we only give small chunks of data to cater to each specific use case. So in the future, no matter how big your business growing or how more data you are getting, you actually probably have these small chunks of data growing a little bit as well. But still, it's much faster and efficient to access this small chunk of data, instead of going through all these cubics to finding and hunting for that specific use case data. Yeah.
So this is what we're doing and designing as an efficient way to extract, transform, and also load the data. So this is basically what we call a data pipeline application as a first responsibility. Also, second one, analytics framework. So actually, a lot of times, my manager will come to me with a very, very vague question. For example, hey, Claire, can you tell me how many bad guys are there on our platform? Great. Okay, so usually when we have this kind of a very vague question, so this is a time we try to understand what is the goal of this question, and transform this question to a mathematical question, to something we can quantify and help them with.
And after we understand the question, we also need to come up with a very logical structure to decompose the question, to provide and also provide solutions from all related dimensions to make sure the business stakeholder or any other stakeholder working with us can understand. So this is what we generally call analytics framework. And the third one is interactive dashboard. So this one is very easy to understand, right? So let's say, instead of me sitting there answering everyone's same question again, and again, day by day, how about I just build everything into a dashboard?
And this dashboard will automatically updated every day with a new data flowing in. So next time you don't even need to come to me, you can just see the dashboard, you can see, oh, we have these many users today, we have these many new inputs today. So this will be a great and also easy way for everyone to be on track of everything.
And the first one is A-B testing. And the first one is A-B testing. This one is even easier to understand. So because for example, my PM came to me, Claire, do you think the TikTok logo can be a darker color or a lighter color? Or do you think we can change the logo to a different one? How can I know it? Okay, so we can basically conduct the A-B test to use the data and use a more statistical, significant way, also scientific way, to tell the PM to tell everyone what is the correct or even better answer.
is the correct or even better answer. Yeah. And the fifth one is about the data research. So for example, there may be a lot of latest model, a lot of latest paper, try to uncover some latest technology can be applied in this industry. So to keep our own work, as always to the top of the industry level, and also to making sure we always, we can always leverage the best technology, data research is one of the direction we need to keep in mind and also keep looked in our daily routine.
And I think this one is very easy to everyone because it's almost like another buzzword is machine learning. So try to think instead of us as a human, try to learning and understanding all this kind of huge label data. how about we just let the machine learn it, let the machine understand it and let the machine just predict everything to help us make a further decision. So machine learning is definitely one of the main content.
next step is programming language. So this one is actually very obvious as well. We use a lot of like programming tools like simple Python or all this kind of language depends on what use case and what tooling we're using every day. And the first one and the final one is actually causal inference. So try to see sometimes when we don't have that enough of data to, for example, to help us set up or design A/B test, but we still need to understand the causal.
So what is the real cost behind it. And usually this time we'll probably refer to a causal inference. Yeah, which is more sophisticated, but also really powerful in this industry as well. Great. So after everyone understand what is the data scientists do, I try to, it will be an easier and better way for people to understand the value of us. Yeah. And as I said, after we extract the data, we try to give the insight to all these stakeholders. So in this case, we actually empower the management with more info and also the way to make a smarter decision. Also, by giving them the analytics framework, we also try to leverage data to help them break down their business problems.
And also by leveraging the interactive dashboard, it will be much easier for our cross-functional partners to monitor their metrics and take action if there's anything on the wall. Also, finally, through this A/B test and the A/B test result, we actually can scientifically test multiple ideas and make the best decision out of it. Yeah. And also through the data research, we can apply the latest technology and also method in the industry to making sure we always keep up to date.
We always use the most efficient method. And also by leveraging this machine learning, we actually can imply or encourage the business with the latest AI technology to be efficient at it. Also is, yeah, as I said, we build up this kind of data pipeline application because we have tons of data. So this is a way for us to efficiently processing millions of data. And also, we provide a very clear business problems into a mathematical analytics model, not only through this kind of machine learning model, but also some color inference settings. Yeah. So basically, this is pretty much about the This is pretty much about the what do we do and what is the value of us. So through this to provide everyone an idea what, who is the data scientist. Great. And now I'm going to transfer to Asia to help us understand how to become one.
Thank you so much, Claire. Yeah, let me walk you through the different data roles that we have in our industry. So when you apply for a job, you'll see there are different data roles. Can you go to the next slide, please? Next one. Next one. Yes, thank you. So yeah, as I said, each role will have different responsibilities, right? So it's very important to know what the difference is between each of the role. And depending on your skill set and the experience you have, you can apply for the right job.
So starting with the data analyst, right? So we have data analyst, data scientist, data engineer, and this ML engineer. So there are four like different categories of the data roles. Starting with the data analyst, you'll see that just highlighted some of the key responsibility here. So obviously there's a lot more to it. But looking at the first one, you'll see they are more responsible for gathering and cleaning data and providing valuable insights and mostly like focusing on the analysis portion and sharing it with these stakeholders.
Going for the next one, we have data scientists. So as a data scientist, as Claire already mentioned what we do, it covers most of our like functions that we work on. So they solve complex data problems and expertise are in mathematics, statistics, machine learning and data analysis. The third one is the data engineering. This is a little different. You'll see that data engineers, they build and test and maintain data pipelines, provide quality data for our machine learning models.
So the skill set required is a little different than data analyst and data science. And the last one is the machine learning engineer. So this is more focused on building and deploying ML models into production. So if you look at the skill set here, data analyst would require proficiency in SQL, building dashboards, storytelling. In the data science, you'll require all that, plus also like mathematics and statistics, AB experiment design, metrics monitoring, and some scripting languages like Python or SQL.
For data engineering, for data engineering, the focus will be more on ETL development. So more focused on scripting skills and obviously proficiency in SQL. Last one is the machine learning engineer. For this, we need to have a solid programming background, would require mostly, it would be equivalent to a software engineer, but mostly focused on the machine learning models.
Yeah. So let's go to the next slide, please. Let's talk about the now we know like different roles, right? We'll be focused in this session, we'll be focusing on the data scientist track, and the skills required to be a data scientist. Next slide, please. Next slide, please. Okay. So when we apply for the data scientist interview, so there are a lot of interview rounds. For example, there will be four onsite rounds. And for that, for the four rounds in the onsite, first would be like product round, or we call us analytical framework.
Second, second would be stats or more focused on the statistics part. Third would be like SQL or the technical round, SQL or Python. And then you'll also have a behavioral round. And based on that, some companies do have machine learning round as well. So we'll also focus on that, like what are some of the resources and things that you can focus on for, for cracking this interview.
So let's start with the first one, that's the product round, or we call the product metrics. This is one of the typical like interview round in data science, the product interview questions from company, they want to understand how you would work on like the whole lifecycle of a product. So usually like practically, you'll work with a product manager, you'll have engineers, data scientists, and they'll work together on building like a feature for a product, and understanding how do you measure the performance of the product.
So starting, we divided into four categories here, these are the general like categories where you have questions from. First is investigating metrics. Second is measuring success of the product. Third is feature changes. And fourth is the metrics trade off. So starting with the first one, which is investigating metrics, the questions that you'll ask would be why is feature X dropping by Y percent. So to, to understand that question first is important to know like clarify the question, gather context information. So they want to know why there is a drop, right?
So we want to understand the cost. For that define like high level reasons, define hypothesis. And then for each of your hypothesis, explain your theory and how, how will you use to fix that problem. So there is a framework follow the framework to answer like each question. So the, each framework is different based on the question asked.
So going to the second one, measure success. So the question typical would be how to measure the success of let's say TikTok live, or how do you measure the success of Facebook marketplace? So it's a very broad question, but it's mostly focused on the success of the feature. So if we launch this feature, how will the company know was it a successful or not?
So identify clear goals or define like clear right metrics, which is very important. What would be the core metrics to measure the success of the product? Understand how users are behaving, uh, using a product. So the, uh, one way to do that would be to test it. So we do like AB test and measure the impact of this new feature.
So understand like how the framework would work and apply that framework in that question. Similarly, third one is feature changes. So this would be more focused on just that feature. So we want to add update or improve a feature. What metrics would you use to track that? Again, similar process, follow the framework for those type of questions.
Last is a little different. That is a metrics trade off. So this is generally asked when, when we want to decide between like two features, we have feature X and feature Y, which one should we launch first? Uh, so for that understanding the goal of each feature is very important. Uh, we want to frame the trade off.
Uh, what are some of the, uh, unifying like metrics between each of the features? What is the impact of launching this feature on our company, like high level goal? So the higher the impact, better the feature would be, and then understand the trade off of that. Uh, from, from that do a AB test and see how your feature is doing.
So that's, uh, one way to, um, go through for the metrics trade off question. Uh, great. Uh, let's go to the next part. There is a prep material. Uh, I personally like these tell our peers. They have a lot of, uh, examples of the product side questions that we just talked about. So they also have some of the examples and how you can walk through that framework.
There are other materials to attach. So feel free to go through that. Um, that would really help you in the interview process. Next slide, please. Great. So for stats and probability, we have attached some popular concepts here. So like mean, median, variance, standard deviation. These are the very popular questions asked in interview would be like, what is P value, confidence interval, confidence level.
And the last part is mostly focused on the AB test. So if we have some question on AB test, very important to know what is hypothesis testing. Z test, D test, sample size estimation, uh, ANOVA, all the different like, uh, tests that you can do. Let's move on. Uh, next attaches the prep material. Feel free to click on the link and go through the, uh, material attached here.
We can move on to the next one. Great. So AB experiment. So this is another important interview round, uh, generally asked. So this is a common method that we use, uh, when, whenever we want to launch like new feature. So the main, uh, main idea behind AB test is to split the users into different group. That is control group and treatment group.
So in the control group, we'll actually provide them with the existing feature and the treatment group will have the new features that we want to launch or be a planning to launch for that. So the idea behind, uh, spreading is to understand how users are behaving in different, uh, groups and then evaluating the metrics and comparing the statistical significance between the treatment and the control group.
So the first step is to always start with the hypothesis, understand the problem, define your hypothesis that are measurable and valuable. So once we have the, uh, hypothesis, the null hypothesis is always that, that there is no difference between the treatment and the control group. And going from that, we are often interested, um, like after formulating the hypothesis, we collect the data.
That's the main part. So the data will have like, what is the sample size? How long do we want to run the experiment? What is the statistical power? The alpha value. So generally common value for statistical power is 80%. So 0.8 and for alpha value is 5%. So 0.05 is the common like value that we use in the industry, but depending on the use case, you can adjust that.
So if you want to have enough statistical power, we want to make sure that we have enough sample size. So that's very important and how long we want to run the experiment. Uh, so to do all that, uh, another thing is obviously start with the success metric. So what's your core metrics and what's the guardrail metrics. Once we have that, we launch the experiment, we analyze the results, and then we see, do we want to launch or not?
Depending on your core metrics, did we see a statistical significant difference between those metrics, uh, that we are looking for? Uh, great care. We can move on in the interest of time. I'll just like to, uh, yeah, can you move on? Yes. For the AB experiment design prep material, the, there are some courses which you can take, and there are some, uh, good examples, the industry examples that are.
Uh, that would be really helpful, uh, to know how the AB experiment works. Great. Thank you. Uh, yes. For the machine learning part, um, we have, uh, there are some common topics. It's very important to know linear regression, logistic regression, decision tree, uh, random forest. Uh, so all the machine learning model and how it works. And to also know like advanced topics, you can, uh, look at the imbalance classification, dimension reduction, feature engineering part.
Uh, another thing that's very important and very commonly asked in the interview is underfitting and overfitting process evaluation metrics. Like what is precision? What is recall? F1, uh, accuracy score metrics. Uh, let's move on to the next slide, please. Yes, we are. So there's a prep material. Uh, StatQuest is another great YouTube channel by Josh Summer. It has like all good materials, not just machine learning, but also statistics too.
Uh, next slide, please. Yes. Last we have like SQL and dashboard. SQL is another very common interview round. Uh, for that, uh, there are a lead core is another like great way to practice. And there's a prep material, which has like more analytics and SQL, which you can do. So the advanced topics would be like windows function, uh, union, uh, joins and, uh, the advanced like, uh, functions that we use.
Uh, make sure you learn that and then practice in the lead core. Uh, for dashboard, we can use like Tableau, which is pretty common in the industry. Many people, uh, many, many companies use Tableau for building the dashboard. So it's good to know the, uh, basics of how to use that. Great. Um, we have like two minutes.
Maybe we can, uh, ask, uh, if anyone has any questions, we are happy to answer that. Uh, yes, we are going to send the slide. I think the process is we have to upload the slide and we can share the link. Uh, feel free to type in the chat too, if you have any questions. Actually, I was thinking, can we just share them the Google doc link or we, we have to go through the system.
Um, yeah, we can share it here. Uh, but I, I'm sure there is a system to share that. Yeah. Should I have a question? What is the most difficult part of your role? Hmm. You sure do you want to take it? Yeah, sure. So I think for me, the difficult part is, uh, when we are working with the product managers and when we have, let's say we have this new feature, we have launched it and we have to convince, uh, like based on our data part.
Right. If things doesn't go well sometimes based on the metrics, uh, but sometimes there's a pressure to launch the feature, uh, but it's very hard for us to convince that the data doesn't support it. Uh, so that's one of the thing that generally happens with me is to like convince them that data is not, uh, we have to follow the data and not just follow our instincts as Claire said.
Um, so that's very hard, but that's a process we have to follow. Um, so that's one of the thing that comes to my mind. Yeah. So for me, the difficult part is actually, um, since I saying I'm working with the bullying and harassment and try to protect all of one, uh, everyone who use our platform from this.
So it's actually a difficult part for my role is sometimes I have to face this kind of a firsthand information or it's kind of these escalations when I see people being bullied. So this actually gave me a lot of like pressure and, but I try to transfer this pressure to more motivation. Like how can I better like protect our user to making sure they're far away from this being harmful, uh, from being harmed.
Yeah. On our web platform. And then, uh, I see Alisa has a question. Any suggestion for PM without technical background work with development team? Hmm. So yeah, go ahead. Yeah. Uh, like the PM without technical, like if they don't have any technical background, uh, but at least if they have some good sense of the product, it's very important to them.
It's very important to know how, what the product is and achieving what the goal of the product is. Right. So for the technical side, I think, uh, as a data scientist, we can help the product manager, uh, have that mindset of like, we have to use our data from the technical side, like help them as much as possible, but stick to the goal here, stick to what your goal is and, uh, impact.
But I have seen like most of the product managers, they are very good. They could have, they have like good product sense. Uh, it's very important to have that good like collaboration with your product manager. Yeah. Um, so I don't, Isha, I think PM is actually also a very vague title. So PM here, do you mean project manager, program manager, or, uh, yeah.
So what specific title and responsibility do you fall into and on your responsibility? Do you, I mean, is technical background that great. And also if you work in a different industry, probably away from technical industry, would that still be like a really, uh, prerequisite? I don't, I probably don't think so. Yeah. Project manager also for project manager.
That in different company will have different responsibility as well. I do know some project manager, they are more on track. Uh, they're more working in a way like help project to deliver on track, help manager all different stakeholders. So in this kind of a case, a technical background is not that important for them. I also see other more technical PM that they actually have, have to have a very strong technical background.
So that all depends on the responsibility. And if you probably don't have that much confidence in the beginning, start with the one that has a light, uh, light foundation on the technical side and gradually moving on and picking up along the work, making until you're comfortable with like more technical in the, in the old work environment. Yeah.
So that's all for me. I think. Hope this helps. And if you guys have any other question or anything you would like to discuss, or even want to know about me, about Isha, about TikTok or any other companies we have been working before, feel free to contact us via LinkedIn. We're happy to connect with everyone and look forward to talking with everyone.
Yeah. I think that's pretty much about today's session. Thanks everyone for attending. Thank you everyone. And we'll share the presentation link. Great. Thanks. Bye.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
What does a data scientist really do, and how do you get hired as one?
Two TikTok data scientists unpack the role, compare four data careers and map every round of the data science interview.
In short
Data scientists turn vague business questions into measurable ones, build the pipelines, dashboards and experiments that answer them, and help leaders decide with evidence rather than instinct. Isha Shah and Claire Zhang, data scientists at TikTok, explain the eight responsibilities behind the title, how the role differs from three neighbouring data jobs, and what each round of a data science interview tests, with prep resources for each.
Key takeaways
The presenters, formerly of Meta and LinkedIn and now at TikTok, share their own journeys into data science.
Attendees should feel confident about the role of data science in the tech industry and how to succeed in their careers.
Attendees learn the various roles in data and the value each provides to the business.
The session provides the resources and skills needed to become a data scientist.
A title that even colleagues find confusing
Isha Shah opens with her own route: a master's in information systems from the University of Illinois Chicago, a first job as a data analyst, and a switch to the data science track a couple of years later. Claire Zhang came from a go-to-market role at LinkedIn and now works on TikTok Trust and Safety, finding harmful content such as harassment and bullying.
Claire is candid that data scientist is one of the most ill-defined job titles in the industry. The word "scientist" sounds impressive, yet even people who work with data scientists daily are unsure how far to rely on them. Her product managers and stakeholders keep asking a version of the same question: "Should I trust data? Or should I follow my instinct?" (2:28)
Eight responsibilities behind the job
To answer that, Claire breaks the role into eight responsibilities. The first is data pipeline applications. Using a metaphor of data split into many small cubes (users, locations, behaviour and more), she explains that data scientists build a second layer that combines cubes into small chunks for each use case, so teams can extract, transform and load data efficiently even as the business grows.
The second is the analytics framework, which starts when a manager asks something as broad as how many bad actors are on the platform. "Okay, so usually when we have this kind of a very vague question, so this is a time we try to understand what is the goal of this question, and transform this question to a mathematical question, to something we can quantify and help them with." (5:15)
Third come interactive dashboards that refresh daily so stakeholders can serve themselves: "So let's say, instead of me sitting there answering everyone's same question again, and again, day by day, how about I just build everything into a dashboard?" (6:00) The remaining five are A/B testing, data research to keep up with new models and papers, machine learning, programming in languages such as Python, and causal inference for cases where there is not enough data to run a proper A/B test.
Each responsibility maps to business value. Claire sums up the impact of insight work simply: "So in this case, we actually empower the management with more info and also the way to make a smarter decision." (9:00) Dashboards let cross-functional partners monitor metrics, experiments let teams test several ideas scientifically, and pipelines make it possible to process millions of records.
Four data roles, four skill sets
Isha then separates four job families that often blur together in job listings. Data analysts gather and clean data and share insights, relying on SQL, dashboards and storytelling. Data scientists solve complex data problems and add mathematics, statistics, A/B experiment design, metrics monitoring and scripting in Python or SQL. Data engineers build, test and maintain pipelines, with a focus on ETL development. Machine learning engineers build and deploy models into production and need a programming background close to a software engineer's.
Knowing the difference matters when applying: "And depending on your skill set and the experience you have, you can apply for the right job." (11:11)
Inside the data science interview loop
Isha describes a typical onsite of four rounds: a product or analytical framework round, statistics, SQL or Python, and a behavioral round, with some companies adding machine learning. The product round matters because "This is one of the typical like interview round in data science, the product interview questions from company, they want to understand how you would work on like the whole lifecycle of a product." (14:47)
She sorts product questions into four types: investigating why a metric dropped, measuring the success of a launch such as TikTok Live or Facebook Marketplace, tracking feature changes, and weighing metric trade-offs between two features. Each calls for clarifying the question, gathering context, forming hypotheses and choosing the right metrics.
For statistics, she lists mean, median, variance, standard deviation, p-values, confidence intervals and hypothesis tests. For A/B experiments, she walks through splitting users into control and treatment groups, defining a measurable hypothesis, setting sample size, duration, power and alpha, then choosing core and guardrail metrics before deciding whether to launch. As she notes, "So generally common value for statistical power is 80%." (20:08) Machine learning prep covers regression, decision trees, random forests, overfitting and evaluation metrics such as precision, recall and F1, with the StatQuest YouTube channel recommended. For SQL, she suggests practising window functions, unions and joins on LeetCode, and learning Tableau basics for dashboards.
The hardest part of the job
In the Q&A, Isha says the toughest moments come when a launched feature's metrics disappoint but there is pressure to ship anyway, and she must convince product managers to follow the data. "Um, so that's very hard, but that's a process we have to follow." (24:34) Claire's challenge is emotional: reviewing firsthand escalations of bullying. "So this actually gave me a lot of like pressure and, but I try to transfer this pressure to more motivation." (24:54) Asked about product managers without a technical background, Claire advises starting in roles with a lighter technical foundation and building skills on the job.
Watch the talk
"Career Path for Data Science - How to be a Data Scientist?" by Isha Shah with Claire Zhang, 28 minutes: https://www.womentech.net/video/career-path-data-science-how-be-data-scientist-isha-shah-claire-zhang
About the speaker
Isha Shah is a Senior Data Scientist at TikTok, formerly at Meta, who works with the product team on launching new features that keep the platform safe. Her co-presenter Claire Zhang is a data scientist at TikTok, formerly at LinkedIn, and co-chairs TikTok's Women Employee Resource Group.
Questions this talk answers
What does a data scientist actually do?
How is a data scientist different from a data analyst, data engineer or ML engineer?
What rounds are in a typical data science interview?
How should you answer product metrics interview questions?
What is the hardest part of being a data scientist?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
A/B testing
Analytics framework
Causal inference
ETL
Hypothesis testing
Product metrics interview framework
Resources mentioned
StatQuest (YouTube channel by Josh Starmer)
LeetCode
Tableau
Cite this talk
Isha Shah. "Career Path for Data Science - How to be a Data Scientist?." WomenTech Network, 2022. https://www.womentech.net/video/career-path-data-science-how-be-data-scientist-isha-shah-claire-zhang