Senior Software Engineer - Revenue Data Platform (Revenue Platform)

Email [email protected] if interested in applying. 

Who We Are:

Revenue data has the largest data challenges in all of Twitter. We are making the Terabytes of data we collect every day along with its history fully queryable and searchable in seconds. Advertisers, data scientists and engineers need that data to be broken down by market segments and user attributes that cover as many as 50 dimensions. To make that data easier to generate, store and query we’re building a next generation analytics framework that will power all Revenue data and eventually do the same for Twitter broadly. We seek to make answering questions at Twitter as easy as asking them. Our team is pioneering new technologies at Twitter and was the first to run production services in Google Cloud. By using cloud technologies to solve problems we've learned that we can offload pager and infrastructure maintenance to Google and focus all of our time on solving the large problems that Twitter faces. Given the enormity of the task we face, we need your help to design and implement the tools and services that will make it easy for Twitter to make data driven decisions.

If you have a philosophy of building the tools that solve a broad set of problems, this is the team for you. If you have a relentless customer focus that will help us to accelerate developer and analytic productivity, this is the team for you.

What You'll Do:

Rather than speak in generalities, here is a sampling of what we’ve been building:

• Large Data Collider (LDC) - Analytics query server with a pluggable backend that gives customers a consistent query interface. By providing consistency at query time, we can swap out backends like Bigtable and Druid to meet the latency or data model requirements of customers.
• Big Executor - BigQuery and DataFlow Job executor that lets users define their data flows using SQL, opening up data pipeline creation to non-engineers.
• SuperConductor - General Purpose data loading system that loads well defined data into pluggable backends for querying by LDC. Supports streaming and batch inserts as well as streaming data compression.
• Automatic Alerting - Will allow users to select the metrics that are important to them and uses models to highlight data outliers so users will be notified of any unexpected product behavior early.

Current/Upcoming projects:

Twitter has broadly recognized the platform as where Twitter as a whole wants to go for data analytics. To that end, we need you to help us expand the platform to serve an ever broader set of use cases.

• Data Aggregator - Beam based, configuration driven aggregator that let’s users define their Aggregations and dimensions that can run against batch and streaming data sets. Allows users to define their aggregations without redeployment.
• LDC Views - Build the ability to define custom views on the queries. Will allow users to define business logic and ensure consistent views on data regardless of backend.

What Else You Can Expect:

We believe that people are our strongest asset, and take special interest in each and every one of our engineers. You can expect a real investment in your professional growth, and a dedication to crafting a strong, sustainable career for you. Our teams are composed of highly talented and passionate 'A' players, and you’ll get the mentorship and coaching you deserve to hone your skills and expertise. Our vibrant culture and high expectations will kindle your passion and bring out the best in you!

In particular, we are an experienced team of senior engineers that work across batch, streaming, and serving systems to deliver solutions to a diverse set of problems here at Twitter. We move quickly and we expect to make an impact. If this describes your approach to writing software, apply to join us!

Who You Are:

• Industry experience working in a distributed computing environment.
• A strong passion for building tools and systems that enable other developers on your team to build great things.
• Proven strengths in algorithms, complex data structures, and dynamic programming.
• Previous experience with cloud server environments such as Amazon Web Services (AWS) and/or Google Cloud Platform (GCP).
• An understanding of continuous integration and a strong drive toward automation.
• Well versed in multi-threading, scaling, concurrency, caching, queuing, RPC, and distributed data storage.
• Ability to thrive in a dynamic, fast-paced, and collaborative environment.
• Strong attention to detail.

For Bonus Points, You Would Also Have:

• Experience administering production systems (preferably Linux) and a general awareness of DevOps best practices.
• Experience with the Lambda architecture and different ways of implementing it.
• Contributed / committed to a well-known open source project and/or interest in doing so in the future.
• Demonstrated ability to excel in whatever you pursue (whether it's work, school, competitions, open source contributions, personal projects, etc. you've always stood out and succeeded).