To apply for this job you must first either login or register

Data Engineer

San Francisco, California  - Permanent



Job Description

Our client is the world's largest discovery and monetization platform that is single-handedly powering the business of mobile games. They work with 90% of the top-grossing applications and are being leveraged by over 300K+ games. They have over 1 billion active players around the world every month. This platform has a real impact on people's lives and their businesses and they are proud and excited about what they've built.

If you're looking for an opportunity to work with over 5 billion+ events per day, and terabytes upon terabytes of data, then you've come to the right place. If you are excited about real-time aggregations over rolling time windows, blazing fast computation speeds across lots of data, data analysis, data science and data exploration, then you're still in the right place. If you want to work on the newest big data tools such as Spark, Scala, Hive, Python, Airflow, and love to think about real-world problems then you're definitely in the right place.

Our client is looking for a data engineer to join their team and help build and maintain a data platform that will support exploration, data analysis, aggregation, and API's to deliver significant impact to both their developers and internal customers. You will be working with hundreds of TB's of data on a daily basis.


Our client's technology stack includes but is not limited to: Spark, Presto, Hive, Kafka, Cloudera, Redshift, Dynamo, Cassandra, Airflow, HDFS, HBase, Scala, Python, MySQL, MongoDB, AWS, Git, Spinnaker, and Puppet.

Some of your day to day responsibilities will include:
-Developing reliable data pipelines that will convert data into powerful signals and features
-Designing real-time and batch data processing pipelines for different systems to consume
-Developing tools to monitor, debug, and analyze data pipelines
-Designing and implementing data schemas and models that can scale hundreds of millions of users
-Collaborating closely with the product team to continuously build new features and infrastructure


Special Perks:

-Opportunity to join a strong foundation and work with the founding team to continue growth
-Stock Options
-Comprehensive Medical, Dental, and Visual insurance options
-Daily catered lunch, fully stocked kitchen
-Commuter Program
-401K
-Flex Vacation/Unlimited Vacation


Must Have Skills:

-2+ years of experience in Data Warehousing, and SQL
-2+ years of experience with any/all of Java, Scala, Python
-Experience with data pipelines in Hive, HDFS, processing larger than 10TB of data is a +
-Experience in improving performance of queries and data jobs, scaling the system for exponential growth in data
-Expert debugging skills and an emphasis on automated tests to build high-quality systems
-Experience with Spark
-BS or Computer Science Degree or related field


Details:
Starting: ASAP
To apply for this job you must first either login or register