Position Summary

This individual will contribute to the evolution of the enterprise DataHub/Delta Lake architecture, driving best practices in cloud-native, Spark-based data engineering. The ideal candidate has hands-on expertise with Databricks, AWS Cloud services, and Apache Spark (Python/Scala), and can serve as both a technical leader and mentor for junior engineers.

What You Will Do:

• Architect and build robust, scalable, and secure data pipelines leveraging Databricks, Apache Spark, and AWS Cloud (EMR, Redshift, S3, Glue, Lambda).
• Participate and represent the data team in critical design discussions with technical leads across various product lines.
• Collaborate closely with application developers, product managers, and business analysts to translate requirements into data models, ETL/ELT workflows, and analytics-ready datasets.
• Conduct pull request reviews and enforce engineering excellence in code quality, testing, and performance optimization.
• Troubleshoot and optimize production data workflows while ensuring observability, resilience, and cost-efficiency at scale.
• Research, evaluate, and apply emerging tools and technologies to continuously modernize the data engineering ecosystem.
• Act as a mentor to junior engineers, fostering a culture of collaboration, innovation, and continuous learning.

What You Will Bring:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
• 7–10 years of professional data engineering experience, including at least 3+ years with modern cloud-based data lake architectures.
• Deep expertise in Apache Spark (PySpark, Scala, or Java) for large-scale distributed data processing.
• Strong experience with Databricks for collaborative data engineering and advanced analytics.
• Hands-on experience with AWS services including EMR, Redshift, S3, Glue, Lambda, IAM, and related cloud-native data tools.
• Proficiency in Python (preferred) as well as Java or Scala.
• Strong understanding of data modeling, data pipelines, and workflow orchestration (Airflow, Step Functions, or similar).
• Solid foundation in algorithms, data structures, and software engineering best practices.
• Excellent communication skills and a proven ability to work cross-functionally with product and engineering teams.

You'll Benefit From:

At Careforth your well-being matters. With flexible schedules, a remote-first culture, and a nationally recognized wellness program, our benefits are designed to help you thrive, both professionally and personally. Discover how we invest in you: https://careforth.com/careers/#benefits

The pay range for this position is $125K - $185K. The actual wage offered may be lower or higher depending on budget and candidate experience, knowledge, skills, qualifications, and geographic location.

#LI-Remote (exceptions NYC, CA, and CO)