Databricks Data Engineer, Senior

The Opportunity:

You will be responsible for designing, building, and maintaining robust data pipelines within Databricks. Your key tasks will include creating and optimizing end-to-end data flows, handling ETL processes, designing data models, ensuring data security and compliance, collaborating with cross-functional teams, and continuously improving performance and scalability. Your expertise will bridge the gap between raw data and actionable insights, enabling efficient data processing and analytics across the business.

Here, you'll mentor a multi-disciplinary team of analysts, data engineers, developers, and data consumers in a fast-paced, agile environment. You'll use your expertise in analytical exploration and data examination while you oversee the assessment, design, building, and maintenance of scalable platforms for your clients.

Work with us to use data for good.

Join us. The world can't wait.

You Have:

• 8+ years of experience in designing and developing ETL processes and managing data pipelines
• 3+ years of experience with Databricks, including in building and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration
• 3+ years of experience with writing Python and PySpark code for ETL scripts and custom transformations
• Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS
• Knowledge of Delta Lake, Databricks Delta, and Unity Catalog, and performance optimization techniques for ETL jobs, such as partitioning, bucketing, and caching
• Ability to develop and deploy ETL processes with limited visibility, utilizing comprehensive logging and robust error handling mechanisms to troubleshoot and resolve issues
• Ability to write SQL queries on a PostgreSQL database and troubleshoot and improve query performance
• Ability to design and implement data models, define schemas, and optimize data structures for efficient querying and analytics
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements
• Bachelor's degree in Computer Science, Computer Engineering, Information Systems, or Information Technology

Nice If You Have:

• Experience in cloud-native data services such as AWS Lambda or Google Cloud Functions, leveraging serverless architectures for data processing
• Knowledge of data governance practices, data lineage, and compliance requirements, such as GDPR or CCPA, ensuring data quality, privacy, and security
• Knowledge of tools like Apache Atlas, AWS Glue Data Catalog, DVSum, or Collibra for managing metadata, data lineage, and cataloging data assets

Vetting:

Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client.

Compensation

At Booz Allen, we celebrate your contributions, provide you with opportunities and choices, and support your total well-being. Our offerings include health, life, disability, financial, and retirement benefits, as well as paid leave, professional development, tuition assistance, work-life programs, and dependent care. Our recognition awards program acknowledges employees for exceptional performance and superior demonstration of our values. Full-time and part-time employees working at least 20 hours a week on a regular basis are eligible to participate in Booz Allen's benefit programs. Individuals that do not meet the threshold are only eligible for select offerings, not inclusive of health benefits. We encourage you to learn more about our total benefits by visiting the Resource page on our Careers site and reviewing Our Employee Benefits page.

Identity Statement

As part of the application process, you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

Work Model
Our people-first culture prioritizes the benefits of flexibility and collaboration, whether that happens in person or remotely.

• If this position is listed as remote or hybrid, you'll periodically work from a Booz Allen or client site facility.
• If this position is listed as onsite, you'll work with colleagues and clients in person, as needed for the specific role.

Commitment to Non-Discrimination

All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, local, or international law.