Senior Data Engineer

Senior Data Engineer – (employer is Grocery Delivery E-Services USA Inc. d/b/a HelloFresh) - Design, implement, and manage scalable and optimized data architectures using Snowflake and AWS services. Collaborate with stakeholders to align data architecture with business goals, ensuring it can support both current and future needs. Build and manage robust ETL/ELT pipelines to extract, transform, and load data from diverse sources into Snowflake on AWS. Use tools such as AWS Glue, Apache Spark, Apache Airflow, or Lambda functions to orchestrate and optimize data workflows. Ensure data pipelines are secure, reliable, and performant. Assist with data migrations from on-premises systems or other cloud platforms to Snowflake on AWS. Develop integration strategies and ensure seamless integration of data from multiple sources. Implement processes for continuous ingestion of data streams, leveraging AWS services such as Kafka. Fine-tune Snowflake queries, optimize warehouse usage, and configure AWS services for cost-effective data processing. Monitor and improve data performance, using monitoring tools within AWS or third-party services. Adjust and scale data architectures based on workload requirements and query performance metrics. Implement security best practices within Snowflake and AWS, including IAM roles, encryption, and data masking. Ensure data systems meet compliance standards (e.g., GDPR, HIPAA) for data privacy and security. Set up automated monitoring and alerts for suspicious data access patterns. Collaborate with Data Scientists, Analysts, and Business Intelligence teams to ensure they have access to well-structured data for analysis. Work with IT and infrastructure teams to ensure proper provisioning and resource allocation on AWS. Provide technical guidance to junior data engineers and conduct code reviews. Document all architecture, data flows, and pipeline processes for easy maintenance and knowledge sharing. Provide training sessions or workshops for team members on Snowflake and AWS best practices. Develop best practices for data management, data modeling, and data governance. Requirements: Bachelor’s degree (US or foreign equivalent) in Computer Science, Computer Engineering, Information Systems, Management Information Systems, or a related field and five (5) years of experience in the position offered or in a related role. All of the required experience must have included experience with: building, testing, and maintaining production-ready data pipelines; Python and PL/SQL; data processing using Spark; working with distributed ecosystems including Hive, Sqoop, MapR, and HDFS; ingesting, consuming, and processing data using both batch and real-time streaming platforms; designing, developing, and maintaining ETL processes and data pipelines; cloud technologies including Azure and AWS; cloud-based data infrastructures, data warehousing and database management solutions; SQL optimization tools including Redgate and Idera; data processing and transformations using various data formats and data processing libraries; Business Intelligence and Reporting tools for data visualization, reporting, and actionable business insights; and version control and CI/CD tools. Telecommuting is permitted from any U.S. location. Salary: $175,947/year. E-mail résumé to: meggan.bird@hellofresh.com. Ref. #128.