Data Engineer, Baton Rouge, LA
We are looking for a Data Engineer for a direct hire position located in Baton Rouge, LA.
Summary:
As an Azure Data Engineer, you will be responsible for the movement and capture of data across varied platforms and data stores, designing, building, and managing data pipelines and data architectures on Microsoft Azure. You will work with large datasets to provide business insights, assist with data integration, and optimize data solutions across various cloud services. You will collaborate with cross-functional teams to ensure data solutions meet organizational needs and are scalable and cost-efficient.
Responsibilities:
• Data Architecture Design: Design and implement scalable, high-performance data pipelines and data lakes on Azure using tools like Azure Data Factory, Azure Synapse Analytics, and Azure Databricks.
• ETL/ELT Pipelines: Develop and manage ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) workflows using Azure Data Factory and other Azure-based tools.
• Data Modeling: Implement data models using Azure SQL Database, Azure Synapse Analytics, and other cloud-based data storage options.
• Data Integration: Integrate data from various sources such as on-premise databases, cloud-based data, and third-party services using Azure Integration Services.
• Cloud Data Solutions: Leverage Azure services like Azure Data Lake, Azure Blob Storage, and Azure SQL Data Warehouse to design cloud-based data architectures that are efficient and scalable.
• Performance Optimization: Optimize query performance and ensure efficient storage and processing of large data sets.
• Data Security and Governance: Implement data security practices and ensure compliance with data privacy regulations. Use Azure security features to protect sensitive data.
• Collaboration: Work closely with data scientists, business analysts, and other stakeholders to ensure data is delivered in a usable and accessible format.
• Automation & Monitoring: Automate data pipelines and monitor performance to ensure reliability and efficiency. Set up monitoring using Azure Monitor and Application Insights.
Required Skills and Experience:
• Proficiency in Azure Services: Experience with core Azure services like Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure Blob Storage, Azure SQL Database, and Azure Data Lake.
• ETL Development: Strong knowledge of ETL frameworks and techniques, including using Azure Data Factory and/or custom data transformation scripts (e.g., Python, SQL).
• Data Modeling: Experience designing relational and non-relational data models.
• SQL and Data Querying: Proficient in writing complex SQL queries and optimizing them for performance in cloud environments.
• Cloud Data Integration: Familiar with integrating data from on-premise systems, cloud data warehouses, and external APIs into Azure-based data storage solutions.
• Data Security: Knowledge of data security practices and data governance principles, including encryption, access control, and compliance standards (e.g., GDPR).
• Automation and Monitoring Tools: Experience with Azure Monitor, Azure Logic Apps, and automation of data workflows.
• Big Data Technologies: Familiarity with big data processing tools such as Azure HDInsight or Databricks for managing large-scale data sets.
• Programming/Scripting: Familiar with scripting languages such as Python, Scala, or PowerShell for data transformation and automation tasks.
• Version Control: Experience using Git or other version control systems for managing code.
Preferred Experience:
• Experience with Data Warehousing: Familiarity with implementing and managing large-scale data warehouses in the cloud (Azure Synapse Analytics).
• Machine Learning/AI: Experience integrating data science and machine learning models in Azure environments (using Azure Machine Learning, Databricks, etc.).
• Agile Methodologies: Familiarity with Agile/Scrum methodologies for project management and development cycles.
• Collaboration Tools: Proficiency in collaboration tools like Azure DevOps, Jira, and Confluence.
Education & Certifications:
• Bachelor’s Degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
• Certifications (Preferred):
o Microsoft Certified: Azure Data Engineer Associate
o Microsoft Certified: Azure Solutions Architect Expert
o Microsoft Certified: Azure Fundamentals (for beginners)
o Other related certifications (e.g., Big Data, Cloud technologies)
