Data Engineer (Azure, ADF, Big Data, ETL) (QR8914) , Amsterdam

Omschrijving:
Functie: Data Engineer (Azure, ADF, Big Data, ETL) (QR8914)
Start: 4-7-2022, 32-40 uur per week
Periode: 3 mnd+
Omgeving: Omgeving Amsterdam

Omschrijving:
For our client in Amsterdam we are looking for a Data Engineer (Azure, ADF, Big Data, ETL)

The person should have experience in technologies for distributed data computing: ADF, Azure Data Lake, Azure Data Lake Analytics, Azure database, Azure data warehouse, Delta Lake and Azure DataBricks.

Should have:
• Should have in-Depth knowledge and experience in design, development of Big Data projects that perform large scale Distributed Data Processing/Analytics Microsoft Azure.
• Experience in roles of Big Data Developer, ETL Lead – specializing in Apache Spark, Kafka and Hbase.
• Developed applications using Pyspark and Spark-SQL for data extraction, transformation and aggregation from multiple file formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
• Experience in Data Platform: Azure Databricks, Azure Data Lake
• Experience in Data Pipelines/Workflow Management Tools: Apache Airflow, Azure Data Factory
• Knowing how to code using the object-oriented paradigm(See below for programming languages)
• Experience in programming/coding with Python, PySpark, SQL, Scala, Shell Scripting and building Python packages/wheels
• Experience with using file formats and technologies like Apache Arrow, Parquet and Delta Lake
• Experience in automation of deployments (Azure and On-Premises) using using CI/CD tools like Jenkins, Github, Azure DevOps, Github Actions
• Knowledge on Azure cloud principles like IAM, Active Directory, PIM etc.
• Experience in Data visualization tools like Power BI, Tableau
• Experience in working with Relational Databases(RDBMS) and No SQL Databases
• Experience in logging & monitoring is preferred, not required
• Worked with Test, QA and DevOps teams to automate the test cases and integrate them with scheduled auto deployment pipeline to increase test coverage.
• For Senior Roles, need to understand Cost management in Azure.
• Working experiences in Agile Scrum team.


Trefwoorden: Data Engineer, Azure, ADF, Big Data, ETL



OPDRACHT​GEVER:

bedrijfsnaam:
Qualogy
contactpersoon:
Marloes
type:
ZZP, freelance, interim vacature
locatie:
Amsterdam
provincie:
Noord-Holland
uurtarief:
Marktconform
start project:
Asap
referentie:
ITC-QR8914
duur opdracht:
3 maanden
uren per week:
40 uur
publicatiedatum:
10-06-2022 11:49:03