loginTroocoo has partnered with a well known Government Agency to assist in finding a skilled Data Engineer to join their Strategy, Innovation & Insights division.
As part of the Research and Analytics team, the successful Data Engineer will be responsible for:
- Designing, implementing, optimising and managing data ingestion on cloud platforms(e.g. Azure) with an emphasis on security and reliability. This includes data migration from existing systems.
- Developing, implementing and managing security features in data warehouses in accordance with Data Governance Frameworks and/or policies.
- Managing data infrastructure in a CI/CD (Continuous Integration/Continuous Development) pipeline with appropriate controls and documentation.
- Building and sustaining multiple ingestion pipelines with ETL (Extract, Transform, Load) tools as per defined business and user requirements with an emphasis on quality.
- Leading the ingestion of data from a range of different systems and databases into the data warehouse (e.g. Microsoft Access, SQL Server, Oracle, SAP)
- Assisting with the planning and coordination of ICT integration testing and verification activities (e.g. User Acceptance Testing (UAT)) and providing support to business stakeholders to achieve successful UAT outcomes.
- • At least five (5) years demonstrated experience in data engineering, with a focus on analytics in the context of a large scale, complex business change program/project. Public sector experience is desired but not essential.
- Demonstrated track record of manipulating, processing, and extracting value from large, decoupled datasets, with attention to detail in data quality, wrangling and validation.
- Required technical skills:
- Extensive experience in setup, migration and management of Data Warehouses in the Cloud (e.g. Azure).
- Advanced skillsin Microsoft Azure Data Factory, Microsoft SQL and Python scripting.
- Data mart setup based on use cases
- Advanced skill in NoSQL (desirable)
- Power BI report creation and secure sharing.