Senior Data Engineer
|Job Title:||Senior Data Engineer|
|Job Published:||March 05, 2021 16:16|
Our client is a global leader in technology solutions and has an unrivalled market reputation. In recent years our client has led the way in producing novel and intricate solutions focused on AI and Analytics. With IP garnered from our client's global partners this business is well positioned to have a stratospheric period of growth during 2021 and beyond.
With a reputation predicated on quality of service to clients and offering un-paralleled career opportunities to its employees, this company has become a destination of choice for the very best talent in Australia.
As a Senior Data Engineer in the Advance Analytics division, you will be responsible for designing and deploying scalable, enterprise grade data platforms.
You will contribute to a variety of technologies, depending on the requirements of the business based in Sydney, Melbourne and Brisbane.
Projects will include but not be limited to advanced analytics, data enablement and machine learning modelling.
Additional responsibilities will include but not be limited to:
- Use engineering principles to design, develop and implement real time decision making solutions for a range of b2c products;
- Develop, deploy, and operate scalable data pipelines, analysis of large scale data sets using a range a cloud based technologies
- Design, build, operate relational and non-relational databases (SQL and NoSQL), integrate them with Data Warehouse solutions, ensure effective ELT/ETL, OLTP, OLAP processes for large datasets.
The successful candidate:
The Senior Data Engineer will have demonstrable experience of working in a complex data warehouse environment working with Big data technologies, applying advanced analytics, and statistical modelling.
The must haves for this role include:
Deep understanding of data engineering, ideally having worked with traditional data warehouse solutions but having made the transition to cloud based real time big data platforms
Design, build and launch extremely efficient and reliable data pipelines to move data across several platforms including Data Lake, online caches and real-time systems
Proven experience working in devops environments, ideally with azure devops, bitbucket or bamboo
Hands on experience with SQL, Python and Pyspark and API development
Real time data pipeline build experience using Pyspark, Spark or Scala
Data warehouse experience including knowledge of data warehouse tables, stored procedures and SQL concepts. Exposure to Cognos is highly desirable
Experience of data modelling and data analysis/manipulation would be advantageous
Proven experience working with cloud solutions, ideally with Azure utilising data factory and data bricks
You will be a motivated self starter who enjoys dealing with a myriad of internal stakeholders and will be able to coach senior executives along the data engineering journey
What's on offer?
Due to current covid working conditions this role is open to remote working for people meeting the above criteria and as such can be done anywhere within Australia.
The position is permanent and will pay an annualised base salary of $130,000 - $150,000 base plus super and bonus.
We guarantee that all candidates who apply for this role will receive a call from the Troocoo team if the following must haves are met:
- Full working rights in Australia
- 7 + years of data engineering experience
- Exposure to SQL, Python, Pyspark, Azure, Azure Data factory, Databricks, Cognos
- Proven experience working in devops environments