Based in Brisbane or Melbourne, the Senior Data Engineer contributes to the development and improvement of data acquisition, usage and distribution through developing data pipelines and applying data governance, and data visualisation techniques, working alongside data science, product management, software engineering, and the forecasting team.
In order to expand and capitalise on these data opportunities the Senior Data Engineer will operationalise data pipelines and data products in scalable cloud environments.
Major Responsibilities & Accountabilities:
- Provide leadership through guiding other data engineers/scientists & software engineers on software engineering best practices, scalable cloud architectures that focus on greater functionality and flexibility in our data systems;
- Own the designs and implementations of data pipelines, and data models, and advise on good practices for operating at scale;
- Work as part of a team of data engineers, specialists and software engineers to streamline and automate the management & control of data for analytical and operational processes;
- Build and automate the infrastructure required to support data pipelines across a wide variety of data sources;
- Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs;
- Responsible for operationally supporting data management tools and data solutions.
- Experience in developing reliable, high quality, well tested software solutions
- Proficient experience within the AWS Cloud environment with data services such as AWS S3, AWS EC2, AWS Lambda, AWS Kinesis, AWS Glue, AWS Athena, AWS
- Hands-on, proficient experience with programming in Python and PySpark building data management solutions and; working with business intelligence and visualisation tools (e.g. Tableau, Power BI);
- Strong technical proficiency in SQL, NoSQL and experience working with queues & event streams;
- Strong experience with Windows, Linux, Unix environments and with a variety of database technologies;
- Knowledge of complexities and challenges with ETL/ELT solutions and data flow processes;
- Ability to present and communicate complex technical issues to a variety of audiences including senior management;
- Degree in computer science or related discipline.
If this sounds like a role that you are interested in and suited to, then we can't wait to hear from you! Please apply directly or for further information, reach out to Martin Castle at martin@troocoo.com or 0479 095 264.