I’ve always loved computers. I remember when our family got our first computer - a 386 built by my uncle’s friend. I couldn’t get enough of it. My dad figured out a way to make a 20-second cartoon in DrawPerfect, and I just couldn’t believe my eyes; it felt like a door of possibility had burst open. I used to spend my time learning DOS commands and trying to make little programs in QBasic, though I never got very far.
My interest in tech continued throughout my early life, and I was always the go-to person for computer-related stuff in my family. When I was in high school, I remember noticing a group of IT people working on our school’s computer systems and was amazed that that could actually be a job. It looked like way too much fun.
I eventually did find myself in the industry and have had a pretty diverse career, starting from working the night shift on a Hewlett Packard help desk, to shipping code for some of Australia’s largest IPTV product launches.
In the last few years, I’ve been focusing a lot on Data and Machine Learning, working as a CTO at an Influencer Marketing startup called Scrunch, and recently becoming a Data Engineer at Thoughtworks.
In Brissy, I love being involved in the local data community, helping to organise Meetups as well as mentoring and tutoring at community Machine Learning and Deep Learning classes.
What does a Lead Data Engineer actually do?
Data Engineer is a pretty broad term and can mean a lot of different things to different organisations. In general, it’s about helping an organisation get value out of their data. Sometimes that work is about building and maintaining data ingestion pipelines, other times it’s about optimising database queries and anything in between.
My specialty is in developing and managing the lifecycle of Machine Learning products, which involves collecting and preparing training datasets, training and evaluating models and deploying and monitoring the performance of models in production.
What does your typical day look like?
Depending on the client and the level of seniority of the team, my day usually begins by syncing with the team, making sure everyone knows what they are working on and why, and that we’re all moving in the same direction. I usually spend a large part of the day pairing with other developers and sometimes coding solo as well. I’ll dedicate a few hours to non-coding tasks like working on architectural diagrams, presenting to stakeholders, dealing with major issues or whatever else needs doing that day.
I manage my time mostly by handwriting a to-do list at the start of every day. I always try to ensure I include a task that will improve our experience in some way: whether it be refactoring some ugly code, introducing automation to laborious manual tasks or improving our documentation.
What’s the most challenging part of your job?
Dealing with interruptions is always a really tough one. Some of the challenges we work on require a lot of focus, but at the same time, there’s usually a lot of stuff going on that involves constant unpredictable interruptions, especially for a senior member of the team.
We have experimented with assigning dedicated time to focused work, and other time for spontaneity and collaboration. It’s not a complete panacea to this problem, unfortunately - I’m still trying to figure it out.
What’s your favourite part about your role?
Learning! I can’t get enough of it. I love the experience of starting with a new concept and having it all feel so alien. Then slowly, as you keep pushing, you start to feel the pieces falling into place, and some of it starts to make sense. Then one day, it all clicks, and the concept suddenly feels so natural, even intuitive, like you’ve known it your whole life. I’m also really fond of taking the lessons learned from taking that journey and using them to help others learn it too.
What are you measured on?
At Thoughtworks, the main performance measurement is feedback. Feedback from your peers, clients, and superiors. It’s not just about receiving positive feedback, but hearing constructive criticism and taking action to improve.
What are some of the industries your clients come from?
Government / Not-for-profit
...to name a few.
What makes a good Data Engineer?
Understanding the business is so important. It’s not enough to know what you’re doing; you need to know why. Who are the customers of the product? How will people interact with the product? Where does data fit into the equation? How likely is the product to change and pivot?
This is true for most technical professionals, but it’s particularly important in the world of data, where off-the-shelf solutions to the problems that organisations face are rare, and you are sometimes making decisions which will have implications far into the future.
What is good Technical Leadership?
To me, great technical leadership is about always asking the question: how we can improve? How can we deliver value to customers sooner? How can we foster a better relationship with stakeholders? How can we make the developer experience better? How can I make sure that my team members are adequately challenged but not burning out?
I think it’s also important to set a good example. If unit tests are important to the project, then you need to be shipping code with unit tests. If you are advocating work-life balance, then you shouldn’t be responding to non-urgent emails when you’re on holidays.
The best tech leads I’ve worked with were always nice, approachable and humble people, happy to share their knowledge and experience to help the team grow and be the best that they can be. But at the same time, very driven to deliver successful projects.
How do you balance coding/non-coding responsibilities?
Since I love to code, it can be important to take a step back sometimes and ask if that is the best investment of my time for the team. I try to start every day and every week with the same question: how can we take a project to the next level of excellence? If that means shipping more features, then that’s where my time is allocated. If documentation is lacking, then that's what I’ll try to focus on.
On your current project, What technology has you most excited?
On my current project, we’re doing a lot of work with geospatial data, as well as streaming data pipelines using AWS Kinesis. I’ve had some exposure to both technologies but never had a chance to explore them deeply, so I’m excited about that.
‘I love being a consultant because…'
It’s a chance to be exposed to a wide range of problems, technologies, and people. I like being a consultant at Thoughtworks because their values are aligned with my own, particularly the focus on social responsibility, and I get to work with some really amazing people.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.