Fastest-growing AI skills in the Australian market

Given the current AI boom and digital skill shortage, it’s time to think about pathways to AI and the skills needed to plug the holes.

Australia might not immediately come to mind when you think about pioneering AI technologies, but there’s some great things happening in Oz. In a world first, James Cook University is utilising AI to accurately predict sediment distribution on the Great Barrier Reef, one of the world’s most fragile ecosystems.

There’s also some really interesting projects happening at AI labs across the country including CSIRO’s Data61, Google Research Australia Hub, Australian Artificial Intelligence Institute at University of Technology Sydney, and Vision and Language at Monash University.

From a business perspective, companies are integrating AI into their systems to improve user experience. A recent example is closer to home for me, where we have built New Relic Grok, a generative AI observability assistant that allows users to interface with the platform and ask questions using natural language prompts.

For those trying to break into AI, my advice is pursue a combination of applied skills and theory. The theory is necessary to truly understand how AI models work and what assumptions they make. Applied skills are necessary to train, serve and maintain models in real-world, large-scale deployments. Below are the theoretical foundations and applied skills I recommend.

Theoretical foundations for AI

These are the theoretical foundations upon which AI solutions are derived:

• Linear algebra
• Calculus
• Probability and statistics
• Machine learning (supervised, unsupervised, semi-supervised)
• Mathematics and mathematical notation for interpreting research papers

If you are interested in pursuing AI research this foundation knowledge will be necessary.

Applied AI skills in high demand

In addition to the theory, a set of applied skills is necessary. The tools used to support these skills change every decade or so. It is essential to be proficient in using the popular tools available today and understand what role they play in the AI ecosystem.

The most in demand AI skills and fastest-growing tools used in Australia and abroad include:

• Machine learning skills encompassing Deep Learning and Reinforcement Learning. This requires familiarity with popular frameworks such as PyTorch, TensorFlow, Keras, OpenCV, OpenSpiel, Open AI Gym and Scikit-learn.
• Knowing how to interface with these frameworks requires knowledge of programming in Python, R, Julia, C++ and Java/Scala.
• Data engineering skills are essential to preparing the data used to train models and draw inferences. Data exploration, extraction, cleansing and curation followed by knowledge of big data systems (eg, Apache Hadoop, Apache Spark and Apache Kafka), data pipelines and data visualisation are necessary for managing and processing large data sets that fuel AI models.
• Developer tools for machine learning (MLOps) are essential for managing the machine learning project lifecycle. Some of the popular tools include MLFlow, Metaflow and Weights and Biases.
• Computer science skills and knowledge of distributed computing are essential for deploying, scaling and optimising AI solutions. These skills are last on my list as cloud providers are starting to offer managed solutions (eg, Google Vertex AI) to simplify this practice.
   

Finally, staying up to date is a skill in itself as the field is rapidly evolving. Here are some of the resources I use:

• Twitter - or X, as it’s now known (following AI researchers and organisations)
• arXiv (for the latest research papers)
• Books (covering AI and machine learning topics)
• YouTube (@YannicKilcher does a great job explaining papers)
• Papers with Code (providing research papers accompanied by code)
• GitHub (offering a wealth of examples and projects to learn from)
   

Image credit: iStock.com/Iryna Spodarenko