What lies ahead for AI?

Where are we heading with AI? Are we hurtling headlong towards a horrific future with mass unemployment, or are we on track to a utopian society where AI handles the mundane, freeing us to pursue creativity and innovation?

Depending on who you speak to or what you read, the picture can vary dramatically. And it is confusing isn't it, with dire predictions of doom and gloom on one end, and promises of a boundless golden age on the other?

I won't even pretend to know the answer. But being in the business of writing – a niche directly threatened by generative AI – does concentrate one's mind. This means I read as widely as I can while trying to leverage generative AI as much as possible for writing.

Here are some observations I've made.

The limits of generative AI

Casual observers would have noticed how quickly AI models are growing, typically measured by the number of parameters. This is related to the volume of training data and can be credited to the discovery outlined in 2020 by OpenAI researchers in the paper: "Scaling Laws for Neural Language Models."

The seminal paper proved that performance in large AI models improves predictably as the size of the model, dataset, and computation are scaled up. This finding influenced the design and development of ever-larger AI models, prompting OpenAI and other tech giants to pour previously unthinkable resources into training gigantic AI models with millions then billions of parameters.

The unspoken caveat: There is a finite amount of human-generated data available. Even if the scaling law holds up as we move to 100,000 GPU clusters and beyond, we are already running out of data. And increasingly, a growing body of research shows that AI trained on AI outputs eventually devolves into gibberish.

I do expect more jaw-dropping AI models to be released in the months and years ahead. However, AI's insatiable demand for fresh human data makes it improbable that this trend of exponential growth can continue indefinitely.

Finding that elusive use case for AI

Enterprises have raced to deploy AI in hopes of gaining an edge over their competitors. Widespread success appears elusive for the moment though. Indeed, the only ones making money hand over fist appear to be technology firms offering AI consultancy services. Will AI have a broad impact, or will it be limited to specific niche applications?

Some argue that the second and third-order effects of AI will change industries in ways that are impossible to predict. The thinking goes like this: Initial successes will lead to more demand for AI and data science professionals, which will in turn give businesses a substantial boost in efficiency and innovation, and even open up new market opportunities and – in a ripple effect that will benefit multiple sectors.

Yet it is worth remembering that AI as a technology is no respecter of human morals or ethics. While there are clear benefits of AI in areas such as code generation, AI can just as easily be used for nefarious purposes. This week, I wrote about how AI tech can now allow scammers to impersonate anyone on video calls with a single photo.

I do not doubt that businesses will find innovative ways to harness AI as new use cases are discovered and the technology matures. It might even help us do work faster and better, too. Whatever it is, just expect the scammers to get there first.

More data centres

Love it or hate it, the AI boom is driving a surge in electricity consumption globally. And it is directly and indirectly leading to the construction of more, massive data centres around the world, including a massive, once-a-generation build-up of data centres in Johor.

Within the data centre ecosystem, projections that I've seen from data centre operators and equipment suppliers are that there is simply no future where AI doesn't feature prominently. Yes, I said earlier that the exponential growth of AI models is unlikely to be maintained indefinitely.

However, it is worth noting that everyone agrees that AI inference will exceed AI training in the future. This means that while the initial training of AI models requires significant computational power, the ongoing use and application of these models will demand even more resources. With this in mind, it is imperative that we build more sustainable data centres.

Where do you think we are headed with AI?