In search of green AI

This is blog 17 in the blog series about Green IT.

The previous blog post outlined how AI uses shocking amounts of energy and water. A lot of research is therefore being done to find (more) sustainable ways to apply AI. In this blog post, we examine the possibilities for green AI. We start by looking at the Four Ms of machine learning, a model that offers important guidelines for the reduction of AI energy consumption. We then transpose those insights to the heritage sector and offer advice on how to use AI more sustainably.

The Four Ms of machine learning

According to researchers from Google and UC Berkeley, there are four factors that could influence the energy use of AI models in the future. (1) A simple mnemonic: they all start with the letter M.

1. Model: the machine learning model addressing the AI problem.

2. Machine: the computer hardware that the model runs on. Together, these determine how long a training phase takes and how much energy it uses.

3. Mechanisation: the data centre where the hardware is housed. This determines how efficiently energy is delivered to the machines inside the data centre.

4. Map: the geographic location of the data centre. This determines how clean the energy supply is. The cloud makes it easy to choose the greenest location – Microsoft is even experimenting with data centres on the ocean floor for better cooling. (2)

The 4M model is proving its worth: Google’s Generalist Language Model (GlaM), which was developed with these four factors in mind, was able to reduce CO2 emission by a factor of 154! The same study showed that optimised hardware can perform complex calculations fifty times faster than conventional hardware, using only half to a third of the energy. (3)

A new chip design

The AI sector is constantly looking for ways to improve the Four Ms. Nvidia, the world’s largest supplier of graphics chips to run generative AI, announced a new generation in November 2023. These more powerful chips improve the performance of AI applications. (4)

All the major tech companies are researching energy-efficient AI chips – the internet is littered with news headlines announcing faster, more powerful new chips, as everyone strives to achieve best possible performance. A study worth emphasising here is the following, conducted by IBM research in collaboration with universities in Switzerland and the United Kingdom.

Since data is stored in memory outside the chip, it constantly has to be ferried back and forth between the memory storage and the processing unit, i.e. the chip. This is also known as the Von Neumann Bottleneck. Continuously moving data around uses a lot of energy, and solving this issue will require a rethink of the electronics involved. In concrete terms, data storage and computations will need to happen in the same place. With that in mind, IBM has developed a neural chip called North Pole. (6)

Although the initial results are promising, a lot of further research is required. The research up to this point has focused on image recognition and classification, but the next step will be to apply this technology to generative AI, which is all about understanding and processing language.

Recycling algorithms

Aside from the development of more efficient hardware, attention is also being devoted to the algorithms we use. Not all algorithms use the same amount of energy, so by programming algorithms differently, we can save energy. Energy consumption can also be reduced by reusing algorithms. This means that the software does not require as much training, because the algorithm can build on existing knowledge. (7)

Simplify the AI model to use less data

Luis Cruz, a researcher at Delft University of Technology, considers simplification the preferred strategy to reduce AI energy consumption. “Not all the data you use to train the model make a real difference,” he explains. “So instead of dumping all the data you’ve got into the training process, a clever data selection strategy that only collects data that matters for the model and application at hand will save a lot of energy.”

He also remarks that “[t]here is a trend of continuously retraining the model as new data become available. We should ask ourselves, is it really necessary? Can we come up with a more energy-efficient solution for users who really need the new data?”

Finally, Cruz points out that “[o]nce the model has been trained, energy can also be saved by reducing the complexity of the model. This means training the most complex AI model you can, and then coming up with a simpler model that behaves the same but only learns the parts of the model that really matter. All obsolete neurons are removed. This could reduce the model complexity by a factor of ten while maintaining high performance. In this way, you reduce energy consumption at no cost.” (8)

Simulating our energy-efficient brain

Research into more efficient AI often looks at the way the human brain works. Compared to computers, our brain uses very little energy, and yet it grows in intelligence from the day we are born. With this in mind, Dutch researchers of the Research Institute for Mathematics and Computer Science in the Netherlands developed an energy-efficient algorithm that can reduce the energy consumption of certain AI applications by a factor of 1000. Like the human brain, this software works with spikes – short pulses – instead of constantly transmitted signals. (9)

Recommendations for the heritage sector

The Four M's provide points of reference for heritage institutions to ask the right questions and weigh considerations when using AI for a specific project. Below are a few recommendations.

Evaluate the CO2 impact

• When implementing an AI model, take a moment in advance to check whether any information is available about the model’s CO2 impact. There is a growing call for an obligation to report on the energy usage of all applications of AI models. But this movement will only really gain momentum when users start asking questions.

• At the start of a project, set the condition that the CO2 impact has to be measurable. There are various calculators (including free ones) that can provide information about the impact of your project. You will need the supplier of your AI to give you the technical specifications of the data centre that hosts your AI application. Include this in the requirements that you set in advance of the project.

• If your supplier refuses to provide this information, at least try to find out the geographic location where your AI model is running, so that you can estimate to what extent the use of green energy compensates for your project’s emissions.

Smaller data sets

• Limit your energy consumption by investigating whether the AI model can be adequately trained with a smaller data set, and whether AI models already exist that you can use. There are various online libraries containing reusable algorithms.

Transparency and documentation

• If you are writing your own algorithm, record how the model is structured so that others can reuse it. LinkedIn has a detailed explanation of how to do this. (11)

At all times, though, the most important question to ask is: are the CO2 emissions caused by your use of AI worth the purpose for which you use this technology? This principle is similar to the point we made about data centres: no matter how green you make them, in the end they still consume energy that cannot be used for something else.

In the next blog series, I will address the dilemmas that you, as a user, encounter when applying Green IT principles in practice.

Sources

1. Patterson, D., 'Reducing the carbon emissions of AI', published on EOCD.ai on 12 April 2022.

2. Read more about Microsofts onderwater datacenters (project Natick).

3. Patterson, D. et al., 'The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink' on TechRxiv, last viewed on 23 November 2023.

4. AG connect, 'In GPU-schaarste door AI-hype komt Nvidia met volgende generatie AI-chip', 14 November 2023.

5. Marelli, V. ‘Hoe vermijden we dat AI uitdraait in een ecologische ramp?’, published on 10 May 2022.

6. Murphy, M., ‘A new chip architecture points to faster, more energy-efficient AI’, published on 19 October 2023 at https://research.ibm.com/blog/northpole-ibm-ai-chip.

7. Kreijveld, M., ‘Hoe de carbon footprint van kunstmatige intelligentie kan worden gereduceerd’, published on 16 June 2019, updated on 5 October 2020.

8. TU Delft stories, ‘Duurzame kunstmatige intelligentie: van ChatGPT naar groene AI: een interview met Luiz Cruz’.

9. Yin B., et al., ‘Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks’, published on 28 July 2020.

10. Two freely available calculators are CodeCarbon.io and The Green Algorithms Calculator. You can use CodeCarbon.io to determine in an easy way how much energy is consumed when using a computer for a specific application or AI-model. Based on the time during which the AI model operates, The Green Algorithms Calculator calculates the type of cores (processor cores), the number of GPUs (graphic card), the actual AI model, the available memory and the used server or platform, and the CO2 impact of your project. In addition to the results, you also receive comparisons with for instance the number of trees you ought to plant, how this translates into road mileage or number of flights.

11. LinkedIn, 'How do you document your algorithms for reuse?', last updated on 23 August 2023.