Generative AI creates a ‘Promethean Point’

As the world adopts generative AI and adapts the technology to various business needs, the flame of artificial intelligence is spreading quickly. As Prometheus discovered, the risks inherent in powerful newfound knowledge can be severe. TransRe’s Otakar G Hubschmann explores some of the risks generated by generative AI.

On 22 March 2023, a picture circulated on Twitter. It showed a large plume of dark smoke rising near a building and included the caption “Explosion near #Pentagon”. As the tweet circulated the globe gaining speed and eyeballs, the stock market dropped. Later that morning, Arlington Fire Department issued a statement saying, “there is NO explosion or incident taking place at or near the Pentagon”. The image had been the product of generative AI.

There are certain points in history when innovation, technology and timing combine in a “Promethean point” of new, transformative knowledge. Like Oppenheimer’s Promethean point, knowledge brings great responsibility. We have reached such a point with the emergence and adoption of generative AI.

The goal of generative AI is to produce new images, video, audio and text that are indistinguishable from the work of humans.

The use and abuse of the technology both increases and changes the nature of risks that (re)insurers face, and we must recognise and manage them.

The foundational research behind generative AI was published in 2014 for Generative Adversarial Networks (GANs) and 2017 for Transformers (Goodfellow et al., Vishwani et al.). Since then, increased computing power and improved model architectures have allowed generative AI to accelerate – if we use the number of model parameters as a proxy, large language models have far outpaced Moore’s law in the past five years, with no end in sight.

To begin to understand the risks, start with how large language models (LLMs) – GPT, LLaMA etc. – work. As a fire needs kindling, so too does an LLM need fuel. An LLM ingests the internet for information, breaks it down for the model to read, applies methodologies to allow the model to understand semantic context, and then stitches it all together. The model is then able to predict the next word or series of words in a given sentence. This technology allows for amazing use cases – summarise anything, query anything, learn anything.

While LLMs already provide a great amount of utility, they come with user risks, both from employing the model output as well as from the provenance of the output itself.

Generative AI hallucinations look and sound real, but are completely fake model outputs, as a lawyer in New York recently discovered when his cited case precedents were discovered to be entirely fictitious. ChatGPT had invented them all, indistinguishable (at first cursory glance) from real precedents. LLMs may return incorrect information because of inaccuracies in the training data, or because the model seeks to deliver the most contextually correct answer. Inaccurate responses may also occur when the model parameter which controls randomness in the model output, called temperature, is dialled up.

Copyright and property infringement is at an early stage. LLMs decompose inputs into something a model can understand, run the data through some more algorithms, and recompose the data at the other end. At some level, the model has seen the data, even if it does not necessarily know or even understand the data.

Diffusion and GANs models work in a similar way with audio, video and images, and a number of generative AI companies are being sued for training their models on copyrighted information (images from corporate databases, books by well-known authors, songs by popular singers).

In a recent non-AI case which may prove to inform generative AI lawsuits, Andy Warhol’s estate was sued over a painting of Prince which borrowed heavily from an earlier photograph by Lynn Goldsmith. The court ruled against the Warhol estate, stating both Goldsmith and Warhol sought the same fair use case from the picture and painting. A similar argument will be made in the upcoming copyright suits against these generative AI companies. This potential risk may have a glacial effect on the employment of generative AI.

Adversarial risks from generative AI include all manner of improved audio and video deepfakes, jailbreaking LLMs and improved business email compromise attacks. How much would it cost to buy a deepfaked audio snippet of your CEO frantically asking for a money deposit for a missed payment into their account? The answer is a 10-second recording of the CEO talking, and a few hundred dollars. In 2019, a UK energy company executive was tricked into transferring almost $250,000 to a Hungarian bank account based on a deepfaked audio call from ‘the boss’. Those funds were quickly transferred to other bank accounts and into the ether.

LLMs will enable and embolden a new wave of phishing attacks. Every business email compromise attack will be written in perfect English. Worm GPT, DarkBert, FraudGPT and other LLMs trained on dark web content will optimise for both frequency and severity of attacks.

When we look back for Promethean points, we can clearly see the ‘before’ and ‘after’. The Gutenberg Press, the Industrial Revolution, and before that the actual adoption and use of fire. To that list, we should now add generative AI, which is being adopted faster than any prior technology. The vast potential for applications and use cases means risks spill over into all sectors, including (re)insurance.

Generative AI will quickly impact the underwriting of lawyers’ E&O, as well as D&O covers for the companies building the models. The adversarial aspects of generative AI will present additional attack vectors in cyber underwriting. Underwriters, actuaries, claims professionals and support teams must be vigilant as they vet outputs for veracity. They must also stay abreast of new legal precedents regarding the copyright and fair use of the output. Welcome to a new world.

Otakar G Hubschmann is the head of applied data at TransRe. He can be contacted to discuss any aspect of this paper or the application of artificial intelligence/machine learning to re/insurance.