AI-Driven Security Threats (Social Engineering)

Earlier in the course, we discussed how to protect your organisation’s data by not feeding sensitive information into the AI. But there is another side to the coin: how AI can be used against you. We are now entering an era where fraud and security threats will become significantly harder to detect, as they are no longer exposed by poor spelling or peculiar phrasing. This is known as AI-driven “Social Engineering”, where technology is used to manipulate people into handing over passwords or executing payments.

The classic “phishing email” has undergone a total transformation. In the past, it was often easy to identify a scam email due to grammatical errors and generic greetings. With generative AI, a fraudster can, in seconds, create a linguistically perfect email that appears to come from your manager, your bank, or a well-known authority. Furthermore, the text can be tailored to your personal profile and written in a manner that feels stressful or urgent, which makes us lower our guard more easily.

Even more sophisticated is the use of “Deepfakes” – AI-generated audio or video. There is technology today that can clone a voice using only a few seconds of an audio clip as a source. There have been cases where finance managers have received a telephone call from what sounds exactly like their CEO, requesting an urgent and secret transfer of funds to “close an important deal”. When the voice, the tone, and the manner of speaking correspond with reality, it is human nature to want to help.

To protect yourself against these threats, your most important defence technique is to be “healthily sceptical”. If you receive an unusual request, whether it is via email, chat, or telephone, you should always use an alternative communication channel to confirm the identity. Call your colleague on the number you have saved, or send a message via another channel. Many companies are now also introducing internal “passwords” or control questions to verify identity during sensitive transactions. Remember: the technology can copy a voice, but it cannot copy the shared memories or the specific context you have with your colleagues. Daring to pause and ask an extra control question is not a sign of suspicion, but of high digital competence.