Since April, Terence Jackson, chief security advisor for Microsoft, warned that the privacy policies of platforms like ChatGPT impose the collection of sensitive user data such as IP address, browser information, and browsing activities, which may be shared with third parties. I didn’t pay attention then due to misrepresentations as to the models being “open”. I sincerely thought that open-source AI models will revolutionize the entire way we think about property. Unfortunately, nothing could be less open and more corporate than current AI models. The open-source hype was designed to put indiscriminate data and IP collection, storing, and sharing in the blind spot of regulators. At this point I fully agree with Jackson that the cyber security threats posed by generative AI expand the attack surface and provide new opportunities for hackers to exploit.
The vulnerabilities of generative AI are further discussed in a Wired article inviting users to take awareness of the threats and implement robust cybersecurity measures. This article showed that it takes hackers a mere few hours to break ChatGPT and turn it into a violence-inciting agent. The process of jailbreaking exposed in the article aims to design prompts that make the chatbots bypass rules around producing hateful content or writing about illegal acts, while closely-related prompt injection attacks can quietly insert malicious data or instructions into AI models. Both approaches try to high-jack a system for malicious purposes. The attacks are essentially a form of unconventional hacking consisting of using carefully crafted and refined sentences, rather than code, to exploit system weaknesses. The jailbreak is succesfully used on other models, such as Bard and Bing Chat.
A List Of Cybersecurity Risks of Generative AI
We already know that Generative AI uses other people’s IP with or without their consent to formulate answers to your prompts. It is 100% guaranteed that your input and prompts are also copied and stored and automatically appropriated into the generative AI tools, thus creating a legal grey zone where you may not only lose ownership of your stuff as a user, but you may inadvertently cause further infringement of other people’s IP.
Creation of Phishing Emails
As more companies use email and other forms of digital communications to market their products or services, criminals are becoming more sophisticated in their efforts to trick people into giving up personal information. “Phishing” involves a fake email sent from a trusted source (such as your bank) that contains an attachment or link that looks legitimate but actually leads to a fake website where you enter your credentials to gain access to your account.
Model Manipulation and Poisoning
This type of attack involves manipulating or changing an existing model so that it produces false results. For example, an attacker could change an image to look like another image from your database instead of what it is. The attacker could then use these manipulated images as part of their attack strategy against your network or organization.
Adversarial Attacks
The use of adversarial examples — an attack that causes an algorithm to make a mistake or misclassify data — has been around since the early days of AI research. However, as adversarial attacks become more sophisticated and powerful, they threaten all types of machine learning systems, including generative models.
Data Privacy Breaches
A common concern with generative models is that they may inadvertently disclose sensitive data about individuals or organizations. For example, an organization may create an image using generative models that accidentally reveal confidential information about its customers or employees.
Deepfakes and Synthetic Media
Generative models can also be used for nefarious purposes by generating fake videos and audio recordings that can be used in deepfakes (fake videos) or synthetic media (fake news). The technology behind these attacks is relatively simple: someone needs access to the right dataset and some basic software tools to start creating malicious content. Actors and other on-screen performers are particularly at risk to be deepfaked.
Fake Data
Generative AI can generate fake data that looks authentic and passable to humans. This data type could be used in various industries, including healthcare, finance, defense, and government. It could even create fake social media accounts or impersonate an individual online.
Malicious Use of Generated Content
Generative AI can also manipulate content by changing the meaning or context of words or phrases within text or images on a webpage or social media platform. For example, if you were using an application that automatically generated captions for images with no human intervention required. It would allow someone to change the caption from “a white dog” to “a black cat” without actually changing anything about the photo itself (just by editing the caption).
How to Strengthen Your Defenses Against Generative AI Cybersecurity Risks
In response to this rising concern, organizations must strengthen their defenses against these risks. Here are some tips for doing so:
Switch to DMARC
DMARC is an email authentication protocol helping prevent email spoofing and phishing attacks that impersonate your own domain. By implementing a DMARC analyzer, organizations can ensure to the extent that only authorized senders can use their domain for email communications, thereby minimizing the risks associated with AI-generated phishing emails.
DMARC provides additional layers of protection by enabling domain owners to receive reports on email delivery and take necessary actions to strengthen email security, thereby acting as a shield against generative AI cybersecurity risks. You need to implement either SPF or DKIM or both (recommended) as a prerequisite for DMARC implementation.
Conduct Security Audits
Another way to prevent hackers from accessing your system is by conducting cybersecurity audits. These audits will help identify potential weaknesses in your system and suggest how to patch them up before they become major problems (such as malware infections).
Adversarial Training
Adversarial training is a way to simulate the adversarial attack and strengthen the model. It uses an adversary (or an attacker) that tries to fool the system by giving it wrong answers. The goal is to find out how the model will react and what its limitations are in order for us to design more robust models.
Robust Feature Extraction
Another solution is Robust Feature Extraction (RFE). RFE uses deep learning to extract relevant features from raw images. The technique is scalable and can be used on large datasets. It can also be combined with other techniques, such as Verification Through Sampling (VTS) and Outlier Detection (OD), to improve the accuracy of feature extraction.
Secure Model Architecture
Secure Model Architecture (SMA) uses a secure model architecture to prevent attacks that exploit vulnerabilities in software code, data files, or other components of an AI system. The idea behind SMA is that an attacker would have to find a vulnerability in the code instead of just exploiting a weakness in the system itself.
Regular Model Auditing
Model auditing has been an essential part of cybersecurity for many years. It involves examining the models used in a system to ensure that they are sound and up to date. Model auditing can also be used to detect vulnerabilities in models, as well as identify models that might have been corrupted or altered by hackers.
Input Validation and Filtering
Input validation is one of the most important steps a model developer can take before deploying their model into production environments. Input validation ensures that data being entered into a model isn’t inaccurate or maliciously altered by hackers who might try to exploit vulnerabilities within the system. Input filtering allows developers to specify which data types should be allowed through their models while preventing any other kinds of data from getting through as well.
Conclusion
While the technology offers numerous benefits and advancements, it also opens the door to potential vulnerabilities and threats. The ability of generative AI to create convincing fake images, videos, and text raises concerns regarding identity theft, misinformation campaigns, and fraud. Moreover, the malicious use of generative AI can amplify existing cyber threats, such as phishing attacks and social engineering.
As this technology continues to evolve, organizations and individuals must prioritize cybersecurity measures, including robust authentication, continuous monitoring, and regular vulnerability assessments, to mitigate the risks associated with generative AI. By doing so, we can harness the potential of this technology while safeguarding against its inherent cybersecurity challenges.