As a cybersecurity blade, ChatGPT can cut both ways
Your email has been sent
The cybersecurity implications of ChatGPT are vast, especially for email exploits, but putting up guardrails, flagging elements of phishing emails that it doesn’t touch and using it to train itself could help boost defense.
ChatGPT — the Large Language Model developed by OpenAI and based on the GPT-3 natural language generator — is generating ethical chatter. Like CRISPR’s impact on biomedical engineering, ChatGPT slices and dices, creating something new from scraps of information and injecting fresh life into the fields of philosophy, ethics and religion.
It also brings something more: vast security implications. Unlike typical chatbots and NLP systems, ChatGPT bots act like people — people with degrees in philosophy and ethics and just about everything else. Its grammar is impeccable, syntax impregnable and rhetoric masterful. That makes ChatGPT an excellent tool for business email compromise exploits.
As a new report from Checkpoint suggests, it’s also an easy way for less code-fluent attackers to deploy malware. The report details several threat actors who recently popped up on underground hacking forums to announce their experimentation with ChatGPT to recreate malware strains, among other exploits.
Richard Ford, CTO at security services firm Praetorian, wondered about the risks of using ChatGPT, or any auto code-generation tool, to write an application.
“Do you understand the code you’re pulling in, and in the context of your application, is it secure?” Ford asked. “There’s tremendous risk when you cut and paste code you don’t understand the side effect of — that’s just as true when you paste it from Stack Overflow, by the way — it’s just ChatGPT makes it so much easier.”
SEE: Security Risk Assessment Checklist (TechRepublic Premium)
Jump to:
A recent study by Andrew Patel and Jason Sattler of W/Labs with the enticing title “Creatively malicious prompt engineering” found that large language models used by ChatGTP are excellent at crafting spear phishing attacks. In their words, these models can “text deepfake” a person’s writing style, adopt stylistic quirks, offer opinions and create fake news without that content even appearing in its training data. This means that processes like ChatGPT can create infinite iterations of phishing emails with each iteration capable of building trust with its human recipient and fooling standard tools that look for suspicious text.
Crane Hassold, an analyst at Abnormal Security, offered an apt demonstration of ChatGPT’s ability to replace people like me by having it craft a workable introduction to an article about itself. He said the framework is a great multitool for malefactors because it doesn’t include phishing indicators that IT teams train personnel and AI to scan for.
“It can craft realistic emails free of red flags and free of indications that something is malicious,” Hassold said. “It can be more detailed, more realistic looking and more diverse.”
When Abnormal Security conducted a test asking ChatGPT to write five new variations of a BEC attack aimed at HR and payroll, it generated in less than a minute five missives that Hassold noted were mutually unique (Figure A).
Figure A
Hassold said bad actors in underground communities for BEC attacks share templates that actors use repeatedly, which is why many people may see the same sorts of phishing emails. ChatGPT-generated phishing mails avoid that redundancy and therefore sidestep defensive tools that rely on identifying malicious text strings.
“With ChatGPT, you can create a unique email every time for every campaign,” Hassold said.
In another example, Hassold asked ChatGPT to create an email that had a high likelihood of getting a recipient to click on a link.
“The resulting message looked very similar to many credential phishing emails we see at Abnormal,” he said (Figure B).
Figure B
When the investigators at Abnormal Security followed this up with a question asking the bot why it thought the email would have a high success rate, it returned a “lengthy response detailing the core social engineering principles behind what makes the phishing email effective.”
SEE: Artificial Intelligence Ethics Policy (TechRepublic Premium)
When it comes to flagging BEC attacks before they reach recipients, Hassold suggests using AI to fight AI, as such tools can scout for so-called behavioral artifacts that are not part of ChatGPT’s domain. This requires a comprehension of the:
Because they are outside the aegis of ChatGPT, Hassold noted they can still be used by AI security tools to identify potentially more sophisticated social engineering attacks.
“Let’s say I know the correct email address ‘John Smith’ should be communicating from: If the display name and email address don’t align, that might be a behavioral indication of malicious activity,” he said. “If you pair that information with signals from the body of the email, you’re able to stack several indications that diverge from correct behavior.”
SEE: Secure corporate emails with intent-based BEC detection (TechRepublic)
As Patel and Sattler note in their paper, GPT-3 and other tools based on it enable social engineering exploits that benefit from “creativity and conversational approaches.” They pointed out that those rhetorical capabilities can erase cultural barriers in the same way the Internet erased physical ones for cybercriminals.
“GPT-3 now gives criminals the ability to realistically approximate a wide variety of social contexts, making any attack that requires targeted communication more effective,” they wrote.
In other words, people respond better to people — or things that they think are people — than they do to machines.
For Jono Luk, vice president of product management at Webex, this points to a larger issue around the ability of tools powered by autoregressive language models to expedite social engineering exploits at all levels and all purposes, from phishing to broadcasting hate speech.
He said guardrails and governance should be inbuilt to flag malicious, incorrect content, and he envisions a red team/blue team approach to training frameworks like ChatGPT to flag malicious activity or the inclusion of malicious code.
“We need to find a similar approach to ChatGPT that Twitter — a decade ago — did by providing information to the government about how it was protecting user data,” Luk said, referencing a 2009 data breach for which the social media company later reached a settlement with the FTC.
Ford offered at least one positive take on how Large Language Models like ChatGPT can benefit non-experts: Because it engages with a user at their level of expertise, it also empowers them to learn quickly and act effectively.
“Models that allow an interface to adapt to the technical level and needs of an end user are really going to change the game,” he said. “Imagine online help in an application that adapts and can be asked questions. Imagine being able to get more information about a particular vulnerability and how to mitigate it. In today’s world, that’s a lot of work. Tomorrow, we could imagine this being how we interact with parts of our complete security ecosystem.”
He suggested that the same principle holds true for developers who are not security experts but want to suffuse their code with better security protocols.
“As code comprehension skills in these models improve, it’s possible that a defender could ask about side effects of code and use the model as a development partner,” Ford said. “Done correctly, this could also be a boon for developers who want to write secure code but are not security experts. I honestly think the range of applications is massive.”
If natural language generating AI models can make bad content, can it use that content to help make it more resilient to exploitation or better able to detect malicious information?
Patel and Sattler suggest that outputs from GPT-3 systems can be used to generate datasets containing malicious content and that these sets could then be used to craft methods to detect such content and determine whether detection mechanisms are effective — all to create safer models.
The buck stops at the IT desk, where cybersecurity skills are in high demand, a shortfall the AI arms race is likely to exacerbate. To upgrade your skills, check out this cheat sheet on how to become a cybersecurity pro.
Strengthen your organization’s IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices.
As a cybersecurity blade, ChatGPT can cut both ways
Your email has been sent
Your message has been sent
TechRepublic Premium content helps you solve your toughest IT issues and jump-start your career or next project.
Looking for the best payroll software for your small business? Check out our top picks for 2023 and read our in-depth analysis.
Next year, cybercriminals will be as busy as ever. Are IT departments ready?
The company, which for several years has been on a buying spree for best-of-breed products, is integrating platforms to generate synergies for speed, insights and collaboration.
Organize a number of different applicants using an ATS to cut down on the amount of unnecessary time spent finding the right candidate.
Whether you are a Microsoft Excel beginner or an advanced user, you’ll benefit from these step-by-step tutorials.
Strengthen your organization’s IT security defenses by keeping abreast of the latest cybersecurity news, solutions, and best practices.
Whether an infection is the result of a disgruntled employee, hardware vulnerability, software-based threat, social engineering penetration, robotic attack or human error, all organizations must be prepared to immediately respond effectively to such an issue if the corresponding damage is to be minimized. Because even the best protected networks become infected, all organizations must have …
Every operating system should be appropriately secured, especially end user workstations which often contain or permit access to company data and upon which most employee job duties are based. To get the maximum security protection out of your Windows 10 deployments follow this checklist from TechRepublic Premium. Also included in this checklist: Security solutions Product …
Third-party cloud-based file storage applications such as Dropbox, OneDrive and Google Drive have all contributed to the success of business cloud data storage. These products have brought proven flexibility and ease of data access to users and businesses alike while ensuring security of confidential information. Cloud data storage offers many benefits to consumers and companies …
If you set up your IT budget the right way now, you can manage spending all year — and save a lot of time and effort when next year’s budgeting chores roll around. This Excel template includes tabs to help you track spending, monitor unplanned purchases, categorize expenses, and record key data like recurring payments, …
As a cybersecurity blade, ChatGPT can cut both ways – TechRepublic
