UTA Researcher Gets $568K NSF Grant To Explore AI Systems’ Vulnerability to Misinformation

AI systems like ChatGPT and Bard offer revolutionary promise. But they can also misinform and even threaten humanity's extinction, top tech and AI leaders warn. At UT Arlington, Shirin Nilizadeh is exploring the systems' vulnerability to misinformation—and working on ways to protect them.

Shirin Nilizadeh, a University of Texas at Arlington researcher is working on the vulnerabilities of AI to online misrepresentation. Misrepresentation is part of the word cloud that depicts artificial intelligence challenges. [Sources: DI, Illustration Rob Wilson/Shutterstock]

The whole world’s abuzz about the wonders of ChatGPT, Bard, and other generative AI systems. But there’s also buzz about AI’s dangers—from its vulnerability to misinformation all the way to concerns that it could pose “a risk of extinction” for humanity.

At the University of Texas at Arlington, a researcher is taking on that first danger—by working to understand the vulnerabilities of artificial intelligence to online misinformation. And she just got a $567,609 grant from the National Science Foundation to help her do it.

Shirin Nilizadeh, an assistant professor in UTA’s department of computer science and engineering, earned the five-year grant for her work to increase the security of natural language generation (NLG) systems, in order “to guard against misuse and abuse that could allow the spread of misinformation online.”

Adversaries may try to ‘poison these systems’ with false information

UTA’s Shirin Nilizadeh

AI misinformation is “an important and timely problem to address,” Nilizadeh says.

“These systems have complex architectures and are designed to learn from whatever information is on the internet,” she added in a statement. “An adversary might try to poison these systems with a collection of adversarial or false information.”

“The system will learn the adversarial information in the same way it learns truthful information. The adversary can also use some system vulnerabilities to generate malicious content. We first need to understand the vulnerabilities of these systems to develop detection and prevention techniques that improve their resilience to these attacks.”

CAREER grant is the NSF’s most prestigious for junior faculty

Nilizadeh’s grant was awarded through the NSF’s Faculty Early Career Development Program, the foundation’s most prestigious honor for junior faculty. Recipients are selected for being not just outstanding researchers but also outstanding teachers through research, educational excellence, and integrating education and research at their home institutions.

Nilizadeh will use the funding to take a comprehensive look at the types of attacks NLG systems are susceptible to. She’ll also create AI-based optimization methods to examine the systems against different attack models. After an in-depth analysis and characterization of vulnerabilities that lead to attacks, she’ll work to develop defensive methods to protect systems like OpenAI’s ChatGPT and GPT4 and Google’s Bard.

Focusing on two key NLG techniques

Two common natural language generation techniques will be the focus of Nilizadeh’s research, UTA said: summarization and question-answering.

In summarization, an AI is given a list of articles and asked to summarize their content. In question answering, the system is given a document and asked to find answers to questions in that document by generating text answers.

Hong Jiang, chair of Nilizadeh’s department at UT Arlington, says the research will address “serious concerns.”

“With large language models and text-generation systems revolutionizing how we interact with machines and enabling the development of novel applications for health care, robotics and beyond, serious concerns emerge about how these powerful systems may be misused, manipulated or cause privacy leakages and security threats,” Jiang said in a statement.

“It is threats like these that Dr. Nilizadeh’s CAREER Award seeks to defend against by exploring novel methods for enhancing the robustness of such systems so that misuses can be detected and mitigated, and end-users can trust and explain the outcomes generated by the systems,” Jiang added.

Open letter warned of ‘profound risks to society and humanity’

In March, an open letter signed by Elon Musk, Apple co-founder Steve Wozniak, and more than 1,000 artificial intelligence experts, researchers, and backers warned of the “profound risks to society and humanity” posed by AI systems with human-competitive intelligence.

Local signers included UT Dallas professor Sriraam Natarajan and student assistant Charles Averill, according to the Silicon Valley Business Journal. Other signers from Texas included J. Craig Wheeler, the Samuel T. and Fern Yanagisawa Regents Professor of Astronomy, emeritus, at UT Austin and an inaugural fellow of the American Astronomical Society and past president of the American Astronomical Society; and Peter Stone, associate chair of computer science and director of robotics at the University of Texas.

Get on the list.
Dallas Innovates, every day.

Sign up to keep your eye on what’s new and next in Dallas-Fort Worth, every day.

One quick signup, and you’re done.

R E A D   N E X T

  • AI Master's Degree UNT

    Innovators across North Texas and entrepreneurs all over the U.S. are racing to launch (and patent) the latest breakthrough AI. That can lead to IP policy issues as emerging tech hits the market. On February 8, the U.S. Patent and Trademark Office will a public meeting to discuss how to promote innovation in AI and Machine Learning tech, to be held virtually and in person at the Arts District Mansion in Dallas. "The takeaways will shape future work on AI and ET policy," says USPTO Director Kathi Vidal.

  • The request was made in an open letter signed by major players including Elon Musk, co-founder of OpenAI, the lab that created ChatGPT and GPT-4; Emad Mostaque, founder of the London-based Stability AI; and Apple co-founder Steve Wozniak. More than 1,000 artificial intelligence experts, researchers, and backers signed the letter, including academics from UT Dallas.

  • Dr. Venu Varanasi, pictured in his lab,an associate professor and lab director at UT Arlington, won the pitch competition.

    After a rigorous process, the top six applications to the Tech Transfer Office Showcase at BioNTX's iC3 life science summit were invited to pitch live at BioNTX's life science summit. Industry experts rated, graded, and discussed the technologies prior to the pitches "We want to cultivate technologies that are coming out of the university ecosystem here—and we want them to stay here," said Chad Ronholdt, managing director at NVB Ventures.

  • Oscar, the AI-powered robotic trash can at AT&T's downtown Dallas HQ, can spot a recyclable in nothing flat. Developed by Vancouver, Canada-based Intuitive AI, the robot checks out any trash item you hold up to it and tells you whether it goes in its recycling, compost, or landfill bins.

  • [Image: Ryzhi/istockphoto]

    North Texas universities are gearing up for the AI revolution, readying students for the jobs of tomorrow. Here's a look at what's happening at SMU, TCU, the University of North Texas, UT Arlington, and UT Dallas, as published in our annual 2023 Dallas Innovates magazine.