#AIEA Lab

The Artificial Intelligence and Explainability in Autonomous (AIEA) Lab at the University of California, Santa Cruz, focuses on enhancing the transparency and reliability of autonomous systems. Led by Assistant Professor Leilani H. Gilpin, the lab's research aims to design methods that enable autonomous systems to explain their decision-making processes.

#Key Projects

Here are the key projects from AIEA Lab:

• Explaining Large Language Model (LLM) Failures: Investigating the causes of hallucinations in LLMs to enhance their reliability and trustworthiness
• Robust Autonomous Vehicles: Creating a framework to detect and explain errors in autonomous vehicles, aiming to improve their safety and reliability.

#Team Members

Leilani H. Gilpin: Assistant Professor leading the lab.
Priyesh Vakharia: Ph.D. student focusing on Explainable AI, Human-Centered AI, and Natural Language Processing. Auther of ProSLM

#ProSLM

ProSLM is a neurosymbolic framework developed to enhance reliability of LLMs in answering questions. Unlike traditional approaches, ProSLM integrates formal logic, utilizing a domain-specific knowledge base and a reasoning system.

It has two key functions:

• context gathering, which generates explainable and relevant context for queries
• validation, which confirms the factual accuracy of outputs.

By linking symbolic reasoning (through Prolog) with LLM capabilities, ProSLM offers both creativity and reliability. This framework operates efficiently without additional LLM training, providing trustworthy and transparent AI responses.

#LLM - Logic Project

Building a system that verifies factual statements through a neural-symbolic process. The user's input is translated into Prolog queries by a neural translator. These queries represent structured assertions, such as the number and names of dining halls. A symbolic component then evaluates each query's truth value against known data. The results determine whether the overall statement is accurate. In this example, one query returns False, resulting in an output indicating the user's statement is incorrect, with specific truth values provided for transparency.

#Converting NL to Prolog

Our first task was to convert Natural Language (NL) to Prolog. The reason behind was we could make goal tree. For example of converting, look on

#What is Prolog

Prolog is a declarative programming language designed for developing logic-based AI applications. Developers can set rules and facts around a problem, and then Prolog's interpreter will use that information to automatically infer solutions.

Here’s an example of using a Prolog-like system in Python for a medical diagnosis:

In this example, the system identifies symptoms and uses logical inference (backward chaining) to assess the likelihood of a flu diagnosis.

#Converting from NL

To convert Natural Language (NL) to Prolog, first review existing methods and datasets, leveraging models like GPT-4 and Gemini to create 100 example pairs for NL-to-Prolog translations. Evaluate resources (e.g., Simpson’s KB) for existing data, expanding with BLEU and ROUGE scores to assess translation accuracy. Manually verify 10 samples for quality, addressing syntax issues and establishing reliable validation methods. If using a Knowledge Base (KB), determine model compatibility for KB-specific Prolog outputs.

For task completion, criteria include dataset creation with validated examples, organized in shared documents for consistency. Implement a framework for ongoing dataset testing and Prolog accuracy validation. Compile resources, such as a spreadsheet and links to relevant guides (e.g., RAG for Retrieval-Augmented Generation), supporting data organization and evaluation.

gpt4 (100 example demo dataset)

gemini-1.5-pro-001 (100 example demo dataset)

#Evaluation dataset

To create an evaluation dataset, we first scraped the relevant data and converted it into Prolog code, forming the foundation of our analysis and quality assurance framework.

The table below shows selected sample questions from UC Santa Cruz's Baskin Engineering FAQ, with official answers, AI-generated responses, and evaluations. Each question entry includes a G Eval Score to quantify response accuracy and a G Eval Reason to explain each score. This information highlights alignment with expected answers or any notable omissions, such as missing procedural steps.

Using this data, we developed a Retrieval-Augmented Generation (RAG) Question Answering (QA) dataset, structured to include both original and generated responses to common questions about UC Santa Cruz's Baskin Engineering School. For each question, the dataset includes the official answer, a generated response, and an evaluation score and reason. This setup allows us to assess the quality and completeness of AI-generated responses against real user needs.

The table above illustrates sample questions and responses, including scores and explanations of any omissions or misalignments. The G Eval Score (e.g., 0.59 or 0.86) quantifies the response accuracy, while G Eval Reason provides reasoning for each score, pointing out alignment with expectations or areas of improvement, such as missing procedural details.

#Evaluation Score

Each entry in the dataset is scored based on its accuracy and completeness, using a G Eval Score to quantify the alignment with expected responses and a G Eval Reason to explain any misalignments or omissions, like missing procedural details.

G-Eval

This evaluation approach allows us to assess the quality of responses by identifying areas for improvement and evaluating the system's ability to generate accurate, contextually relevant answers. The score highlights if a generated response missed important details or aligned closely with the expected answer. Also resources like BLEU and ROUGE scores are used to measure translation accuracy in converting NL to Prolog for logic-based AI systems.

To further streamline quality assessment, we leverage tools like RAGAS and G-Eval for automated scoring and human evaluations to verify dataset accuracy, ensuring that AI-generated responses are effectively tailored to meet user needs in education and support contexts.

RAGAS