Apr 10, 2024 · We’re designing tools to help ensure that AI systems are trustworthy, reliable and can optimize business processes. We create tests to simulate real-life scenarios and localize the faults in AI systems. We’re working on automating testing, debugging, and repairing AI models across a wide range of scenarios.

We created AGENT, a benchmark for evaluating an AI model’s core psychological reasoning ability, or common sense, that will enable us to build and test AI models that reason and learn about other minds the same way humans do. This post is part of our paper explainer series.

Neuro-symbolic AI approaches aren’t new: Researchers have long tried to combine the superb perception capabilities of neural networks (such as the ability to recognize objects in an image) with the reasoning ability (like identifying abstract relationships between objects such as their shape, size or color) provided by symbolic AI.

4 days ago · What’s Next in AI is foundation models at scale. AI is revolutionizing how business gets done, but popular models can be costly and are often proprietary. At IBM Research, we’re designing powerful new foundation models and generative AI …

Oct 6, 2024 · Trustworthy AI. Our trust in technology relies on understanding how it works. It’s important to understand why AI makes the decisions it does. We’re developing tools to make AI more explainable, fair, robust, private, and transparent.

A new open-source adversarial-AI toolkit goes beyond images to test the robustness of a wide range of AI models. Even if you’ve never heard the term “adversarial evasion attack,” you may have heard of one of the first experiments to show how easy it can be to trick a computer vision model into misclassifying an image.

The biggest bottleneck in AI inferencing is memory. Running a 70-billion parameter model requires at least 150 gigabytes of memory, nearly twice as much as a Nvidia A100 GPU holds. But if the compiler can split the AI model’s computational graph into strategic chunks, those operations can be spread across GPUs and run at the same time.

In this post, I discuss the importance, challenges, and some of IBM Research’s efforts on AI model testing towards ensuring AI trustworthiness. Types of Testing in AI Lifecycle. At a high level, the Data and AI Lifecycle contains three phases: The first phase is the Data Lifecycle where data is pre-processed and made ready for building an AI ...

Aug 22, 2023 · Learn how to deploy RAG with AI agents and how to build RAG for AI agents. Explore how to deploy AI across your business with IBM watsonx and RAG . RAG is an AI framework for retrieving facts from an external knowledge base to ground large language models (LLMs) on the most accurate, up-to-date information and to give users insight into LLMs ...

In our recent paper, “Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Text” accepted to AAAI 2021, we describe how our software, dubbed GYC, generates test cases to check the reliability of AI models. 1 GYC can evaluate other AIs for accuracy, gender bias, and run a sensitivity analysis, as well as check a model’s adversarial …