Evaluating multi-step LLM systems requires moving beyond end-to-end metrics. Our researchers introduced RAFFLES, an iterative reasoning-based fault attribution framework that uses a central Judge and specialized Evaluators to pinpoint step-level faults in complex AI workflows.
Read the full paper to see how RAFFLES diagnoses step-level errors across multi-agent and mathematical reasoning benchmarks: capitalone.science/publicati…
While numerous Direct Preference Optimization (DPO) extensions have been introduced, discerning which components drive downstream gains remains tough. We collaborated with researchers from @Columbia to build RainbowPO, a unified framework that categorizes and combines seven key DPO improvements.
Instead of relying on third-party API wrappers, we build custom, proprietary AI models trained on our modern data ecosystem. Our Chief Scientist Prem Natarajan recently joined @IEEESpectrum to discuss how our research team advances model customization for complex financial environments.
Defending LLMs requires a proactive offensive strategy. That’s why Capital One researchers published an end-to-end overview of LLM red teaming, which involves proactively attacking models to identify vulnerabilities. Their work maps out attack methods, software packages, and evaluation metrics for practical applications.
As part of our commitment to AI building AI fluency at scale, we recently brought together over 10,000 of our engineers for a month of hands-on training to master agentic coding workflows with partners like @Google and @AnthropicAI.
Complex RAG queries often force a choice between missing context and bloated prompts. Capital One researchers introduced FB-RAG, a training-free framework using lightweight LLMs to sample future outputs and guide the final generator—cutting latency up to 48%.
At Capital One, we’re building custom AI stack layers and task-specialized models to run AI reliably at scale. By pairing open-source models with proprietary data and breaking down workflows, we get better latency, compute efficiency, and deterministic logic.
Capital One researchers developed APT (Adversarially Pre-trained Transformer) for zero-shot tabular prediction. APT uses adversarial synthetic data agents and a mixture block architecture to handle arbitrary class counts with sub-second runtimes.
AI security improves when organizations share research, tools, and real-world experience.
We’re joining other industry leaders, including @NVIDIA, in the Open Secure AI Alliance to help organizations identify and address software vulnerabilities and strengthen critical systems.
Learn more: blogs.nvidia.com/blog/open-s…
AI security advances when the industry builds in the open, together.
We're introducing the Open Secure AI Alliance with industry leaders to develop new techniques and tools to safeguard software and agents.
By sharing models, tooling and research in the open, we can broaden the community of defenders.
Learn more about the founding members’ contributions: nvda.ws/4pD8Fc5
ALT Logos of various companies including Adobe, Cisco, and Microsoft.
Earlier this month, we wrapped up our July tour stops at #ICML2026 and #ACL2026NLP! Our teams shared peer-reviewed research on LLM reasoning and agentic safety with a focus on how we build responsible and scalable foundation model architectures.