Barrett, VA Officials Discuss IT Modernization And Claim Processing in VA Subcommittee Hearing
Washington, D.C. — Today, Congressman Tom Barrett (MI-07) chaired a House Veterans’ Affairs Subcommittee on Technology Modernization hearing where he spoke with officials from the Department of Veterans Affairs (VA) to conduct oversight of the department’s modernization of the Veterans Benefits Management System (VBMS). Specifically, the officials discussed how new technology can be used to deliver decisions on veterans’ claims more quickly, but must have necessary guardrails to ensure AI does not make decisions without human oversight or transparency.
Witnesses at today’s hearing included:
- Mr. Robert Orifici, Acting Deputy Chief Information Officer, Product Delivery Service, VA’s Office of Information and Technology
- Mr. Derek Herbert, Acting Chief Production Officer, Veterans Benefits Administration
- Dr. Sterling Thomas, Chief Scientist, Science, Technology Assessment, and Analytics, Government Accountability Office
- Mr. Caleb Max, President and CEO, National Artificial Intelligence Association
- Mr. Jeremy Villanueva, Associate Legislative Director, Paralyzed Veterans of America
- Mr. Gregg Orto, Deputy Director, National Veterans Service, Veterans of Foreign Wars
Click here or the image above to watch Rep. Barrett’s testimony.
Below are highlights from their conversation.
Barrett: The first question I have, Mr. Orifici, if you could just, for the benefit of myself and others, can you give me some examples of the difference between AI and automation and how they are distinct from one another while perhaps trying to achieve the outcome we're going for?
Orifici: Yes. So thank you for that question. So when we talk about automation, we're talking about very explicit tasks that are rules-based, that could follow a very explainable set of logic to accomplish that task, like ordering mail from one point and having that delivered into the system, or repopulating a form with data that's set from a basis of rules. When we look at AI, we're looking at various models. We're looking at some models which look at predictions and patterns to really make recommendations forward from data sets that are available, and then we have generative AI which is really using that to generate a response or some piece of action from that. And so those are three different categories in their distinction.
Barrett: Is AI being used in a chat function with veterans as they're going about interfacing with the VA in different ways right now? You see a lot of these on customer service platforms and things like that. Is the VA using that technology to communicate with veterans right now?
Orifici: I do not believe we're using AI to communicate in chats to veterans at this point in time.
Barrett: Okay. Assuming that may be an interest of the VA at some point, are we able to get assurances that there would be a disclaimer of sorts or a notice to veterans that this is being done using an AI feature?
Orifici: Yeah, that's a great question, and I will commit to taking that back with our AI governance to make sure that that is a disclaimer.
Barrett: Okay, thank you. I wanted to next ask you about a situation that came up in my district speaking with one of the county VSOs [veteran service organizations] in my district. They had a veteran that came to them who had filed a claim with 14 unique situations within it or 14 separate conditions. And he was using the quick submit system that the VA has. Now, he didn't have one of those Adobe subscriptions to fill out the PDF and everything else. So he used an online tool that he downloaded for free and it offset some of the formatting by enough to throw off the system. It flagged it as being incomplete, and it sent it to a VA employee for review, which is ideally what should happen is that a human takes over to look at it. But then the employee entered only the very first condition into the system, not the remaining 13 other conditions that were claimed. And obviously, that set him back significantly and it became very difficult to overcome that.
Where within the VBA's metrics or error reporting or accuracy logs would those, you know, 13 failed submissions, where would that be counted and where would it be accounted for?
Herbert: So for quick submit, my understanding of that specific issue, right, is that the automation worked as intended. The pattern recognition on the front end of the actual 526 failed the initial review, which then defaulted over into human review. The compounding problem with that specific issue is that it's a combination of automation doing what it was intended to do, diverting to a human for involvement, but then compounded by the employee making the mistake of not capturing all of the different contentions, 14 instead of just the one. Regarding the error reporting, I'm happy to take that back where that specifically would be counted.