A hot potato: In the latest instance of generative AI being used for something that AI probably shouldn’t be used for, Nevada is set to become the first state to use the technology to make recommendations on unemployment benefit appeals. Since these systems are prone to errors, bias, and hallucinations, there are understandable concerns about the move. The fact that courts may not be able to overturn rulings made on the basis of an AI’s mistake is compounding these fears.
Nevada’s Google-powered generative AI system is set to launch within the next few months, reports Gizmodo. Its function will be to analyze transcripts of unemployment appeals hearings and issue recommendations to the humans that give a final verdict.
Nevada’s Department of Employment, Training, and Rehabilitation (DETR) paid Google $1,383,838 for the AI tech, which uses a Google cloud service called Vertex AI Studio. It will generate recommendations based on transcripts of unemployment hearing appeals and rulings, comparing the information to previous cases.
The system is being introduced to expedite the appeals process. DETR’s information technology administrator, Carl Stanfield, said the employees take about three hours on average to write a determination. The AI system will reduce this to just five minutes in some cases. That could be especially helpful in Nevada, which has a backlog of more than 40,000 appeals resulting from a rise in unemployment claims during the pandemic.
The downside, of course, is that generative AI remains notorious for spouting falsehoods and generally making things up. The DETR says recommendations made by its AI will be reviewed by at least one human. If the person agrees, the decision will be signed off. If they don’t, the document will be revised and the agency will investigate what happened.
“There’s no AI [written decisions] that are going out without having human interaction and that human review,” said DETR director Christopher Sewell.
Google said it will work with Nevada to “identify and address any potential bias” and to “help them comply with federal and state requirements.”
There are concerns that DETR employees will feel forced to rush their reviews of the AI’s recommendations, thereby clearing the backlog faster. It’s especially worrying as this could impact a claimant’s ability to appeal a case.
“In cases that involve questions of fact, the district court cannot substitute its own judgment for the judgment of the appeal referee,” Elizabeth Carmona, a senior attorney with Nevada Legal Services, told Gizmodo. This means a court might not be able to overturn a human decision based on an AI error. It would likely be difficult to pinpoint where an error originated, too.
Despite these legitimate concerns, don’t be surprised to see more states start using AI to speed up processes by helping to make important decisions.
