Last October New York City introduced a new AI-powered chatbot on its MyCity portal as a way to help businesses in the city get information they need faster. As it turns out, that chatbot might be giving those businesses incorrect information.
According to The Markup, the chatbot is providing incorrect information about everything from housing policies to workers’ rights. For instance, the chatbot responded to the question Can I make my store cashless? With Yes, you can make your store cashless in New York City even though NYC banned cashless stores four years ago in 2020.
According to the report, the chatbot also has responded inaccurately to questions about whether or not landlords are required to accept Section 8 vouchers and even questions about whether or not employers can take their workers’ tips.
The report says that the bot doesn’t return the same responses each time it’s asked a question. For instance, in one case it told a reporter that landlords did have to accept housing vouchers (the truth) but in ten other cases it responded to the same question by saying that they were not required to do so.
The chatbot is powered by Microsoft’s Azure AI services which are also used by Reddit and AT&T, amongst others.
A spokesperson for the NYC Office of Technology and Innovation told the site that the chatbot is a pilot program and will improve over time.
