Resettlement agencies face concerns over the use of artificial intelligence (AI) in refugee resettlements, as it raises the risks of discrimination. While these agencies are tasked with ensuring the well-being of refugees and making relocation decisions impartially, the implementation of automated decision-making (ADM) systems may contradict this requirement.
ADM systems utilize algorithms and data to make decisions typically made by humans. At least half of the ten agencies overseeing refugee resettlement rely on this technology for assistance in making resettlement decisions.
In 2018, HIAS, a refugee resettlement agency, collaborated with researchers to develop Annie MOORE, one of the initial ADM systems used to aid in resettlement. This system recommended placements that maximized employment opportunities for refugees entering the country.
However, the developers of Annie MOORE acknowledged that the system’s predictive capabilities were hindered by limited data. As a result, researchers heavily relied on data that only measured short-term employment outcomes of refugees 90 days after their arrival.
Unfortunately, Annie MOORE did not consider crucial factors such as refugees’ long-term employment prospects, physical and mental health, education, or household earnings. These elements are essential in measuring the success of refugee resettlement, alongside short-term employment outcomes.
Although HIAS has since implemented an updated ADM system called RUTH, the concern of limited data remains. The developers highlight that while HIAS possesses data on hundreds of thousands of refugees, it spans only 15 fiscal years, with earlier years being incomplete and on a smaller scale.
Although ADM technologies can expedite the refugee resettlement process, basing algorithms on limited data can perpetuate inequality. This becomes evident when assessing how ADM systems for refugee resettlement yield different decisions across various demographic groups.
A working paper from Harvard Business School in February 2023 delves into the legal issues stemming from the use of current resettlement algorithms, which can have significant discrepancies based on a refugee’s age, education, and country of origin. The authors emphasize that the use of a refugee assignment algorithm can be jeopardized from a legal standpoint if it has disparate impacts on refugees from different origin countries. They also draw attention to Recital 71 of the EU’s General Data Privacy Regulation, which calls for preventing discriminatory effects based on racial or ethnic origin.
While comprehensive data privacy legislation is yet to be passed by Congress, Title 8 of the US Code grants the government sufficient authority to regulate decision-making processes of resettlement agencies if they discriminate against individuals based on nationality.
Case law offers insights into potential problems that may arise for resettlement agencies using ADM systems. For instance, in the Bauserman v. Unemployment Insurance Agency case, plaintiffs alleged that Michigan’s Unemployment Insurance Agency’s ADM system automatically labeled applications as fraudulent and collected fines without due process.
In July 2022, the Michigan Supreme Court ruled in favor of the plaintiffs, allowing them to seek monetary damages from the state for an alleged violation of the state’s constitution. Michigan ultimately approved a $20 million settlement for individuals wrongly accused of fraud by the ADM system.
While ADM systems used in refugee resettlement differ from those employed by Michigan’s Unemployment Insurance Agency, resettlement agencies must ensure their programs comply with due process requirements and do not result in discriminatory effects.
Refugee resettlement agencies play a vital role in the process of aiding individuals who have endured unimaginable hardships and strive to help them thrive in the United States. It is understandable that agencies would seek assistance from new technologies. However, it is imperative for these agencies to ensure that the systems they employ do not disadvantage the very people they aim to support.
