AINOW 2019 ReportRECOMMENDATIONS1. Regulators should ban the use of affect recognition in important decisions that impact people’s lives and access to opportunities. Until then, AI companies should stop deploying it. Given the contested scientific foundations of affect recognition technology—a subclass of facial recognition that claims to detect things such as personality, emotions, mental health, and other interior states—it should not be allowed to play a role in important decisions about human lives, such as who is interviewed or hired for a job, the price of insurance, patient pain assessments, or student performance in school. Building on last year’s recommendation for stringent regulation, governments should specifically prohibit use of affect recognition in high-stakes decision-making processes.
2. Government and business should halt all use of facial recognition in sensitive social and political contexts until the risks are fully studied and adequate regulations are in place. In 2019, there has been a rapid expansion of facial recognition in many domains. Yet there is mounting evidence that this technology causes serious harm, most often to people of color and the poor. There should be a moratorium on all uses of facial recognition in sensitive social and political domains—including surveillance, policing, education, and employment—where facial recognition poses risks and consequences that cannot be remedied retroactively. Lawmakers must supplement a moratorium with (1) transparency requirements that allow researchers, policymakers, and communities to assess and understand the best possible approach to restricting and regulating facial recognition; and (2) protections that provide the communities on whom such technologies are used with the power to make their own evaluations and rejections of its deployment.
3. The AI industry needs to make significant structural changes to address systemic racism, misogyny, and lack of diversity. The AI industry is strikingly homogeneous, due in large part to its treatment of women, people of color, gender minorities, and other underrepresented groups. To begin addressing this problem, more information should be shared publicly about compensation levels, response rates to harassment and discrimination, and hiring practices. It also requires ending pay and opportunity inequality and providing real incentives for executives to create, promote, and protect inclusive workplaces. Finally, any measures taken should address the two-tiered workforce, in which many of the people of color at tech companies work as undercompensated and vulnerable temporary workers, vendors, or contractors.
4. AI bias research should move beyond technical fixes to address the broader politics and consequences of AI’s use. Research on AI bias and fairness has begun to expand beyond technical solutions that target statistical parity, but there needs to be a much more rigorous examination of AI’s politics and consequences, including close attention to AI’s classification practices and harms. This will require that the field center “non-technical” disciplines whose work traditionally examines such issues, including science and technology studies, critical race studies, disability studies, and other disciplines keenly attuned to social context, including how difference is constructed, the work of classification, and its consequences.
5. Governments should mandate public disclosure of the AI industry’s climate impact. Given the significant environmental impacts of AI development, as well as the concentration of power in the AI industry, it is important for governments to ensure that large-scale AI providers disclose the climate costs of AI development to the public. As with similar requirements for the automotive and airline industries, such disclosure helps provide the foundation for more informed collective choices around climate and technology. Disclosure should include notifications that allow developers and researchers to understand the specific climate cost of their use of AI infrastructure. Climate-impact reporting should be separate from any accounting for offsets or other mitigation strategies. In addition, governments should use that data to ensure that AI policies take into account the climate impacts of any proposed AI deployment.
6. Workers should have the right to contest exploitative and invasive AI—and unions can help. The introduction of AI-enabled labor-management systems raises significant questions about worker rights and safety. The use of these systems—from Amazon warehouses to Uber and InstaCart—pools power and control in the hands of employers and harms mainly low-wage workers (who are disproportionately people of color) by setting productivity targets linked to chronic injuries, psychological stress, and even death and by imposing unpredictable algorithmic wage cuts that undermine economic stability. Workers deserve the right to contest such determinations, and to collectively agree on workplace standards that are safe, fair, and predictable. Unions have traditionally been an important part of this process, which underscores the need for companies to allow their workers to organize without fear of retaliation.
7. Tech workers should have the right to know what they are building and to contest unethical or harmful uses of their work. Over the last two years, organized tech workers and whistleblowers have emerged as a powerful force for AI accountability, exposing secretive contracts and plans for harmful products, from autonomous weapons to tracking-and-surveillance infrastructure. Given the general-purpose nature of most AI technology, the engineers designing and developing a system are often unaware of how it will ultimately be used. An object-recognition model trained to enable aerial surveillance could just as easily be applied to disaster relief as it could to weapons targeting. Too often, decisions about how AI is used are left to sales departments and executives, hidden behind highly confidential contractual agreements that are inaccessible to workers and the public. Companies should ensure that workers are able to track where their work is being applied, by whom, and to what end. Providing such information enables workers to make ethical choices and gives them power to collectively contest harmful applications.
8. States should craft expanded biometric privacy laws that regulate both public and private actors. Biometric data, from DNA to faceprints, is at the core of many harmful AI systems. Over a decade ago, Illinois adopted the Biometric Information Privacy Act (BIPA), which has now become one of the strongest and most effective privacy protections in the United States. BIPA allows individuals to sue for almost any unauthorized collection and use of their biometric data by a private actor, including for surveillance, tracking, and profiling via facial recognition. BIPA also shuts down the gray and black markets that sell data and make it vulnerable to breaches and exploitation. States that adopt BIPA should expand it to include government use, which will mitigate many of biometric AI’s harms, especially in parallel with other approaches, such as moratoriums and prohibitions.
9. Lawmakers need to regulate the integration of public and private surveillance infrastructures. This year, there was a surge in the integration of privately owned technological infrastructures with public systems, from “smart” cities to property tech to neighborhood surveillance systems such as Amazon’s Ring and Rekognition. Large tech companies like Amazon, Microsoft, and Google also pursued major military and surveillance contracts, further enmeshing those interests. Across Asia, Africa, and Latin America, multiple governments continue to roll out biometric ID projects that create the infrastructure for both state and commercial surveillance. Yet few regulatory regimes govern this intersection. We need strong transparency, accountability, and oversight in these areas, such as recent efforts to mandate public disclosure and debate of public-private tech partnerships, contracts, and acquisitions.1
10. Algorithmic Impact Assessments must account for AI’s impact on climate, health, and geographical displacement. Algorithmic Impact Assessments (AIAs)2 help governments, companies, and communities assess the social implications of AI, and determine whether and how to use AI systems. Those using AIAs should expand them so that in addition to considering issues of bias, discrimination, and due process, the isues of climate, health, and geographical displacement are included.
11. Machine learning researchers should account for potential risks and harms and better document the origins of their models and data. Advances in understanding of bias, fairness, and justice in machine learning research make it clear that assessments of risks and harms are imperative. In addition, using new mechanisms for documenting data provenance and the specificities of individual machine learning models should also become standard research practice. Both Model Cards3 and Datasheets4 offer useful templates. As a community, machine learning researchers need to embrace these analyses and tools to create an infrastructure that better considers the implications of AI.5
12. Lawmakers should require informed consent for use of any personal data in health-related AI. The application of AI in healthcare requires greater protections around data. While the informed-consent process that biomedical researchers and healthcare professionals generally employ in clinical settings requires discussion of the risks and benefits involved, affirmative approval before proceeding, and reasonable opportunities to withdraw from the study or treatment, engineers and scientists commonly create training sets by scraping content from whatever public sources are available. In order to ensure a future that does not amplify and reinforce historic injustices and social harms, AI health systems need better informed-consent approaches and more research to understand their implications in light of systemic health inequities, the organizational practices of healthcare, and diverse cultural approaches to health.
Source:
https://ainowinstitute.org/reports.html |
cacheFull report:
pdf |
cache
