EEOC priorities on automated systems and technological discrimination

Dear Chair Burrows,

We, the undersigned groups, write to follow up on the July 13, 2021, letter that several of our organizations sent to you, Assistant Attorney General Kristen Clarke, then-Secretary of Labor Martin Walsh, and then-OFCCP Director Jenny Yang titled Addressing Technology’s Role in Hiring Discrimination (the “2021 letter”), and the “Automated systems, including artificial intelligence (AI)” section of the October 26, 2023, letter that the Leadership Conference on Civil and Human Rights’ Employment Task Force sent to the EEOC outlining broader employment-related civil rights priorities.

We have been pleased to see the progress that the EEOC and other federal agencies have made in the past three years on issues relating to technological discrimination. Throughout the current administration, the EEOC has shown a strong and persistent focus on the rising threat of discrimination from new workplace technologies, particularly automated decision systems (ADSs) and electronic surveillance and automated management (ESAM) systems. Within months after the beginning of the current term, the EEOC established an Artificial Intelligence (AI) initiative, which has resulted in multiple listening sessions, public hearings, and publications dedicated to exploring the potential of discrimination from technologies whose deployment has often outpaced an understanding of the risks they pose. We were especially gratified to see that the recently published Strategic Enforcement Plan (SEP) repeatedly emphasized the Commission’s intent to use its authority to target technological discrimination through its regulatory and enforcement activities.

In November 2023, President Biden issued a wide-ranging Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (the “EO”). Among other things, the EO focused on the need for agencies to “address civil rights and civil liberties violations and discrimination related to AI,” asking the Department of Justice to coordinate with the relevant heads of independent regulatory agencies such as the EEOC to achieve that goal. It also called attention to AI’s potential harm to workers' well-being and access to job opportunities. The EO followed this summer’s White House Office of Science and Technology Policy (OSTP) Request for Information on Automated Worker Surveillance and Management, underscoring the Administration’s strong interest in managing the risks associated with emerging workplace technologies.

We urge the EEOC to consider the below proposals when deciding how to strengthen protections against technological discrimination, in keeping with both the Commission’s own prior efforts and the Administration’s heightened focus on equity in the face of new technology. This letter first identifies the areas in which we believe the EEOC can make the greatest impact on these issues in the short term, then highlights the steps we believe the Commission should prioritize during the remainder of the five-year period covered by the SEP (2024-2028).

Create worker-facing "know your rights" fact sheets on Automated Decision-making Systems and Electronic Surveillance and Automated Management

Workers are at a severe information disadvantage when it comes to employers’ use of ADSs and ESAM. Current laws do not explicitly require employers to disclose when they are using these technologies, much less the specific types of technologies they are using and how those technologies may affect workers. However, employers who fail to communicate about the use and results of ADSs and ESAM may run afoul of existing civil rights laws. Because many forms of ADSs measure characteristics tied to and reflective of disability or that may be affected by disability or pregnancy, employers cannot comply with the Americans with Disabilities Act (ADA) or the Pregnant Workers Fairness Act (PWFA) unless they provide sufficient notice to allow disabled workers the opportunity to raise concerns and request either an accommodation or an alternative form of assessment that tests the worker’s ability to perform the relevant essential job functions. Moreover, if an employer plans to introduce ESAM or ADSs that could adversely impact workers with known or suspected disabilities, deploying the ADS without first notifying workers could violate those workers’ right to reasonable accommodation.

To help ensure that workers can seek the accommodations they need, the EEOC should issue worker-facing “know your rights” fact sheets and other guidance to better inform workers of how the use of new workplace technologies may affect their legal rights. Such materials would build on the EEOC’s recent technical assistance on the use of software, algorithms, and AI to assess job applicants and employees. These employer-facing documents explored ways in which algorithmic decision-making tools could violate the ADA and Title VII. The EEOC also published a brief “Tips for Workers” fact sheet that explained how uses of algorithmic systems for the screening, hiring, and firing of workers may violate the ADA. However, these documents were limited in scope and only offered examples of screening tests and related scoring practices.

The EEOC should publish expanded “Tips for Workers” guidance, building on its efforts to educate workers on their rights with respect to the use of ADSs and ESAM in the workplace. The EEOC should work with labor unions and worker rights organizations to inform the development of these resources. Such materials should include information about:

• ADSs and ESAM typically used in the workplace and how their use may impact workers and applicants.

• When workers can ask for reasonable accommodations in relation to employers’ use of ADSs or ESAM in the workplace that affect the terms and conditions of employment.

• When applicants and workers have a right to know that employers are using ADSs and ESAM.

• When workers can ask employers for justification of their use of ADSs or ESAM to make decisions affecting the terms and conditions of employment.

These fact sheets should cover workers’ rights under the ADA, Title VII, and other laws against employment discrimination, including the Age Discrimination in Employment Act (ADEA) and the PWFA.

Expand on employer-facing guidance regarding the ADA and Title VII

The 2021 letter urged the Commission to publish informal guidance encouraging hiring technology vendors and employers to design, test, and audit their technologies to prevent discriminatory effects. Such guidance is necessary to clarify for employers what information they should seek from vendors when procuring hiring technologies, and the testing employers themselves should conduct before and while using these technologies.

We were pleased to see the Commission publish technical assistance documents on how ADSs may violate the ADA and Title VII, and the White House’s Blueprint for an AI Bill of Rights elevated the ADA publication as an example of how to put protections against algorithmic bias into practice. This guidance was an important first step, correctly recognizing that existing federal employment law applies to ADSs in the same manner as to traditional selection procedures. This guidance affirms employers’ obligations to prevent discrimination arising from their ADSs, regardless of whether they create the tool themselves or use technology developed by third parties. Additionally, the ADA guidance appropriately recognizes that ADSs that purport to measure personality traits, speech patterns, or other attributes that do not inherently reflect a candidate’s ability to perform essential job functions may violate the ADA if they screen out disabled workers who could perform the essential functions of the job with or without a reasonable accommodation.

We urge the EEOC to expand on this technical assistance by: (1) providing greater detail as part of providing formal guidance or conducting rulemaking; and (2) issuing guidance on employers’ use of ESAM systems.

• Issue formal guidance or rules on ADSs that provide greater detail on employers’ obligations under Title VII, the ADA, PWFA and other anti-discrimination laws

While technical assistance provides employers with useful advice on how the EEOC may apply civil rights laws, the Commission could maximize its impact by incorporating this advice into official materials backed by a vote of the full Commission. Once it becomes formal guidance, the legal interpretations and applications set forth therein would be more likely to be adopted by courts considering civil rights cases involving ADSs. Employers would, in turn, have a much stronger incentive to heed the documents’ admonitions.

Such formal guidance (or rules) should also provide additional details on employers’ obligations to ensure that ADSs and other modern selection tools comply with federal employment discrimination law. Such guidance should, at a minimum, cover:

• Steps employers should take to notify prospective applicants in job postings or at the beginning of the application process about the assessment methods that will be used. For example, the EEOC should state that, to ensure that disabled workers have a meaningful opportunity to raise accessibility concerns or request accommodation in compliance with the ADA, employers must provide applicants with pre-assessment information about the characteristics a tool measures and how it measures them.

• Potential ways ADSs can assess people differently based on disability, race, gender, age, or other protected characteristics. For example, ADSs trained on data sets where Black, female, or other protected groups of workers are underrepresented may not learn how to recognize strong candidates from those groups, and ADSs that measure personality characteristics could discriminate against workers with a number of disabilities.

• Examples of data that create a high risk of discrimination against protected classes when used to develop, train, or deploy ADSs.

• Examples of assessment mechanisms that create a high risk of discrimination against protected classes.

• How employers should evaluate whether adequate steps have been taken to identify and mitigate algorithmic bias in ADSs.

• How often employers should refresh an ADS’s training data and retest the ADS for adverse impact and validity (e.g., due to evolving candidate pools or changes in job requirements).

• How employers can assess whether a less discriminatory alternative has comparable validity.

• How employers should inform candidates of the availability of accommodations and how to obtain them.

• When employers should proactively make alternative assessment methods and accommodations available (i.e., without requiring a candidate request).

• How ADS-driven decisions could violate the ADEA, PWFA, and other anti-discrimination laws.

Additional regulations or guidance on these and other subjects would help employers and employment agencies comply with their nondiscrimination obligations before their conduct triggers enforcement actions. Such proactive measures are particularly important given that ADSs can harm large numbers of employees and applicants before algorithmic bias is uncovered and addressed.

• Issue guidance for ESAM addressing civil rights employment protections

The EEOC should build on its existing work addressing ADSs by issuing guidance on compliance obligations for the use of ESAM under the ADA, Title VII, and other employment discrimination laws. The lack of transparency around employers’ use of ESAM, and the lack of clear guidance on how anti-discrimination laws apply, mean that many workers may not even be aware when they are being monitored and managed by automated systems, and employers may struggle to comply with the law.

There are good reasons to believe, however, that these tools will negatively impact protected groups of workers. A 2020 Data & Society report noted that low-wage and hourly work is more susceptible to datafication—and thus to ESAM—“because these jobs’ tasks are easily measured. These workers are also often immigrants, women, and people of color, populations historically facing higher scrutiny and levels of surveillance and monitoring.” Additionally, ESAM systems frequently operate by flagging behavior that is considered “atypical.” A tracking system using facial scanning may not function for workers with darker skin. A worker with a physical disability may move in ways that an automated video surveillance system identifies as suspicious. Immigrant workers in call centers monitored through speech-recognition systems may speak with accents that the algorithm may not accurately decipher—and so forth. ESAM that disproportionately flags members of protected groups as engaging in suspicious or disfavored behavior, or that otherwise tends to generate unfavorable evaluations of or actions towards protected groups of workers, thus may lead to unlawful discrimination.

Moreover, ESAM may limit workers’ opportunities to request reasonable accommodations and incorrectly flag required breaks as inappropriate. A primary use case for ESAM is to identify and eliminate inappropriate breaks and other employee downtime. This poses a particular risk for disabled and pregnant or breastfeeding workers. These workers often require more frequent breaks and, indeed, often are legally entitled to breaks as reasonable accommodations—yet ESAM systems may flag appropriate breaks as somehow inappropriate. Increasing the pace of work can also exacerbate underlying medical conditions and lead to job strain, which can cause or worsen a variety of conditions. The automated nature of these systems can also short-circuit the ability of disabled and pregnant workers to engage in an interactive process to request or identify a reasonable accommodation.

Consequently, using ESAM to alter productivity requirements, subject workers to intrusive monitoring, and enforce one-size-fits-all requirements will often tend to disadvantage and discriminate against workers with protected characteristics. Furthermore, most workers lack meaningful protection from this surveillance. Federal law currently does not provide workers with meaningful electronic surveillance or workplace privacy protections and, aside from California’s Privacy Rights Act, the series of comprehensive privacy laws that many states have recently enacted exclude the employment context from their protections.

Given these considerations, the EEOC should ensure that employers relying on ESAM understand the impact of these systems on their workforce as well as their continuing legal obligations under employment anti-discrimination laws. The EEOC should detail employers’ obligations to ensure that deployments of ESAM do not violate civil rights laws. Such rules or guidance should include:

• Clarifying that ESAM practices that tend to disadvantage protected workers can violate applicable anti-discrimination laws if they negatively impact the compensation, terms, conditions, or privileges of affected workers’ employment or result in loss of employment.

• Identifying steps employers should take to comply with the ADA, the PWFA, the Rehabilitation Act, and the Pregnancy Discrimination Act (PDA), including employers’ obligation to ensure that deployments of ESAM do not threaten the rights of disabled, pregnant, and other protected workers, such as the right to reasonable accommodations. This should include providing notice of electronic monitoring and providing workers with information on how they can request accommodations.

Issue guidance on vendor liability

As the EEOC is aware, many employers utilize screening tools and assessments developed by third-party software vendors in their hiring processes, as well as third-party recruiting and ad-targeting platforms. The design and functioning of these tools has an enormous and often determinative impact on recruiting and hiring decisions: screening and assessment tools regularly reject or rank and filter down applicants such that they will not be considered by a human recruiter, while sourcing, recruiting, and ad-targeting platforms determine who will or will not get contacted or targeted for a job opportunity. While the EEOC has made clear that employers can be liable for civil rights violations when they discriminate through the use of tools or platforms developed by third parties, the EEOC should issue guidance that clarifies when digital platforms and software vendors can themselves be liable for the discriminatory functioning of their tools.

Title VII states that an employment agency includes “any person regularly undertaking . . . to procure employees for an employer or to procure for employees opportunities to work for an employer.” Existing EEOC guidance states that an “entity that regularly refers potential employees to employers or provides employers with the names of potential employees” can be liable as an employment agency. But the guidance does not provide sufficient clarity on this issue and needs to be updated to explicitly address the application of the law to the kinds of roles played by today’s vendors and platforms in the hiring and recruiting process. For example, the EEOC’s existing guidance on what constitutes an employment agency provides some examples of cases that have deemed various kinds of entities such as newspapers or professional associations to be employment agencies under certain circumstances. But it is unclear how those examples would compare to the ways that modern-day recruiting platforms function.

The EEOC should update its guidance to include examples of modern-day employment agencies such as a job-matching platform that algorithmically matches jobs to candidates and vice versa. Such entities are plainly in the business of procuring both employees for employers and employment opportunities for workers, bringing them within the Title VII definition. Likewise, when employers retain social media platforms to design or deliver targeted recruitment ads aimed at only selected groups of workers, the platforms are, in the words of the EEOC’s guidance, “refer[ring] potential employees to employers” and, when the users click on those ads, “provid[ing] employers with the names of potential employees.”

It would also be useful for the EEOC to directly address when a vendor whose tool is assessing and ranking candidates may be accountable under other theories of liability. For example, the EEOC can address whether a software vendor may be liable as an indirect employer for third-party interference with the relationship between the applicant and the employer using the vendor’s tools. Likewise, while the EEOC’s recent guidance on assessing the adverse impacts of ADSs and ESAM under Title VII states that software vendors may be agents of an employer “if the employer has given them authority to act on the employer’s behalf” or “an employer relies on the results of a selection procedure that an agent administers on its behalf,” the EEOC could further clarify what kind of reliance or delegation of authority would be sufficient and include examples based on some of the different ways that employers commonly work with ADS or ESAM vendors.

Such guidance is critical for several reasons. First, EEOC guidance in this area would be extremely useful to courts, which are beginning to grapple with the question of when vendors and platforms may be liable as employment agencies or otherwise, and could benefit from the EEOC’s expertise in both the law and the intricacies of how these tools work. Second, to the extent that the EEOC finds that vendors or platforms may potentially be held liable under certain circumstances for their discriminatory tools, it would provide such entities with greater incentives to better design, audit, and deploy their tools in conformity with civil rights laws. Employers, especially smaller businesses with fewer resources, often lack the expertise and/or resources to readily and thoroughly assess a third party’s claims about its products. Third, clarifying vendor/platform liability would allow the EEOC and workers’ advocates to address more directly a critical upstream source of discrimination—the development and marketing of discriminatory tools that are used by numerous employers—instead of more inefficiently addressing discrimination one employer at a time.

Use authority to perform technical studies to increase agency information collection

The 2021 letter urged the Commission to help shed light on employers’ ADS practices by using its “authority to ‘make such technical studies as are appropriate to effectuate the purposes and policies of [Title VII] and to make the results of such studies available to the public.’” Information collected during such studies could reduce the massive information advantage employers and vendors currently enjoy, guide enforcement efforts at all levels of government, and spur the development of new industry standards.

The Commission could conduct such studies by partnering with agencies with greater experience in non-enforcement-related information-gathering. The Department of Labor’s Office of Federal Contractor Compliance Programs (OFCCP), for example, recently updated its scheduling letter to require federal contractors to provide information on their use of ADSs and similar assessment tools as part of audits. The Commission could work with the OFCCP to conduct studies using the information gathered through such audits, as well as through EEO-1 data (including the additional EEO-1 data recommended by subsection II.C of this letter).

As we noted in the original letter, “[g]iven the degree to which employers and vendors have an information advantage in this space, agencies should be proactive and creative in their strategies to collect data and gain glimpses into the nature and extent of employers’ use of hiring technologies.” That remains no less true today, and we urge the Commission to look for ways to use non-enforcement levers to close that information gap.

Pursue Commissioner charges and directed investigations to investigate discrimination related to ADSs and ESAM

The 2021 letter urged the Commission to pursue Commissioner charges under Title VII and the ADA, and directed investigations under the Age Discrimination in Employment Act (ADEA) and the Equal Pay Act, to investigate discrimination in the absence of individual complaints. Directed investigations and commissioners’ charges have the potential to be an important tool in enforcing prohibitions on algorithmic bias in employment. Because employees often lack information about whether and how their employer or prospective employer uses ADSs and ESAM, they may not have sufficient information to seek counsel and file discrimination complaints. Commissioner charges would allow the Commission to act upon information they receive to further investigate algorithmic bias, such as through the technical studies discussed above.

We applaud the EEOC for stating its intent to prioritize enforcement of technological discrimination throughout its 2024-2028 SEP. We were also pleased to see the Commission bring its first case involving an ADS, which alleged that iTutorGroup used an ADS to automatically reject women applicants over 55 and men over 60. We urge the EEOC to consider Commissioner charges as part of this enforcement effort, including against employers that use ADSs or ESAM that have a disparate impact on protected groups or otherwise violate civil rights laws.

Adopt an updated Internet Applicant Rule that applies to all applicants screened or evaluated by hiring technologies, and issue formal guidance on when targeted advertisements and other algorithmic screening techniques violate civil rights laws

The 2021 letter urged all federal agencies enforcing employment discrimination laws to adopt an updated version of OFCCP’s Internet Applicant Rule (IAR) that clarifies that covered employers must collect data on people screened or evaluated by targeted job advertisements and other “passive” candidate screening techniques. We reiterate this request and ask that the EEOC also issue formal guidance on when these screening techniques violate civil rights laws.

Employers use a variety of algorithmic tools, including targeted advertisements, to solicit applications for employment and screen potential applicants. Unfortunately, these tools can replicate existing patterns of occupational inequities and segregation. For example, researchers have found that algorithms used to decide who is shown advertisements regarding employment opportunities discriminate based on race and gender, often reflecting stereotypes about who works certain kinds of jobs. Significantly, a worker who does not receive a targeted ad, and thus is adversely impacted by the underlying algorithm, generally will not know of the ad’s existence at all, let alone the adverse impact they suffered.

At present, employers are not required to collect data on all people who are evaluated using such screening techniques, and there is insufficient guidance on when techniques that decide which potential candidates learn of job opportunities, or that otherwise pre-screen workers who have not formally applied for a job, qualify as actionable employment decisions. For example, the IAR, which applies to federal contractors’ recordkeeping obligations, considers candidates to be applicants only if they have affirmatively expressed an interest in a position—an overly narrow interpretation in light of the increasing role that targeted advertisements and other “passive candidate” screening techniques play in determining which workers are selected for employment.

We were gratified to see the SEP explicitly mention “targeted job advertisements” and recruiting methods as potential sources of unlawful discrimination. This decision is in keeping with the Blueprint for an AI Bill of Rights, which likewise critiques “[a]dvertisement delivery systems that predict who is most likely to click on a job advertisement end up delivering ads in ways that reinforce racial and gender stereotypes.”

The EEOC should work with OFCCP and other relevant agencies to update the IAR to require covered employers to collect data on all people who are evaluated by candidate screening techniques, regardless of whether they have taken steps to obtain a position, and otherwise harmonize relevant federal rules and regulations. In addition, the EEOC should work with Congress on legislation requiring all companies to disclose which targeted ads they use and how those ads were targeted.

The EEOC should also issue formal guidance or rules recognizing that when employers use targeted advertising and other sourcing/recruitment activities as the exclusive means of obtaining applicants for a position or to substantially limit the number of potential applicants, it is effectively a screening technique and thus subject to anti-discrimination laws covering employment decisions. This guidance would be consistent with existing case law holding that sourcing practices that have a disparate impact on who is hired can violate Title VII as well as the Uniform Guidelines for Employee Selection Procedures’ (UGESPs) recognition that an “employment decision” encompasses not only final hiring, promotion, retention, and similar decisions but also “[o]ther selection decisions . . . if they lead to any of” those final decisions.

Require employers to retain records about types of reasonable accommodations provided and methods of providing them, and include disability status data in EEO-1 reporting

To ensure greater transparency and accountability in how employers are complying with their obligation to provide reasonable accommodations under the ADA and Rehabilitation Act, the 2021 letter recommended that the EEOC “require employers to maintain records about the types of reasonable accommodations offered when hiring technologies are used.” Presently, 29 C.F.R. Part 1602 requires employers that keep employment records to preserve various personnel or employment records, including “requests for reasonable accommodation.” However, there is still no requirement that employers actually maintain records on how they responded to those requests.

The need for such recordkeeping has become more urgent in light of employers’ increasing use of one-size-fits-all ESAM and ADSs, often in the form of technologies that provide no obvious means of requesting or providing accommodation. The passage of PWFA in 2022 expanded the scope of workers with a right to reasonable accommodation, further underscoring the need for employers to keep records of their responses to accommodation requests. Consequently, we urge the EEOC to begin a rulemaking process that requires employers to maintain records of their responses to accommodation requests under applicable laws, including any specific accommodations requested, offered, provided, or denied.

Similarly, the EEOC should work with stakeholders to ensure that disability information can be appropriately collected as part of future EEO-1 data collections, as suggested in the 2021 letter. Disability status information could be collected using the Census Bureau’s disability categorization framework or through an alternative process developed with input from disability rights groups and other stakeholders. This data collection would both help bring further transparency to the participation of disabled workers in the workforce and also allow the EEOC to track the impact of employment decisions and management technologies on such workers, both at individual companies and in the workforce as a whole.

Hold workshops and convenings on hiring technologies

The 2021 letter recommended that the EEOC and other relevant agencies “hold workshops and convenings to gather information about industry practices.” Since then, the EEOC has held a variety of hearings, presentations, and other public-facing events on AI. These have included listening sessions, panel discussions hosted by the Miami and Memphis field offices, and full meetings of the Commission with ADSs as the focus. In addition, the team that works on the EEOC’s AI initiative has met regularly with members of civil rights, workers’ rights, and other civil society organizations to gather additional insights into how employers are using AI in the workplace and employment decisions. These events have provided the EEOC and the public with valuable information about how workers are experiencing AI and automation.

We hope to see the EEOC continue and expand on these efforts in the coming years. The EEOC field office sessions, for example, could either be incorporated regularly into CLEs that cover various topics or be part of an ongoing program that delves into these issues in greater detail. Organizers could tailor such sessions to provide information to employers or workers about how employers use AI and the risks associated with automated decision and management systems under federal anti-discrimination laws.

Additionally, the EEOC should hold a follow-up to the January 2023 full Commission hearing devoted to AI in the workplace, including broad and diverse perspectives, in advance of rulemaking and formal guidance on the other issues discussed in this letter.

Encourage employers and vendors to include members of vulnerable groups in the development and testing of ADSs

Finally, including diverse groups of workers in the development and testing of ADSs would be one potentially effective way to identify potential sources of disparate impact, inaccessibility, or other forms of discrimination before they have the chance to affect real-world employment decisions. Consequently, the EEOC should use its platform to encourage vendors and employers to incorporate members of vulnerable groups into the design, testing, and validation of modern selection procedures. It could do this through the avenues identified in earlier sections of this letter, including holding workshops and issuing technical assistance and other employer-facing documents.

Conclusion

Technological change in the workplace will continue to present new challenges to workers, employers, and regulators in the coming years. By taking the steps outlined in this letter, we believe the EEOC can help ensure that those changes work to benefit workers and prevent the use of technologies that reinforce or exacerbate existing inequities in the workplace and labor market. We thank the Commission for its attention and look forward to engaging with the EEOC further on these vital issues.