| 1 | 1 | | CALIFORNIA LEGISLATURE 20192020 REGULAR SESSION Assembly Bill No. 2269Introduced by Assembly Member ChauFebruary 14, 2020 An act to add Title 1.81.8 (commencing with Section 1798.400) to Part 4 of Division 3 of the Civil Code, relating to automatic decision systems. LEGISLATIVE COUNSEL'S DIGESTAB 2269, as introduced, Chau. Personal rights: automated decision systems.Existing law establishes the Department of Business Oversight in the Business, Consumer Services, and Housing Agency, headed by the Commissioner of Business Oversight. Under existing law, the department has charge of the execution of specified laws relating to various financial institutions and financial services.Existing law, the California Fair Employment and Housing Act, protects and safeguards the right and opportunity of all persons to seek, obtain, and hold employment without discrimination, abridgment, or harassment on account of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, or military and veteran status. Existing law regulates the use of personal information, including the California Consumer Privacy Act of 2018, which grants a consumer various rights with regard to personal information relating to that consumer that is held by a business. The act requires a business that collects personal information about a consumer to disclose the consumers right to delete personal information in a form that is reasonably accessible to consumers and in accordance with a specified process.This bill would enact the Automated Decision Systems Accountability Act of 2020. The bill would require a business in California that provides a person, as defined, with a program or device that uses an automated decision system (ADS) to take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS, conduct an ADS impact assessment on its program or device to determine whether the ADS has a disproportionate adverse impact on a protected class, as specified, examine if the ADS in question serves reasonable objectives and furthers a legitimate interest, and compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. The bill would require a business, by March 1, 2022, and annually thereafter, to submit a report to the Department of Business Oversight summarizing the results of its ADS impact assessment for each program or device that uses an ADS. The bill would also require a business, if it makes any significant modification to an ADS, to reconduct an ADS impact assessment under these circumstances. The bill would require the department, by January 1, 2022, to develop a procedure for businesses to use in making the required reports and to make general information on the reporting process available on its internet website. The bill would require the department, if a business fails to comply with these procedures, to send a written notice to the business regarding its failure to comply, and would require the business to submit the report within 60 days of the date of that notice. The bill would make violations of these provisions subject to a civil penalty. The bill would also require the department, by March 1, 2022, to establish an Automated Decision Systems Advisory Task Force, composed of various representatives from the public and private sectors, for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings.Digest Key Vote: MAJORITY Appropriation: NO Fiscal Committee: YES Local Program: NO Bill TextThe people of the State of California do enact as follows:SECTION 1. The Legislature finds and declares all of the following:(a) State law protects the rights of all persons in a variety of contexts without discrimination on account of certain protected characteristics, such as on the basis of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, or marital status, among other characteristics, as described in Section 12940 of the Government Code.(b) The rise of big data has raised concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly.(c) If the underlying data used for an algorithm or automated decision system is biased, incomplete, or discriminatory, the decisions made by using such devices has the potential to result in massive inequality.(d) The state has a legitimate and substantial interest in ensuring that automated decision systems used do not result in discrimination.(e) Therefore, the Legislature finds that it is necessary to require a review of the use of algorithmic decision systems also known as automated decision systems (ADS) in order to detect and prevent discrimination.SEC. 2. Title 1.81.8 (commencing with Section 1798.400) is added to Part 4 of Division 3 of the Civil Code, to read:TITLE 1.81.8. Automated Decision Systems Accountability Act of 20201798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020.1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind.1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 2 | 2 | | |
|---|
| 3 | 3 | | CALIFORNIA LEGISLATURE 20192020 REGULAR SESSION Assembly Bill No. 2269Introduced by Assembly Member ChauFebruary 14, 2020 An act to add Title 1.81.8 (commencing with Section 1798.400) to Part 4 of Division 3 of the Civil Code, relating to automatic decision systems. LEGISLATIVE COUNSEL'S DIGESTAB 2269, as introduced, Chau. Personal rights: automated decision systems.Existing law establishes the Department of Business Oversight in the Business, Consumer Services, and Housing Agency, headed by the Commissioner of Business Oversight. Under existing law, the department has charge of the execution of specified laws relating to various financial institutions and financial services.Existing law, the California Fair Employment and Housing Act, protects and safeguards the right and opportunity of all persons to seek, obtain, and hold employment without discrimination, abridgment, or harassment on account of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, or military and veteran status. Existing law regulates the use of personal information, including the California Consumer Privacy Act of 2018, which grants a consumer various rights with regard to personal information relating to that consumer that is held by a business. The act requires a business that collects personal information about a consumer to disclose the consumers right to delete personal information in a form that is reasonably accessible to consumers and in accordance with a specified process.This bill would enact the Automated Decision Systems Accountability Act of 2020. The bill would require a business in California that provides a person, as defined, with a program or device that uses an automated decision system (ADS) to take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS, conduct an ADS impact assessment on its program or device to determine whether the ADS has a disproportionate adverse impact on a protected class, as specified, examine if the ADS in question serves reasonable objectives and furthers a legitimate interest, and compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. The bill would require a business, by March 1, 2022, and annually thereafter, to submit a report to the Department of Business Oversight summarizing the results of its ADS impact assessment for each program or device that uses an ADS. The bill would also require a business, if it makes any significant modification to an ADS, to reconduct an ADS impact assessment under these circumstances. The bill would require the department, by January 1, 2022, to develop a procedure for businesses to use in making the required reports and to make general information on the reporting process available on its internet website. The bill would require the department, if a business fails to comply with these procedures, to send a written notice to the business regarding its failure to comply, and would require the business to submit the report within 60 days of the date of that notice. The bill would make violations of these provisions subject to a civil penalty. The bill would also require the department, by March 1, 2022, to establish an Automated Decision Systems Advisory Task Force, composed of various representatives from the public and private sectors, for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings.Digest Key Vote: MAJORITY Appropriation: NO Fiscal Committee: YES Local Program: NO |
|---|
| 4 | 4 | | |
|---|
| 5 | 5 | | |
|---|
| 6 | 6 | | |
|---|
| 7 | 7 | | |
|---|
| 8 | 8 | | |
|---|
| 9 | 9 | | CALIFORNIA LEGISLATURE 20192020 REGULAR SESSION |
|---|
| 10 | 10 | | |
|---|
| 11 | 11 | | Assembly Bill |
|---|
| 12 | 12 | | |
|---|
| 13 | 13 | | No. 2269 |
|---|
| 14 | 14 | | |
|---|
| 15 | 15 | | Introduced by Assembly Member ChauFebruary 14, 2020 |
|---|
| 16 | 16 | | |
|---|
| 17 | 17 | | Introduced by Assembly Member Chau |
|---|
| 18 | 18 | | February 14, 2020 |
|---|
| 19 | 19 | | |
|---|
| 20 | 20 | | An act to add Title 1.81.8 (commencing with Section 1798.400) to Part 4 of Division 3 of the Civil Code, relating to automatic decision systems. |
|---|
| 21 | 21 | | |
|---|
| 22 | 22 | | LEGISLATIVE COUNSEL'S DIGEST |
|---|
| 23 | 23 | | |
|---|
| 24 | 24 | | ## LEGISLATIVE COUNSEL'S DIGEST |
|---|
| 25 | 25 | | |
|---|
| 26 | 26 | | AB 2269, as introduced, Chau. Personal rights: automated decision systems. |
|---|
| 27 | 27 | | |
|---|
| 28 | 28 | | Existing law establishes the Department of Business Oversight in the Business, Consumer Services, and Housing Agency, headed by the Commissioner of Business Oversight. Under existing law, the department has charge of the execution of specified laws relating to various financial institutions and financial services.Existing law, the California Fair Employment and Housing Act, protects and safeguards the right and opportunity of all persons to seek, obtain, and hold employment without discrimination, abridgment, or harassment on account of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, or military and veteran status. Existing law regulates the use of personal information, including the California Consumer Privacy Act of 2018, which grants a consumer various rights with regard to personal information relating to that consumer that is held by a business. The act requires a business that collects personal information about a consumer to disclose the consumers right to delete personal information in a form that is reasonably accessible to consumers and in accordance with a specified process.This bill would enact the Automated Decision Systems Accountability Act of 2020. The bill would require a business in California that provides a person, as defined, with a program or device that uses an automated decision system (ADS) to take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS, conduct an ADS impact assessment on its program or device to determine whether the ADS has a disproportionate adverse impact on a protected class, as specified, examine if the ADS in question serves reasonable objectives and furthers a legitimate interest, and compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. The bill would require a business, by March 1, 2022, and annually thereafter, to submit a report to the Department of Business Oversight summarizing the results of its ADS impact assessment for each program or device that uses an ADS. The bill would also require a business, if it makes any significant modification to an ADS, to reconduct an ADS impact assessment under these circumstances. The bill would require the department, by January 1, 2022, to develop a procedure for businesses to use in making the required reports and to make general information on the reporting process available on its internet website. The bill would require the department, if a business fails to comply with these procedures, to send a written notice to the business regarding its failure to comply, and would require the business to submit the report within 60 days of the date of that notice. The bill would make violations of these provisions subject to a civil penalty. The bill would also require the department, by March 1, 2022, to establish an Automated Decision Systems Advisory Task Force, composed of various representatives from the public and private sectors, for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. |
|---|
| 29 | 29 | | |
|---|
| 30 | 30 | | Existing law establishes the Department of Business Oversight in the Business, Consumer Services, and Housing Agency, headed by the Commissioner of Business Oversight. Under existing law, the department has charge of the execution of specified laws relating to various financial institutions and financial services. |
|---|
| 31 | 31 | | |
|---|
| 32 | 32 | | Existing law, the California Fair Employment and Housing Act, protects and safeguards the right and opportunity of all persons to seek, obtain, and hold employment without discrimination, abridgment, or harassment on account of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, or military and veteran status. |
|---|
| 33 | 33 | | |
|---|
| 34 | 34 | | Existing law regulates the use of personal information, including the California Consumer Privacy Act of 2018, which grants a consumer various rights with regard to personal information relating to that consumer that is held by a business. The act requires a business that collects personal information about a consumer to disclose the consumers right to delete personal information in a form that is reasonably accessible to consumers and in accordance with a specified process. |
|---|
| 35 | 35 | | |
|---|
| 36 | 36 | | This bill would enact the Automated Decision Systems Accountability Act of 2020. The bill would require a business in California that provides a person, as defined, with a program or device that uses an automated decision system (ADS) to take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS, conduct an ADS impact assessment on its program or device to determine whether the ADS has a disproportionate adverse impact on a protected class, as specified, examine if the ADS in question serves reasonable objectives and furthers a legitimate interest, and compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. The bill would require a business, by March 1, 2022, and annually thereafter, to submit a report to the Department of Business Oversight summarizing the results of its ADS impact assessment for each program or device that uses an ADS. The bill would also require a business, if it makes any significant modification to an ADS, to reconduct an ADS impact assessment under these circumstances. |
|---|
| 37 | 37 | | |
|---|
| 38 | 38 | | The bill would require the department, by January 1, 2022, to develop a procedure for businesses to use in making the required reports and to make general information on the reporting process available on its internet website. The bill would require the department, if a business fails to comply with these procedures, to send a written notice to the business regarding its failure to comply, and would require the business to submit the report within 60 days of the date of that notice. The bill would make violations of these provisions subject to a civil penalty. |
|---|
| 39 | 39 | | |
|---|
| 40 | 40 | | The bill would also require the department, by March 1, 2022, to establish an Automated Decision Systems Advisory Task Force, composed of various representatives from the public and private sectors, for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. |
|---|
| 41 | 41 | | |
|---|
| 42 | 42 | | ## Digest Key |
|---|
| 43 | 43 | | |
|---|
| 44 | 44 | | ## Bill Text |
|---|
| 45 | 45 | | |
|---|
| 46 | 46 | | The people of the State of California do enact as follows:SECTION 1. The Legislature finds and declares all of the following:(a) State law protects the rights of all persons in a variety of contexts without discrimination on account of certain protected characteristics, such as on the basis of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, or marital status, among other characteristics, as described in Section 12940 of the Government Code.(b) The rise of big data has raised concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly.(c) If the underlying data used for an algorithm or automated decision system is biased, incomplete, or discriminatory, the decisions made by using such devices has the potential to result in massive inequality.(d) The state has a legitimate and substantial interest in ensuring that automated decision systems used do not result in discrimination.(e) Therefore, the Legislature finds that it is necessary to require a review of the use of algorithmic decision systems also known as automated decision systems (ADS) in order to detect and prevent discrimination.SEC. 2. Title 1.81.8 (commencing with Section 1798.400) is added to Part 4 of Division 3 of the Civil Code, to read:TITLE 1.81.8. Automated Decision Systems Accountability Act of 20201798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020.1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind.1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 47 | 47 | | |
|---|
| 48 | 48 | | The people of the State of California do enact as follows: |
|---|
| 49 | 49 | | |
|---|
| 50 | 50 | | ## The people of the State of California do enact as follows: |
|---|
| 51 | 51 | | |
|---|
| 52 | 52 | | SECTION 1. The Legislature finds and declares all of the following:(a) State law protects the rights of all persons in a variety of contexts without discrimination on account of certain protected characteristics, such as on the basis of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, or marital status, among other characteristics, as described in Section 12940 of the Government Code.(b) The rise of big data has raised concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly.(c) If the underlying data used for an algorithm or automated decision system is biased, incomplete, or discriminatory, the decisions made by using such devices has the potential to result in massive inequality.(d) The state has a legitimate and substantial interest in ensuring that automated decision systems used do not result in discrimination.(e) Therefore, the Legislature finds that it is necessary to require a review of the use of algorithmic decision systems also known as automated decision systems (ADS) in order to detect and prevent discrimination. |
|---|
| 53 | 53 | | |
|---|
| 54 | 54 | | SECTION 1. The Legislature finds and declares all of the following:(a) State law protects the rights of all persons in a variety of contexts without discrimination on account of certain protected characteristics, such as on the basis of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, or marital status, among other characteristics, as described in Section 12940 of the Government Code.(b) The rise of big data has raised concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly.(c) If the underlying data used for an algorithm or automated decision system is biased, incomplete, or discriminatory, the decisions made by using such devices has the potential to result in massive inequality.(d) The state has a legitimate and substantial interest in ensuring that automated decision systems used do not result in discrimination.(e) Therefore, the Legislature finds that it is necessary to require a review of the use of algorithmic decision systems also known as automated decision systems (ADS) in order to detect and prevent discrimination. |
|---|
| 55 | 55 | | |
|---|
| 56 | 56 | | SECTION 1. The Legislature finds and declares all of the following: |
|---|
| 57 | 57 | | |
|---|
| 58 | 58 | | ### SECTION 1. |
|---|
| 59 | 59 | | |
|---|
| 60 | 60 | | (a) State law protects the rights of all persons in a variety of contexts without discrimination on account of certain protected characteristics, such as on the basis of race, religious creed, color, national origin, ancestry, physical disability, mental disability, medical condition, genetic information, or marital status, among other characteristics, as described in Section 12940 of the Government Code. |
|---|
| 61 | 61 | | |
|---|
| 62 | 62 | | (b) The rise of big data has raised concerns about the use of algorithmic or automated decision systems to make hiring and other workplace decisions, eligibility decisions, insurance eligibility, lending decisions, and marketing decisions quickly, automatically, and fairly. |
|---|
| 63 | 63 | | |
|---|
| 64 | 64 | | (c) If the underlying data used for an algorithm or automated decision system is biased, incomplete, or discriminatory, the decisions made by using such devices has the potential to result in massive inequality. |
|---|
| 65 | 65 | | |
|---|
| 66 | 66 | | (d) The state has a legitimate and substantial interest in ensuring that automated decision systems used do not result in discrimination. |
|---|
| 67 | 67 | | |
|---|
| 68 | 68 | | (e) Therefore, the Legislature finds that it is necessary to require a review of the use of algorithmic decision systems also known as automated decision systems (ADS) in order to detect and prevent discrimination. |
|---|
| 69 | 69 | | |
|---|
| 70 | 70 | | SEC. 2. Title 1.81.8 (commencing with Section 1798.400) is added to Part 4 of Division 3 of the Civil Code, to read:TITLE 1.81.8. Automated Decision Systems Accountability Act of 20201798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020.1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind.1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 71 | 71 | | |
|---|
| 72 | 72 | | SEC. 2. Title 1.81.8 (commencing with Section 1798.400) is added to Part 4 of Division 3 of the Civil Code, to read: |
|---|
| 73 | 73 | | |
|---|
| 74 | 74 | | ### SEC. 2. |
|---|
| 75 | 75 | | |
|---|
| 76 | 76 | | TITLE 1.81.8. Automated Decision Systems Accountability Act of 20201798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020.1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind.1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 77 | 77 | | |
|---|
| 78 | 78 | | TITLE 1.81.8. Automated Decision Systems Accountability Act of 20201798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020.1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind.1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 79 | 79 | | |
|---|
| 80 | 80 | | TITLE 1.81.8. Automated Decision Systems Accountability Act of 2020 |
|---|
| 81 | 81 | | |
|---|
| 82 | 82 | | TITLE 1.81.8. Automated Decision Systems Accountability Act of 2020 |
|---|
| 83 | 83 | | |
|---|
| 84 | 84 | | 1798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020. |
|---|
| 85 | 85 | | |
|---|
| 86 | 86 | | |
|---|
| 87 | 87 | | |
|---|
| 88 | 88 | | 1798.400. This act shall be known and may be cited as the Automated Decision Systems Accountability Act of 2020. |
|---|
| 89 | 89 | | |
|---|
| 90 | 90 | | 1798.401. For the purposes of this title, the following definitions apply:(a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following:(1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose.(2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following:(A) Data minimization practices.(B) The duration for which personal information and the results of the ADS are stored.(C) What information about the ADS is available to consumers.(D) The extent to which consumers have access to the results of the ADS and may correct or object to its results.(E) The recipients of the results of the ADS.(F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers.(G) The measures the business will employ to minimize the risks, including technological and physical safeguards.(c) Business means a digital or software company that creates or distributes an ADS.(d) Department means the Department of Business Oversight.(e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind. |
|---|
| 91 | 91 | | |
|---|
| 92 | 92 | | |
|---|
| 93 | 93 | | |
|---|
| 94 | 94 | | 1798.401. For the purposes of this title, the following definitions apply: |
|---|
| 95 | 95 | | |
|---|
| 96 | 96 | | (a) Automated decision system or ADS means a computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that makes a decision or facilitates human decision making, that impacts persons. |
|---|
| 97 | 97 | | |
|---|
| 98 | 98 | | (b) Automated decision system impact assessment or ADS impact assessment means a study evaluating an ADS and the ADSs development process, including, but not limited to, the design and training data of the ADS, for impacts on accuracy, fairness, bias, discrimination, privacy, and security that includes, at a minimum, all of the following: |
|---|
| 99 | 99 | | |
|---|
| 100 | 100 | | (1) A detailed description of the ADS, its design, training provided on its use, its data, and its purpose. |
|---|
| 101 | 101 | | |
|---|
| 102 | 102 | | (2) An assessment of the relative benefits and costs of the ADS in light of its purpose, taking into account relevant factors, including, but not limited to, all of the following: |
|---|
| 103 | 103 | | |
|---|
| 104 | 104 | | (A) Data minimization practices. |
|---|
| 105 | 105 | | |
|---|
| 106 | 106 | | (B) The duration for which personal information and the results of the ADS are stored. |
|---|
| 107 | 107 | | |
|---|
| 108 | 108 | | (C) What information about the ADS is available to consumers. |
|---|
| 109 | 109 | | |
|---|
| 110 | 110 | | (D) The extent to which consumers have access to the results of the ADS and may correct or object to its results. |
|---|
| 111 | 111 | | |
|---|
| 112 | 112 | | (E) The recipients of the results of the ADS. |
|---|
| 113 | 113 | | |
|---|
| 114 | 114 | | (F) An assessment of the risks posed by the ADS to the privacy or security of personal information of consumers and the risks that the ADS may result in or contribute to inaccurate, unfair, biased, or discriminatory decisions impacting consumers. |
|---|
| 115 | 115 | | |
|---|
| 116 | 116 | | (G) The measures the business will employ to minimize the risks, including technological and physical safeguards. |
|---|
| 117 | 117 | | |
|---|
| 118 | 118 | | (c) Business means a digital or software company that creates or distributes an ADS. |
|---|
| 119 | 119 | | |
|---|
| 120 | 120 | | (d) Department means the Department of Business Oversight. |
|---|
| 121 | 121 | | |
|---|
| 122 | 122 | | (e) Person means an individual, firm, association, organization, partnership, limited liability company, business trust, corporation, or public entity of any kind. |
|---|
| 123 | 123 | | |
|---|
| 124 | 124 | | 1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following:(1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS.(2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following:(A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility.(B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest.(C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. |
|---|
| 125 | 125 | | |
|---|
| 126 | 126 | | |
|---|
| 127 | 127 | | |
|---|
| 128 | 128 | | 1798.402. (a) A business in California that provides a person with a program or device that uses an ADS shall do all of the following: |
|---|
| 129 | 129 | | |
|---|
| 130 | 130 | | (1) Take affirmative steps to ensure that there are processes in place to continually test for biases during the development and usage of the ADS. |
|---|
| 131 | 131 | | |
|---|
| 132 | 132 | | (2) Conduct an ADS impact assessment on its program or device that uses an ADS to do all of the following: |
|---|
| 133 | 133 | | |
|---|
| 134 | 134 | | (A) Determine whether the ADS under review has a disproportionate adverse impact on a protected class, as described in Section 12940 of the Government Code. A business may contract with a third party to independently create the ADS impact assessment for the purpose of providing an additional level of credibility. |
|---|
| 135 | 135 | | |
|---|
| 136 | 136 | | (B) Examine if the ADS in question serves reasonable objectives and furthers a legitimate interest. |
|---|
| 137 | 137 | | |
|---|
| 138 | 138 | | (C) Compare the ADS to alternatives or reasonable modifications that may be taken to limit adverse consequences on protected classes. |
|---|
| 139 | 139 | | |
|---|
| 140 | 140 | | (3) On or before March 1, 2022, and annually thereafter, a business shall submit a report to the department, in a format developed by the department pursuant to subdivision (c), summarizing the results of its ADS impact assessment for each program or device that uses an ADS. If a business makes any significant modification to an ADS, the business shall reconduct an ADS impact assessment and resubmit the results of that assessment to the department no later than 60 days from the modification. |
|---|
| 141 | 141 | | |
|---|
| 142 | 142 | | (b) On or before January 1, 2022, the department shall develop a procedure, including a form, if necessary, for businesses to use in making the reports required pursuant to this section. The department also shall make general information on the reporting process accessible on its internet website on or before January 1, 2022. |
|---|
| 143 | 143 | | |
|---|
| 144 | 144 | | (c) If a business fails to comply with this section, the department shall send a written notice to the business of its failure to comply. The business shall have 60 days from the date of the written notice in which to comply, by completing the report and submitting it to the department. Failure by a business to submit the required report shall result in a civil penalty. |
|---|
| 145 | 145 | | |
|---|
| 146 | 146 | | 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following:(a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code.(b) Two members from state or local government agencies.(c) Two representatives from digital or software companies who use or create automated decision systems.(d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|
| 147 | 147 | | |
|---|
| 148 | 148 | | |
|---|
| 149 | 149 | | |
|---|
| 150 | 150 | | 1798.403. On or before March 1, 2022, the department shall establish an Automated Decision Systems Advisory Task Force for the purpose of reviewing and providing advice on the use of automated decision systems in businesses, government, and various other settings. The task force shall consist of all of the following: |
|---|
| 151 | 151 | | |
|---|
| 152 | 152 | | (a) Two representatives from advocacy organizations that represent consumers or protected classes of communities, as described in Section 12940 of the Government Code. |
|---|
| 153 | 153 | | |
|---|
| 154 | 154 | | (b) Two members from state or local government agencies. |
|---|
| 155 | 155 | | |
|---|
| 156 | 156 | | (c) Two representatives from digital or software companies who use or create automated decision systems. |
|---|
| 157 | 157 | | |
|---|
| 158 | 158 | | (d) Two representatives from universities or research institutions with expertise in automated decision systems. |
|---|