Amended IN Senate March 26, 2025 CALIFORNIA LEGISLATURE 20252026 REGULAR SESSION Senate Bill No. 833Introduced by Senator McNerneyFebruary 21, 2025An act relating to artificial intelligence. An act to add Section 8592.51 to the Government Code, relating to state government.LEGISLATIVE COUNSEL'S DIGESTSB 833, as amended, McNerney. Artificial intelligence: critical infrastructure. Critical infrastructure: automated decision systems: human oversight.Existing law, the California Emergency Services Act, establishes the California Cybersecurity Integration Center within the Office of Emergency Services to serve as the central organizing hub of state governments cybersecurity activities and to coordinate information sharing with various entities. Existing law also requires the Technology Recovery Plan element of the State Administrative Manual to ensure the inclusion of cybersecurity strategy incident response standards for each state agency to secure its critical infrastructure controls and information, as prescribed.This bill would require an operator, defined as a state agency in charge of critical infrastructure, that deploys artificial intelligence to establish a human oversight mechanism to monitor the systems operations in real time and review and approve any plan or action proposed by the artificial intelligence system before execution, except as provided. The bill would require the Department of Technology to administer specialized training in artificial intelligence safety protocols and risk management techniques to oversight personnel. The bill would require an operator to conduct an annual assessment of its artificial intelligence systems and automated decision systems, as specified, and to submit a summary of the findings to the department.Existing law, the Generative Artificial Intelligence Accountability Act, among other things, requires the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the Department of Human Resources, to update the report to the Governor, as required by Executive Order No. N-12-23, as prescribed, and requires the Office of Emergency Services to perform, as appropriate, a risk analysis of potential threats posed by the use of generative AI to Californias critical infrastructure, including those that could lead to mass casualty events. This bill would declare the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure.Digest Key Vote: MAJORITY Appropriation: NO Fiscal Committee: NOYES Local Program: NO Bill TextThe people of the State of California do enact as follows:SECTION 1. Section 8592.51 is added to the Government Code, to read:8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.SECTION 1.It is the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure without human oversight.
Amended IN Senate March 26, 2025 CALIFORNIA LEGISLATURE 20252026 REGULAR SESSION Senate Bill No. 833Introduced by Senator McNerneyFebruary 21, 2025An act relating to artificial intelligence. An act to add Section 8592.51 to the Government Code, relating to state government.LEGISLATIVE COUNSEL'S DIGESTSB 833, as amended, McNerney. Artificial intelligence: critical infrastructure. Critical infrastructure: automated decision systems: human oversight.Existing law, the California Emergency Services Act, establishes the California Cybersecurity Integration Center within the Office of Emergency Services to serve as the central organizing hub of state governments cybersecurity activities and to coordinate information sharing with various entities. Existing law also requires the Technology Recovery Plan element of the State Administrative Manual to ensure the inclusion of cybersecurity strategy incident response standards for each state agency to secure its critical infrastructure controls and information, as prescribed.This bill would require an operator, defined as a state agency in charge of critical infrastructure, that deploys artificial intelligence to establish a human oversight mechanism to monitor the systems operations in real time and review and approve any plan or action proposed by the artificial intelligence system before execution, except as provided. The bill would require the Department of Technology to administer specialized training in artificial intelligence safety protocols and risk management techniques to oversight personnel. The bill would require an operator to conduct an annual assessment of its artificial intelligence systems and automated decision systems, as specified, and to submit a summary of the findings to the department.Existing law, the Generative Artificial Intelligence Accountability Act, among other things, requires the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the Department of Human Resources, to update the report to the Governor, as required by Executive Order No. N-12-23, as prescribed, and requires the Office of Emergency Services to perform, as appropriate, a risk analysis of potential threats posed by the use of generative AI to Californias critical infrastructure, including those that could lead to mass casualty events. This bill would declare the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure.Digest Key Vote: MAJORITY Appropriation: NO Fiscal Committee: NOYES Local Program: NO
Amended IN Senate March 26, 2025
Amended IN Senate March 26, 2025
CALIFORNIA LEGISLATURE 20252026 REGULAR SESSION
Senate Bill
No. 833
Introduced by Senator McNerneyFebruary 21, 2025
Introduced by Senator McNerney
February 21, 2025
An act relating to artificial intelligence. An act to add Section 8592.51 to the Government Code, relating to state government.
LEGISLATIVE COUNSEL'S DIGEST
## LEGISLATIVE COUNSEL'S DIGEST
SB 833, as amended, McNerney. Artificial intelligence: critical infrastructure. Critical infrastructure: automated decision systems: human oversight.
Existing law, the California Emergency Services Act, establishes the California Cybersecurity Integration Center within the Office of Emergency Services to serve as the central organizing hub of state governments cybersecurity activities and to coordinate information sharing with various entities. Existing law also requires the Technology Recovery Plan element of the State Administrative Manual to ensure the inclusion of cybersecurity strategy incident response standards for each state agency to secure its critical infrastructure controls and information, as prescribed.This bill would require an operator, defined as a state agency in charge of critical infrastructure, that deploys artificial intelligence to establish a human oversight mechanism to monitor the systems operations in real time and review and approve any plan or action proposed by the artificial intelligence system before execution, except as provided. The bill would require the Department of Technology to administer specialized training in artificial intelligence safety protocols and risk management techniques to oversight personnel. The bill would require an operator to conduct an annual assessment of its artificial intelligence systems and automated decision systems, as specified, and to submit a summary of the findings to the department.Existing law, the Generative Artificial Intelligence Accountability Act, among other things, requires the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the Department of Human Resources, to update the report to the Governor, as required by Executive Order No. N-12-23, as prescribed, and requires the Office of Emergency Services to perform, as appropriate, a risk analysis of potential threats posed by the use of generative AI to Californias critical infrastructure, including those that could lead to mass casualty events. This bill would declare the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure.
Existing law, the California Emergency Services Act, establishes the California Cybersecurity Integration Center within the Office of Emergency Services to serve as the central organizing hub of state governments cybersecurity activities and to coordinate information sharing with various entities. Existing law also requires the Technology Recovery Plan element of the State Administrative Manual to ensure the inclusion of cybersecurity strategy incident response standards for each state agency to secure its critical infrastructure controls and information, as prescribed.
This bill would require an operator, defined as a state agency in charge of critical infrastructure, that deploys artificial intelligence to establish a human oversight mechanism to monitor the systems operations in real time and review and approve any plan or action proposed by the artificial intelligence system before execution, except as provided. The bill would require the Department of Technology to administer specialized training in artificial intelligence safety protocols and risk management techniques to oversight personnel. The bill would require an operator to conduct an annual assessment of its artificial intelligence systems and automated decision systems, as specified, and to submit a summary of the findings to the department.
Existing law, the Generative Artificial Intelligence Accountability Act, among other things, requires the Department of Technology, under the guidance of the Government Operations Agency, the Office of Data and Innovation, and the Department of Human Resources, to update the report to the Governor, as required by Executive Order No. N-12-23, as prescribed, and requires the Office of Emergency Services to perform, as appropriate, a risk analysis of potential threats posed by the use of generative AI to Californias critical infrastructure, including those that could lead to mass casualty events.
This bill would declare the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure.
## Digest Key
## Bill Text
The people of the State of California do enact as follows:SECTION 1. Section 8592.51 is added to the Government Code, to read:8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.SECTION 1.It is the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure without human oversight.
The people of the State of California do enact as follows:
## The people of the State of California do enact as follows:
SECTION 1. Section 8592.51 is added to the Government Code, to read:8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.
SECTION 1. Section 8592.51 is added to the Government Code, to read:
### SECTION 1.
8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.
8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.
8592.51. (a) For purposes of this section, the following definitions apply:(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:(A) Transportation.(B) Energy.(C) Food and agriculture.(D) Communications.(E) Emergency services.(F) Financial services.(4) Operator means a state agency in charge of critical infrastructure.(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:(A) Monitor the artificial intelligence systems operations in real time.(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:(1) Evaluates compliance with this section.(2) Evaluates system performance and safety.(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.(e) An operator shall submit a summary of the assessment findings to the Department of Technology.(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.
8592.51. (a) For purposes of this section, the following definitions apply:
(1) Artificial intelligence means an engineered or machine-based system that varies in its level of autonomy that can, for explicit or implicit objectives, infer from the input it receives how to generate outputs that can influence physical or virtual environments.
(2) Automated decision system means a computational process derived from machine learning, statistical modeling, data analytics, or artificial intelligence that issues simplified output, including a score, classification, or recommendation, that is used to assist or replace human discretionary decisionmaking and materially impacts natural persons. Automated decision system does not include a spam email filter, firewall, antivirus software, identity and access management tools, or a calculator.
(3) Critical infrastructure means systems or assets so vital to the state that the incapacity or destruction of those networks, systems, or assets would have a debilitating impact on public health, safety, economic security, or any combination thereof, including any of the following infrastructure:
(A) Transportation.
(B) Energy.
(C) Food and agriculture.
(D) Communications.
(E) Emergency services.
(F) Financial services.
(4) Operator means a state agency in charge of critical infrastructure.
(b) (1) An operator deploying artificial intelligence shall establish a human oversight mechanism to do both of the following:
(A) Monitor the artificial intelligence systems operations in real time.
(B) Review and approve any plan or action proposed by an artificial intelligence system before execution.
(2) This subdivision shall not apply to an existing automated decision system that is critical to state infrastructure if the required human oversight would cause an immediate pause that would destabilize that system.
(c) The Department of Technology shall administer specialized training in artificial intelligence safety protocols and risk management techniques to be given to oversight personnel.
(d) An operator shall conduct an annual assessment of its artificial intelligence systems and automated decision systems that does all of the following:
(1) Evaluates compliance with this section.
(2) Evaluates system performance and safety.
(3) Identifies and evaluates potential risks and vulnerabilities, including those that could lead to mass casualty events.
(e) An operator shall submit a summary of the assessment findings to the Department of Technology.
(f) The assessment shall coincide with any requirement on the operator to perform a risk analysis pursuant to subdivision (b) of Section 11549.65.
It is the intent of the Legislature to enact subsequent legislation that would prohibit artificial intelligence from making a plan or executing a plan on critical infrastructure without human oversight.