Critical infrastructure: artificial intelligence systems: human oversight.

Existing law, the California Emergency Services Act, establishes the California Cybersecurity Integration Center within the Office of Emergency Services to serve as the central organizing hub of state government's cybersecurity activities and to coordinate information sharing with various entities. Existing law also requires the Technology Recovery Plan element of the State Administrative Manual to ensure the inclusion of cybersecurity strategy incident response standards for each state agency to secure its critical infrastructure controls and information, as prescribed. This bill would require, on or before July 1, 2026, an operator, defined as a state agency responsible for operating, managing, overseeing, or controlling access to critical infrastructure, that deploys a covered artificial intelligence (AI) system, as defined, to establish a human oversight mechanism that ensures a human monitors the system's operations in real time and reviews and approves any plan or action proposed by the covered AI system before execution, except as provided. The bill would require the Department of Technology to develop specialized training in AI safety protocols and risk management techniques to oversight personnel. The bill would require oversight personnel for an operator to conduct an annual assessment of its covered AI systems, as specified, and to submit a summary of the findings to the department. The bill would make findings and declarations related to its provisions. The bill would preclude disclosure of specified information by the office. Existing constitutional provisions require that a statute that limits the right of access to the meetings of public bodies or the writings of public officials and agencies be adopted with findings demonstrating the interest protected by the limitation and the need for protecting that interest. This bill would make legislative findings to that effect.