(First of two parts)
Most organizations didn’t adopt AI with a single decision. Business Intelligence (BI) dashboards evolved into predictive systems, then generative AI tools, and now autonomous agents acting on the enterprise’s behalf. Each new capability introduces distinct risks and requires tailored controls. Governance designed for one stage will not protect the organization at the next. These layers often coexist and mature unevenly across the business; they are not a strict ladder. Boards should regularly assess which layers are in use, how they connect, and whether governance has kept pace. By staying proactive, Directors help the organization anticipate and manage emerging risks.
Boards govern and challenge; management implements. That role has two sides. On performance, directors ask whether data and AI are advancing strategy, productivity, customer outcomes, and resilience. On conformance, they ask whether risk appetite, accountability, controls, and assurance are in place. Governance demands both performance and conformance.
This piece presents a framework for data and AI governance, organized into four layers: BI, Predictive Machine Learning, Generative AI, and AI Agents. For each layer, it outlines objectives, core controls, common failure modes, and key questions for Boards and Risk or Audit Committees.
THE GOVERNING PRINCIPLE: RISK MOVES FROM THE DATA TO THE DECISION
As systems progress from reporting to predicting, generating, and acting, the primary risk shifts from data accuracy to the soundness and limitations of system decisions. In BI, risks center on data quality and access. For predictive models, reliability and fairness are paramount. Generative AI introduces risks of data loss, privacy breaches, and inaccurate content. For agents, the main concerns are the extent of their autonomy and our ability to monitor and intervene.
The layers are cumulative. An agent using a generative AI model that relies on predictive scores and enterprise data inherits the vulnerabilities of all underlying layers. Governance that focuses only on the newest layer leaves foundational risks unaddressed. Boards should require evidence of strong controls and oversight at every layer. This comprehensive approach enhances organizational resilience and accountability.
LAYER 1: GOVERNING BI
Objective: a single source of truth with high accuracy and auditability.
BI is the oldest and most overlooked layer, yet it underpins financial reporting, regulatory submissions, and most management decisions. Errors at this stage are carried forward into all downstream models and agents.
Core controls:
• Role-based access so that people see only the data their roles require.
• Data governance as an enterprise program with defined data owners, stewards, and quality standards.
• Data dictionaries and metadata management so that terms, such as “active customer” or “net revenue,” have the same meaning everywhere.
• Master data management to maintain authoritative records for customers, products, suppliers, and counterparties.
• An enterprise data catalog that documents what data exists, where it comes from, and how it moves (data lineage).
Common failures include inaccurate source records and broken extract-transform-load (ETL) data pipelines. Outdated logic is especially problematic because calculations that were once correct may continue to produce inaccurate results after changes to products, policies, or accounting standards.
Board and Risk/Audit focus: Boards should ensure compliance with applicable financial reporting standards (such as IFRS/PFRS) and with the reporting and internal control requirements of the relevant securities regulator, such as the SEC. Data retention mandates, access controls, and the maturity of the data governance program should also be on the agenda. Audit Committees should ask whether the data used to prepare financial and regulatory reports is traceable from source to disclosure.
LAYER 2: GOVERNING PREDICTIVE MACHINE LEARNING
Objective: ensure that probabilistic outputs remain accurate, stable, and ethically sound over time.
Predictive systems are used to score credit applicants, price insurance contracts, detect financial fraud, forecast demand, and screen job candidates. Unlike BI, they generate probabilities rather than facts and degrade over time as customer behavior and real-world conditions change.
Core controls:
• Drift monitoring. Two types of drift require attention. Data drift occurs when the statistical properties of input data change, such as when a new customer segment is added. Concept drift occurs when the relationship between inputs and outcomes shifts, for example, due to changing economic conditions. Both can gradually reduce model effectiveness.
• Explainability (XAI). Techniques, such as SHAP and LIME, help identify which factors drove a particular outcome. For high-impact decisions, such as those in credit, underwriting, and hiring, the organization must be able to justify the outcome to a customer, a regulator, or a court.
• Fairness and bias audits. Test models across population segments before deployment to detect disparate impact, defined as a disproportionately adverse outcome for a protected group, even if unintentional. Ongoing testing is required after deployment.
Typical failures include silent performance decay, in which model accuracy declines without detection; feedback loops that reinforce historical human bias; overfitting; and excessive false positives, which raise operational and customer costs.
Board and Risk/Audit focus: The central issues are algorithmic bias liability, consumer protection compliance, and model risk management. In the Philippines, the Bangko Sentral ng Pilipinas’ credit risk management guidelines under Circular No. 855 address credit scoring models and risk-rating systems. Beyond the circular’s specific requirements, the central bank’s supervisory expectations in recent years have pointed banks toward full life cycle model risk management: an inventory of models, independent validation, ongoing monitoring, and clear accountability. To support effective oversight, Boards should require regular management reporting on the complete model inventory, the status of independent validation, and the results of ongoing model monitoring and bias audits. This keeps Directors informed about how risks are managed in practice and enables timely, risk-based oversight. (The Bangko Sentral has also issued guidance on the ethical and responsible use of AI. Boards should confirm whether it is binding or an expectation.)
LAYER 3: GOVERNING GENERATIVE AI
Objective: reduce hallucinations, safeguard proprietary intellectual property and personal data, and prevent confidential information from leaving the enterprise.
Generative AI is easy to adopt because any employee with a browser (or by downloading the free AI mobile apps) can enter sensitive information into external tools. The primary governance challenge is preventing data from leaving the organization, not model effectiveness.
Core controls:
• Model gateway. A model gateway sits between users or applications and foundation models. It screens prompts and files to ensure that confidential information and customer personally identifiable information (PII) do not reach external models or become part of vendor training data. It also provides centralized logging and access control.
• Guardrails. Real-time guardrails screen inputs and outputs for toxicity, jailbreak attempts, prompt injection attacks, and likely hallucinations. While these controls reduce risk, they do not eliminate it. Generative models may still produce convincing but fabricated content, so critical use cases require human review and validation against verified enterprise sources.
Typical failures include hallucinated or fabricated data, such as fabricated citations and figures; data exfiltration and loss through unauthorized tools; and copyright infringement, either from generated content or from employee-submitted material.
Board and Risk/Audit focus: Boards should align governance with strategy and confirm that AI investments support business objectives. Oversight of AI vendors is critical. Boards must require robust due diligence on any external AI vendor, focusing on three key contract terms: 1.) clear data-use restrictions to prevent vendors from retaining or reusing enterprise data, 2.) enforceable commitments to zero data retention, and, 3.) strong indemnification and liability provisions for intellectual property infringement. Boards should also expect management to regularly review vendors’ s ecurity practices and audit rights as part of ongoing oversight.
Enterprise risk policies should address “Shadow AI,” defined as employee use of unapproved AI tools, and management should demonstrate its ability to detect and manage such use. The Board should also expect a plan to comply with emerging AI regulations in the Philippines and in other jurisdictions where the organization operates. The 2024 EU AI Act is binding only where the organization operates in or serves the EU; elsewhere, it serves as a reference point.
LAYER 4: GOVERNING AI AGENTS
Objective: enforce boundaries on autonomous decisions and actions.
Agents differ from earlier layers by taking action, as systems that combine models, tools, permissions, and instructions, not just models. They can query systems, execute code, modify records, and send communications, often chaining multiple steps without human intervention. The risk shifts from incorrect answers to incorrect actions executed rapidly across systems.
Core controls:
• Agent registry. Records which agents are live, who owns each agent, the authority each agent holds, and which data and systems each agent may access. An agent not in the registry should not be running.
• Human oversight. Scaled to risk, two common models are Human-in-the-Loop (HITL), where a person approves actions before execution, and Human-on-the-Loop (HOTL), where a person supervises and can intervene or stop the agent. Governance should set verification thresholds so that high-risk actions, such as executing transactional code or altering databases, require approval, while lower-risk activities are monitored.
• Observability. Maintain complete logs of what each agent did, why, and with what data, so the enterprise can reconstruct and audit actions.
• Failover and shutdown process. If an agent must be stopped, the supported business process must continue via manual fallback, rollback procedures, or a degraded mode. A “kill switch” is effective only if it has been tested and the process remains operational.
Typical failures include unconstrained agent behavior and processes that fail when agents are unavailable. The latter is often overlooked; organizations that automate processes and lose manual capabilities trade resilience for efficiency.
Board and Risk/Audit focus: Boards should ask about agent observability, guardrails, “kill switches,” and process resilience. Internal audit should verify that these controls exist and have been tested.
(To be continued.)
(Thanks to Anton Bonifacio, Pam Cabudoy, Pete Maniego, and Dan Natindim for their valuable comments.)
Gil B. Genio is governor and secretary of the Management Association of the Philippines or MAP. He is a retired banker and Globe and Ayala executive, and a member of the Analytics and AI Association of the Philippines and the Institute of Corporate Directors. He serves as an independent director at GT Capital Holdings, Puregold Price Club, and Megawide Construction.