Artificial intelligence is changing what organizations can do faster than it is changing how well they are prepared to do it. Two problems that security and data leaders have carried for decades — data quality and the security of the systems that hold it — are now the limiting factors on everything AI promises. This piece, first published in 2024 and updated as the technology has moved, sets out where I believe the real work lies: in data discipline, in securing the application layer, and in building teams that can work across the old boundaries.
Why data quality decides what AI can do
“Garbage in, garbage out” has never mattered more. A model trained, tuned or prompted on incomplete, duplicated or wrong data produces confident, wrong output at scale. Before any organization can benefit from AI it has to answer questions it has usually avoided: What data do we have? Where is it? Who owns it? Is it labelled and described well enough that a system — or a colleague — can use it without asking?
Answering them forces a second, healthier conversation about privacy. Why do we have this data at all? Is it necessary? The “collect everything” era is ending, partly because regulators demand it and partly because hoarded data is a liability: it costs money to store, it widens the impact of a breach, and it poisons the models that touch it. Collecting only what is needed is both the ethical position and the practical one.
Why AI starts with a question, not a dataset
“AI data” means nothing without context. As with research, the work begins with a question or a business case; only then can you say which data is needed, where it lives and how it must be prepared. That journey is multidisciplinary by nature — data scientists, architects, engineers, analysts and security officers have to work as one team — and organizations that skip the question and start with the tooling produce expensive demos rather than durable capability.
What new attack surface AI creates
Models are executed by software, and software can be exploited. AI adds new forms to an old threat:
- Prompt injection — instructions hidden in content the model reads (a web page, a document, an email) that hijack what it does next.
- Data poisoning — corrupting training or retrieval data so the model learns or repeats what the attacker wants.
- Jailbreaking — bypassing a model’s safety and policy constraints to extract information or actions it should refuse.
- Supply-chain risk — third-party models, plugins and agents with access to internal systems, adopted faster than they are reviewed.
The defensive techniques — heuristics, anomaly detection, least privilege, input validation — are not new. What is new is how they must be applied to systems that take natural-language input and act on it autonomously. An AI agent with access to production tools is, from a security standpoint, a privileged user who can be socially engineered by a document.
Where the real security risk sits
For all the novelty, the greatest risk to AI systems is the same one that has plagued IT for decades: the application layer. Layer 7 — HTTP, DNS, SSH, the APIs and interfaces through which AI systems are reached — is where exposed storage, unauthenticated endpoints and over-privileged service accounts live. An AI platform with a public management interface and a default credential is no more sophisticated a target than an open database, and it is attacked the same way.
Securing that layer is inherently multidisciplinary. It requires tearing down the silos between infrastructure, application, identity and data teams; assessing information resources collectively; removing redundancy; and integrating systems so there is one view of what exists and who can reach it. The organizations that do this well are the ones whose security and IT already share a single source of truth for assets and identities.
What skills the next security team needs
The future security professional blends data science, security engineering, identity and access management, data loss prevention, threat and vulnerability management and operations. No single person will master all of them, which is why cross-training becomes a necessity rather than a perk. DevOps and later DevSecOps were not buzzwords; they were the recognition that development, security and operations had to understand each other’s work to move at the required pace. AI raises the stakes again.
The leadership task is to keep these teams motivated, informed and accountable while they learn: pair them, have them shadow each other, and measure shared outcomes rather than departmental ones.
What leaders should do now
- Inventory your data — what, where, who owns it, how sensitive — and delete what has no purpose.
- Start every AI initiative from a business question, with security in the room from the first meeting.
- Treat AI agents as privileged identities: least privilege, monitored access, revocable credentials.
- Fix the basics at Layer 7 — exposed interfaces, weak authentication, unpatched components — before worrying about exotic model attacks.
- Cross-train deliberately. Budget time for security engineers to learn data, and data teams to learn security.
Where BOSSS fits
The AI revolution will bring consequences we cannot yet foresee, and the way through is collaboration, continuous learning and disciplined attention to the systems that hold the data. That conviction shaped how we built BOSSS AI Agents: specialized agents that act across GRC, IAM, CMDB and XDR on one unified data model, so the data they reason over is consistent and every action they take is recorded and governed by the same identity and audit controls as a human user. Our companion article on why AI needs unified security data goes deeper into that design.