So, you just rolled out Claude Enterprise to the business, and in effect, handed every employee and every department an on-demand development team.

No interviews. No vetting. No SDLC. No visibility.

That’s the reality of AI-powered Citizen Development.  And, it’s moving faster than most CIOs, CTOs, and Business Operations leaders realize.  Once an employee is exposed to vibe coding, leveraging MCP servers, packaging Skills, and building agents in Cowork, their ability to proliferate new applications, automation, and decisioning tools is unleashed and often paired with unbridled excitement to create.

I’ve been working with a global insurance company, evaluating and identifying AI opportunities in their Claims Operations process.  As one of the more conservative, regulated, and risk-aware industries, they have been slower to roll-out AI.  And even there, the same uncomfortable questions keep surfacing.  Questions that your organization is probably facing too.

On the plus side, AI tools like Copilot, Cowork, and embedded agents are genuinely powerful. They put prototyping in the hands of domain experts. Compress timelines. And, embed real business knowledge directly into the build process.  That’s the upside of AI and it is VERY REAL.

But here’s what doesn’t come in the box:

Version control. Prompts, skills, and agents are being built and changed with zero audit trail. No ownership. No rollback.  When an adjuster is vibe coding a new consolidated report skill and changes the data transformation logic in the PY script, there is no way to recover an older version if the new one breaks.

Data governance. Sensitive data is flowing into AI tools without anyone asking: if this complies with our privacy policy or regulatory requirements.  The FNOL Service Rep is using AI to compile policy and loss data and generate a claim summary.  Is the LLM being used secure (not public)?  Are these uploads protected and excluded from future LLM training?

Hallucination awareness. Your business users aren’t thinking about non-determinism. They’re trusting the output and psyched to be on the innovation edge.  You users need better awareness and enablement to  building controls and tests into their prompts, scripts, architecture, etc. to detect and/or minimize hallucinations.

A lifecycle. Most of this development is happening directly in production. There’s no dev environment. No QA. No staged rollout. No change controls. No governance.  How are changes and issues on shared services, enterprise skills, etc. being communicated to the broader enterprise?

In addition to these risks, there are real costs associated with enabling citizen developers that need to be considered in order to derive the business value and protect the enterprise:

• Training people not just on AI tools, but on basic development practices and version control (GitHub, anyone?)
• Defining and enabling an AI SLDC that balances innovation demand with risk management
• Employing metrics that measure token consumption, connector usage, and cloud compute and storage costs to prevent runaway spend that inverts ROI capture.
• Thinking about Day 2 and associated break-fix costs when something goes wrong because it will

A great example happened just as I was kicking off the engagement, an early pilot team was using a Cowork and the MCP Filesystem server to organize a network share.  The instructions in the prompt had conflicting (and untested) checks that caused files to be misclassified and accidentally optimized to the trash can (aka. deleted from the network drive).  This was being built and prototyped in production.  Fortunately, the helpdesk and data recovery team were able to restore the drive.  Remember, you are liable for what AI does or doesn’t do.

The SDLC wasn’t bureaucracy; it was risk management. New AI tools are quietly circumventing that process.  Not out of malice but out of speed and excitement.  The answer isn’t to shut it down and stifle adoption and innovation. The answer is to build a governance model and process that matches the pace of adoption.  I believe that balancing innovation with security and risk management is not a constraint on growth, rather it’s the condition for sustainable growth.

If you’re a CIO, CTO, or Operations leader trying to figure out where your AI guardrails need to be, let’s compare notes.

Drop a comment or send me a DM.

#AIGovernance #EnterpriseAI #ProductStrategy #DigitalTransformation #RiskManagement