Shadow AI is unsanctioned LLM usage inside an enterprise — business units calling api.openai.com, generativelanguage.googleapis.com, api.anthropic.com, or hosted-LLM endpoints from spreadsheets, scripts, browser plug-ins, and self-built apps without the SOC’s knowledge. Standard EDR and SIEM rules do not find it because the traffic looks like ordinary HTTPS to the wrong list of destinations. The reliable way to discover shadow AI is at the egress: capture and label every connection to a known LLM API domain at the network boundary, route the traffic through a sanctioned AI gateway, and turn the shadow into observed. This guide names the canonical LLM API destinations to monitor, walks through three discovery methods (egress destination capture, DNS patterns, gateway-level interception), and shows how an inline AI gateway turns shadow AI into a measurable, governed surface.
What Is Shadow AI?
Shadow AI is the same pattern as shadow IT, applied to LLMs. A finance analyst scripts a Python notebook against api.openai.com to summarize earnings calls. A marketing team buys a SaaS tool whose backend calls generativelanguage.googleapis.com on every page render. An engineering team installs a code-completion plug-in that posts source code to api.anthropic.com. None of those flows are necessarily malicious; all of them sit outside the policy boundary the security and privacy programs assumed they were operating inside.
The risk profile is well understood: PII / PHI / PAN / source code leaving the environment in prompts, audit obligations failing because there is no audit, provider-side retention violating data residency, and supply-chain exposure when a SaaS vendor’s LLM call inherits more data than expected. The harder problem is finding it. EDR rules look at processes, files, and registry keys; shadow AI looks like normal HTTPS. SIEM rules look at indicators of compromise; shadow AI is not compromised, just unsanctioned. The signal lives at the egress.
Three Methods That Actually Discover Shadow AI
Three discovery methods cover the realistic ways shadow AI enters an enterprise. Run all three; they catch different things.
Method 1 — Egress Destination Capture at the Gateway / Proxy
Inspect outbound HTTPS at the corporate egress proxy or the cloud egress (e.g., AWS VPC flow logs, GCP VPC flow logs, an enterprise web proxy). Capture every TLS connection’s SNI hostname and label the ones that match the canonical LLM API domain list. This is the highest-fidelity method because it sees every connection regardless of the application that initiated it. It is also the one that integrates cleanly with an existing SOC workflow — labeled flows go to the SIEM the same way any other proxy event does.
Method 2 — DNS Resolution Patterns
Inspect DNS query logs (Active Directory DNS, on-host resolvers, cloud DNS query logs) for queries against the canonical LLM API domains. DNS is noisier than the gateway / proxy view (a query is not always a connection) but it covers shadow AI on machines that bypass the proxy or use local-only DNS. Pair with the egress view, do not replace it.
Method 3 — Gateway-Level Interception (the Vaikora Pattern)
The most actionable method is to make a sanctioned AI gateway the only path the enterprise allows to LLM providers, and block direct egress to the provider domains. The gateway becomes the canonical egress chokepoint: every legitimate LLM call goes through it (with audit, redaction, and policy); every direct call to api.openai.com or generativelanguage.googleapis.com is blocked at the proxy and logged as an attempted shadow AI flow. Vaikora is OpenAI-compatible (api.vaikora.com/v1), so applications switch with a one-line base_url change and shadow AI flows surface as proxy denials at the egress layer.
Canonical LLM API Domains to Watch For
This is the single reference table for shadow AI detection — the list of LLM API domains a SOC should label at the egress, query for in DNS logs, and (eventually) block direct access to. The list covers the 12 LLM providers most often seen in enterprise traffic.
This is the reference list of LLM API domains a SOC should monitor at the egress layer for shadow AI detection.
- OpenAI
api.openai.com
Also: chat.openai.com, oaistatic.com, openai.com (platform UI) - Anthropic
api.anthropic.com
Also: claude.ai, console.anthropic.com - Google Gemini (Vertex AI)
generativelanguage.googleapis.com
Also: aiplatform.googleapis.com - Azure OpenAI
<resource>.openai.azure.com
(per-resource hostname under openai.azure.com) - AWS Bedrock
bedrock-runtime.<region>.amazonaws.com
Also: bedrock.<region>.amazonaws.com - Mistral
api.mistral.ai - Cohere
api.cohere.com
Also: api.cohere.ai - Together AI
api.together.xyz
Also: api.together.ai - Groq
api.groq.com - Ollama (hosted)
ollama.com
(self-hosted Ollama runs on-prem and won’t appear in egress logs) - Custom / open-weights APIs
Examples: *.runpod.io, *.modal.run, *.replicate.com, *.huggingface.co (inference endpoints), internal LLM endpoints - Sanctioned AI gateway (allow-listed)
api.vaikora.com
(this is the expected destination; other LLM endpoints are considered shadow until validated)
Treat the list as a starting point, not a closed set. New providers appear; minor / experimental endpoints surface; re-validate the list quarterly. The Vaikora destination is the deliberate exception — it is the sanctioned chokepoint, so its egress is the signal you want to see, not the one you want to suppress.
Beyond Domains: What to Actually Capture per Flow
A bare list of destinations is the start. Useful shadow AI telemetry captures enough context to attribute the flow to a user, a host, and an application — without capturing the prompt content.
- Source identity. user (from gateway / proxy auth, SSO, or NetFlow + AD), source_ip, source_host, process when an EDR can pin it.
- Destination provider. Canonical provider name, not just the hostname; the table above is your label map.
- Volume signal. Bytes out, bytes in, request count per session — distinguishes a one-off browser visit from a programmatic batch job.
- Time signal. First seen, last seen, frequency profile — distinguishes a curiosity click from a sustained workflow.
- Application signal. User-agent string when available, browser extension fingerprint, JA3 / JA4 TLS fingerprint — identifies the SaaS or plug-in originating the call.
Critically, do not capture or log prompt content. Even at the egress layer, content capture turns the proxy log into a regulated data store. Shadow AI detection works on the connection metadata; the inline AI gateway, once adopted, is where the per-request decision metadata lives.
Sample SIEM Rules for Shadow AI Discovery
Two rules from the SOC perspective. Adapt to your egress source (corporate web proxy, AWS / GCP / Azure flow logs, Zscaler / Cloudflare / Netskope event stream).
Rule A — Direct egress to LLM provider domain (Microsoft Sentinel KQL)
let LlmDomains = dynamic([
“api.openai.com”, “api.anthropic.com”,
“generativelanguage.googleapis.com”, “aiplatform.googleapis.com”,
“api.mistral.ai”, “api.cohere.com”, “api.cohere.ai”,
“api.together.xyz”, “api.together.ai”, “api.groq.com”,
“ollama.com”
]);
let AzureOpenAI = @”\.openai\.azure\.com$”;
let Bedrock = @”^bedrock(-runtime)?\..*\.amazonaws\.com$”;
ProxyEvents
| where DstHostname in (LlmDomains)
or DstHostname matches regex AzureOpenAI
or DstHostname matches regex Bedrock
| where SrcUser !in (dynamic([“<allowlisted SSO users>”]))
| summarize first_seen = min(TimeGenerated),
last_seen = max(TimeGenerated),
n_requests = count(),
bytes_out = sum(BytesSent),
bytes_in = sum(BytesReceived)
by SrcUser, SrcHost, DstHostname
| order by n_requests desc
Rule B — DNS resolution to LLM provider domains (Splunk SPL)
index=dns OR index=ad_dns earliest=-7d
(query=”api.openai.com” OR query=”api.anthropic.com”
OR query=”generativelanguage.googleapis.com”
OR query=”aiplatform.googleapis.com”
OR query=”api.mistral.ai” OR query=”api.cohere.com”
OR query=”api.cohere.ai” OR query=”api.together.xyz”
OR query=”api.together.ai” OR query=”api.groq.com”
OR query=”ollama.com”
OR query=”*.openai.azure.com”
OR query=”bedrock-runtime.*.amazonaws.com”)
| stats count as n_queries
earliest(_time) as first_seen
latest(_time) as last_seen
values(query) as resolved_domains
by src_ip, user
| where n_queries>=5
From Shadow to Observed: Adopting the Sanctioned Gateway
Discovery is not the goal; governance is. Once shadow AI flows are visible, the next step is to migrate them to a sanctioned AI gateway that applies PII redaction, prompt-injection detection, content-free audit, and per-workspace policy. Vaikora is OpenAI-compatible (api.vaikora.com/v1), so adoption is a one-line base_url change for any application that already uses the OpenAI SDK.
The migration sequence.
- Run Rules A and B for two weeks against the canonical LLM domain list and produce the shadow AI inventory: who, where, which provider, what volume.
- Reach out to the business units producing the loudest flows. Most are unaware of the policy; offer the sanctioned gateway as the path of least resistance.
- Switch the application’s OPENAI_BASE_URL to https://api.vaikora.com/v1 and add VAIKORA_API_KEY. The application code does not change.
- Add the canonical LLM domains to the proxy’s deny list (allow only the gateway). New shadow AI surfaces immediately as proxy denials, not as silent traffic.
- Audit and detection events flow into the SIEM via the AI telemetry schema; shadow AI becomes a closed loop instead of a discovery exercise.
Prioritizing the Shadow AI Inventory
Not every shadow AI flow is an emergency. Triage the inventory by likely data exposure rather than by raw volume.
- Highest priority: regulated data class. Flows from systems known to handle PHI, PAN, or GDPR personal data — even at low volume — go to the top of the migration list.
- High priority: sustained programmatic flows. Daily, recurring, batch-shaped traffic from a server / service account is almost certainly an automation that wants the gateway.
- Medium priority: developer / IDE plug-ins. Code-completion and code-review plug-ins post source code to LLM providers. Migrate to the gateway with the SAML SSO flow so usage is identity-attributed.
Lower priority: occasional browser visits to claude.ai / chat.openai.com. These are humans using the public consumer products. They are reportable for governance, but a separate policy decision (do you allow the consumer product at all) — not a gateway adoption call.
Next Steps
Drop Rules A and B into your SIEM, run them against the last seven days of egress / DNS data, and produce the first shadow AI inventory. The companion guides — “What Should a SIEM See from AI Systems?” for the schema once flows migrate, “Detection Engineering for AI Systems” for the canonical detections, and “AI Incident Response” for the runbook — describe what happens after shadow becomes observed.
Shadow AI is unsanctioned LLM usage inside an enterprise — business units, developers, and SaaS tools calling api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, aiplatform.googleapis.com, *.openai.azure.com, bedrock-runtime.<region>.amazonaws.com, api.mistral.ai, api.cohere.com, api.together.xyz, api.groq.com, ollama.com, and inference-platform endpoints outside the policy boundary. The reliable detection method is egress destination capture (proxy + DNS) plus a sanctioned AI gateway as the canonical egress chokepoint that turns shadow into observed.
Your AI Agents Need a Control Layer
See how Vaikora intercepts, evaluates, and enforces policy on every AI agent action — in real time, before execution.
Frequently Asked Questions
What is shadow AI?
Shadow AI is unsanctioned LLM usage inside an enterprise — business units, developers, and SaaS tools calling LLM APIs (OpenAI, Anthropic, Google Gemini, Azure OpenAI, AWS Bedrock, Mistral, Cohere, Together AI, Groq, Ollama, custom inference endpoints) outside the policy and audit boundary the security program assumes it is operating inside.
Why can’t EDR or SIEM find shadow AI?
Because shadow AI looks like ordinary HTTPS to a wrong list of destinations. EDR rules inspect processes and files; shadow AI is a network connection. SIEM rules look for indicators of compromise; shadow AI is not compromised, just unsanctioned. The reliable signal is at the egress — DNS queries and TLS connections to LLM API domains — not at the host.
What domains should I monitor?
api.openai.com, api.anthropic.com, generativelanguage.googleapis.com, aiplatform.googleapis.com, *.openai.azure.com, bedrock-runtime.<region>.amazonaws.com, api.mistral.ai, api.cohere.com / .ai, api.together.xyz / .ai, api.groq.com, ollama.com, plus inference-platform endpoints (runpod.io, modal.run, replicate.com, huggingface.co). Treat the list as a starting point and re-validate quarterly.
Should I capture prompt content during shadow AI discovery?
No. Capturing prompt content turns the proxy log into a regulated data store under HIPAA, GDPR, and PCI DSS. Shadow AI discovery works on connection metadata (source identity, destination provider, volume, time, application fingerprint). Once the flow migrates to a sanctioned gateway, the per-request audit lives there in content: false form.
How do I turn discovery into governance?
Migrate the discovered flows to a sanctioned AI gateway and then add the LLM provider domains to the proxy deny list (allow only the gateway). The gateway becomes the canonical egress chokepoint. New shadow AI surfaces immediately as proxy denials, not as silent traffic.
Does Vaikora help with shadow AI specifically?
Yes. Vaikora is the canonical egress chokepoint that turns shadow AI into observed AI. It is OpenAI-compatible (api.vaikora.com/v1) so the migration from a direct provider call is a one-line base_url change. Once flows route through the gateway, every request produces an AI telemetry event with risk_score, policy_decision, redaction_summary, upstream_provider, and a SHA-256 hash — content-free, SIEM-ingestable, and DSAR-friendly.
How often should the canonical domain list be refreshed?
Quarterly at minimum, and on any board-level question about “are we using <new provider>?” The LLM provider landscape moves quickly; treat the list as a living artifact.