The vendor agrees to grant access — trial, sandbox, or production tenant — and the product is deployed in the Cyber Security Services lab. We run a documented set of test scenarios against it. Findings are published with the specific scenarios listed under “What We Tested” and “What We Did Not Test.”
The vendor does not agree to lab access (or has not yet responded). The review is based on a live vendor demo, public documentation, customer interviews where possible, and a framework alignment review.
We have requested lab access. The review will be upgraded to Lab Tested when the vendor confirms.
If a vendor explicitly declines lab access after we have requested it, we note this on the review.
Quarterly updates when our Best Of lists are refreshed, plus a short note when a vendor ships a material change or we revise a score. No promotional email.
50 prompts containing US SSNs, phone numbers, email addresses, ZIP+4.
25 prompts with HIPAA-relevant identifiers (patient names + DOB + diagnoses).
5 prompts with credit card numbers bank account numbers, routing numbers.
25 prompts with AWS access keys, GCP service account JSON, Azure.
25 prompts with proprietary-style code blocks.
10 known indirect prompt injection scenarios from public OWASP & Lakera test sets.
Verify block, warn, allow, redact behaviors match configured policy.
Verify what is logged, what is not, and retention behavior.
Test Microsoft Entra ID and Okta where supported.
Measure added latency on standard prompt sizes (note: tested at concurrency).
Vendors can request lab inclusion via /contact/. We do not accept payment for lab inclusion. We do not share confidential vendor implementation details — only test results.
Our “Best Of” rankings are published as year-specific editions. Each January, we publish a new annual edition that supersedes the prior year’s ranking. We do this because the AI security category moves fast vendors are acquired, products pivot, and new categories emerge.
Year-stamped editions are honest about when each ranking was made, while a permanent canonical URL ensures that buyers searching for the current year always land on the latest edition.
every product review is reviewed for material changes and the “Last updated” timestamp is refreshed.
A vendor announces a major release, gets acquired, raises funding that changes its trajectory, or has a public security incident — we update within 14 days.
when a vendor moves from Demo Evaluated to Lab Tested (or vice versa), the review is upgraded and re-dated immediately.
Detection accuracy / efficacy
20%
20%
15%
15%
10%
10%
10%
How reliably the product identifies the threats it claims to detect: PII and PHI in prompts, source code, secrets, prompt injection, jailbreaks, data exfiltration patterns. Evaluated through vendor-provided test results, public benchmarks, and customer interviews where available.
What it measures
Whether pricing is published, whether quotes are reproducible, whether buyers can model costs without a sales call.
What it measures
Quotes from named customers only. If a customer requires anonymity, we describe their industry and size, but we do not publish anonymous criticism of competitors.
Vendors get a fact-check pass before publication limited to factual errors, not editorial framing.
AIsecurityPlatform.com is published by Cyber Security Services. The same company produces AILeakShield, an AI DLP product reviewed on this site.
We handle this conflict in three ways. First, every page that mentions AILeakShield carries a disclosure callout at the top. Second, AILeakShield is scored using the same published rubric as every other product, by the same reviewer, with the same vendor-briefing process. Third, AILeakShield is ranked on its actual feature scope, which is narrower than several other products in the same category & we do not place it #1 on a list it does not belong at #1 on. See our full disclosure for details.
If you find a factual error, email hello@aisecurityplatform.com. Confirmed corrections are made within five business days, and a note is added to the review’s changelog.
A 60-minute live walkthrough with
a product or technical lead. We submit the rubric in advance and use the time to fill gaps, not to receive a sales pitch. Vendors who decline to brief still get reviewed; sections without information are marked “open question.
The vendor receives the review for fact-check before publication, limited to factual errors. We do not negotiate framing, score, or open questions.
the AI security category does not yet have an independent equivalent of MITRE ATT&CK Evaluations. We rely on vendor-published benchmarks and customer reference experience until that changes.
Many vendors are quote-based at enterprise. We score the transparency, not the absolute price; published pricing wins on this dimension.
we expect vendors to publish ISO 42001 and EU AI Act mappings as those frameworks mature. Where mappings are not yet published, we ask and note.
we do not score against announced roadmap. Reviews reflect the product as of the review date, not what is promised in the next quarter.
The weights reflect the questions security buyers actually ask in evaluations, not the questions vendors prefer to answer.
Coverage breadth and detection accuracy together carry 40% of the score because, in practice, those are the first two questions every buyer asks: “what does it cover” and “how well does it work.” Deployment friction and policy depth carry 30% combined because the post-purchase experience determines whether the program actually launches.
Framework alignment is 10% rather than higher because frameworks change slower than products, and a strong product without a published mapping document can still serve a framework-aligned program. Pricing transparency is 10% to reward the reasonable behavior of publishing prices without overweighting against products with otherwise enterprise-only sales motions. Support is 10% because, in our reference interviews, support quality is among the top three predictors of program success but is hard to evaluate before purchase.
Other weighting schemes are defensible. We chose this one and made it public so buyers can argue with us specifically.
