Companies have been buying cybersecurity tools for some time, but now they face another expense - privacy tools. Earlier this month, as we wiggled through the crowded exhibit hall at the International Association for Privacy Professionals’ annual Privacy.Security.Risk conference, we heard from privacy pros that they have difficulty sorting through all the come-ons for unfamiliar products enhanced with AI.

Companies’ data governance councils will struggle to find independent tests to reassure them before spending tens of thousands of dollars on cyber or privacy products. “Sadly, there’s little public information that benchmarks tools against each other because of a longstanding censorship regime [enacted by] DeWitt clauses” in tool contracts, David A. Wheeler, The Linux Foundation’s director of open-source supply chain security, told the Cybersecurity Law Report.

Named for the computer science professor who upset Oracle by middling test results for its software, a tool license’s DeWitt clause bars publishing any competitive performance results without the product company’s permission. These terms dissuade researchers, even NIST, from assessing cyber and privacy tools in a way that helps users. The lack of independent benchmarking undermines the collective defense and protection of personal data, Wheeler argued.

Chainguard is one cyber company breaking ranks, telling NIST recently that DeWitt clauses stifle knowledge of which products work, and which are snake oil. Another vendor, Databricks, lamented in a blog post that “this practice is bad for customers and bad for innovation, and it’s time for it to go. That’s why we are removing the DeWitt clause from our service terms, and calling upon the rest of the industry to follow.” Tech giants are shifting, too: Amazon Web Services and Azure products now permit tit-for-tat statistical comparisons – not independent, but at least some testing.

Do you think your company has enough good information to know which tools to buy for your cyber and privacy programs? Unsure that your tool is finding the sensitive data or stopping the bot attacks as promised? We’d like your help taking a deeper look at this crucial operational decision for data protection and liability reduction in the automated age – tell us what you found during your tool shopping, and any other factors that make this spending feel dicey. If you find independent testing of these tools, we’d be happy to share that information with readers.  

Hope this didn’t scare you too much – Happy Halloween!