DeWitt Clause vs the public cloud

The DeWitt Clause is the name of a clause in the terms of service that prevents a customer from disclosing performance of a licensed product. Things turned out OK for David DeWitt and he even went on to work for at least one company that now uses a DeWitt Clause for their database products.

I am not a lawyer but I have a strong opinion on the DeWitt Clause and non-compete agreements. I think both are anti-competitive and hope more is done to limit their usage.

One of the arguments for DeWitt Clauses is that benchmarking is hard, others will get it wrong and that can mislead customers. While the first parts are correct (it is hard, people will get it wrong) the last clause feels like a bad faith argument to me. Vendors haven't always pursued truth while running their benchmarks. I agree there will be plenty of benchmarketing in a world without DeWitt Clauses. I think the benefits outweigh the problems.

Database vendors

We used to think of this as a thing for legacy enterprise database vendors and many of the new vendors don't use it -- not MongoDB Atlas as of 24-Jun-2021, not CockroachDB as of 19-Oct-2021, not TimescaleDB as of 24-Sep-2020 and 5-Mar-2019, not Altinity as of 1-Jul-2021.

But other new vendors use it and this became news after Databricks published a great result for 100TB TPC-DS and a less than great result for SnowflakeDB. They then changed their license to the DeWitt Embrace Clause which is like copyleft for the DeWitt Clause. This means that if you disclose benchmark results for them, they are allowed to do the same for you. I hope the DeWitt Embrace Clause is embraced by more vendors who use a DeWitt Clause.

For the record, Snowflake quickly replied to the Databricks result. Frankly, I prefer to see a benchmarketing battle between Snowflake and Oracle. But it is awesome that the Snowflake response explains that anyone can repeat their result by signing up, spending about $300 and clicking a few buttons. Cloud DBMS is fun. The Snowflake ToS (as of 1-Nov-2021) don't have a DeWitt Clause.

Public cloud vendors

Switching from managed database providers to public cloud vendors the DeWitt Clause usage is interesting. From what I was able to find, Microsoft has the worst terms followed by Google:

• AWS (as of 28-Oct-2021) allows benchmark disclosure via the DeWitt Embrace Clause
  
• Microsoft was harder to find docs on. In this undated agreement if you offer a service on Azure that competes with them, then they are allowed to benchmark and disclose the results of your service. This is far more aggressive than any DeWitt Clause variant. But I didn't find any other mention of benchmarks (to allow or prevent them for Azure).
• Oracle (a PDF dated 22-Jun-2020) has a DeWitt Clause and requires pre-approval for disclosure. They also don't allow performing a benchmark without pre-approval.
  
• Google (as of 18-Oct-2021) has a DeWitt Clause and requires pre-approval for disclosure. They  bans competitors from running benchmarks even when not disclosed. They added the DeWitt Clause on 6-Oct-2020.
  

Public cloud benchmarks

Examples of useful public cloud benchmarks that will be prevented in the future by DeWitt Clauses. I will add to this list as I look for them. For now there was just one from long ago that triggered this post:

• Public cloud object storage perf by Zach Bjornson

More on Microsoft

This is the part in the Microsoft agreement that seems aggressive:

If Customer offers a service competitive to an Online Service, by using the Online Service, Customer agrees to waive any restrictions on competitive use and benchmark testing in the terms governing its competitive service. If Customer does not intend to waive such restrictions in its terms of use, Customer is not allowed to use the Online Service.