The Method

How do we define and identify responsible AI enabling companies?


There are a lot of companies out there that profit on the buzzword "responsible." We make sure they don't make it into EAIDB.

Verification Procedure

Prerequisites

Founded post-2015 Series C or earlier Responsible enabling Active
We prioritize younger companies. As a whole, the RAI industry really began post-2015, and most companies present in the market prior to this year had primary business lines outside of RAI. Startups that are too mature typically become too diversified and begin to lose a little bit of their initial meaning. We still track them and include them in reports, but it's difficult to compare a late-stage startup to early ones. The main business line of the company must be easily traceable to one of our categories. If it doesn't fit, it's usually out of scope. We regularly check for "outward RAI activity" as a method of verification. If a startup preaches RAI principles on their corporate blog or social media or if they conduct regular RAI reesarch, it's usually a sign that they prioritize and care about RAI.

Secondary Verification

As a next step, we try to grab time on the founders' calendar and discuss the specifics and technical details (and sometimes get a demo). This is how startups become "directly verified" on EAIDB. We've directly verified about 45% of the full database. We're a small team and we're constantly working to increase this number!

Ecosystem Categories & Evolution

As the market evolved, we noticed that Alternative ML was not getting the same attention as we had expected from mid-2023. Causal AI was still very slow to grow, neurosymbolic was barely understood, etc. We decided to wrap providers of these technologies into Model Builders as they were offering these unique models and built platforms around them for distribution. We also cut open-source (for reasons listed below) and added the Privacy Preservation category. We re-organized Data for AI to place a greater emphasis on sourcing, annotation, and licensing while moving synthetic data, federated learning, and differential privacy over to this new privacy category. This allowed us to look at all Privacy Enhancing Technologies (or PETs) in one category and perform likewise comparisons. The other categories stayed the same.

  1. AI Security: startups offering testing and mitigation solutions for attacks on machine learnign and AI systems.
  2. AI Governance, Risk & Compliance: startups offering comprehensive visibility across models in an enterprise from a risk and compliance perspective.
  3. Data Operations: startups offering data governance, sourcing, annotation, and/or debiasing.
  4. Model Operations: startups offering developer tools for monitoring, testing, and downside mitigation or organizations focusing on alignment or fine-tuning.
  5. Model and Platform Builders: any vertical or horizontal implementations of models or platforms that are responsible alternatives to existing solutions.
  6. Consulting: organizations offering advisory or legal services around responsible AI implementation.

In contrast to older categorizations, we noticed that (again) open-source was too hard to track. We were, however, seeing a good amount of differentiation between companies in some of the blurrier categories (ex. a lot of MLOps companies also provided some level of AI GRC, but by separating these we were able to distill each to their real essence).

In order to provide better differentiation between the categories in the database, we expanded to eight categories. The new entrants, AI GRC and AI Security, were direct reactions to a market that was becoming more and more proactive instead of reactive. Solutions around AI controls were moving into the pre-production realm. Other categories like Alternative ML were plays on some of the newer kinds of machine learning entering the market (causal, neurosymbolic).

  1. AI Security: startups offering testing and mitigation solutions for attacks on machine learnign and AI systems.
  2. AI Governance, Risk & Compliance: startups offering comprehensive visibility across models in an enterprise from a risk and compliance perspective.
  3. Alternative ML: solutions offering access and democratization to different kinds of learning. Federated learning for better privacy or neurosymbolic and causal for better explainability are a few examples.
  4. Data for AI: startups offering synthetic data, preprocessing, and/or debiasing services.
  5. MLOps and ModelOps: startups offering monitoring and mitigation or testing tools for machine learning downsides.
  6. Model and Platform Builders: any vertical or horizontal implementations of models or platforms that are responsible alternatives to existing solutions.
  7. Consulting: organizations offering advisory or legal services around responsible AI implementation.
  8. Open-Source: companies or libraries offering comprehensive bias testing or monitoring capabilities.

In contrast to older categorizations, we noticed that (again) open-source was too hard to track. We were, however, seeing a good amount of differentiation between companies in some of the blurrier categories (ex. a lot of MLOps companies also provided some level of AI GRC, but by separating these we were able to distill each to their real essence).

When EAIDB first started, we had five different categories for startups in the RAI space.

  1. Data for AI: startups offering synthetic data, preprocessing, and/or debiasing services.
  2. MLOps and Monitoring: startups offering bias monitoring and mitigation or testing tools for machine learning downsides.
  3. Targeted Solutions & Horizontal Technology: any vertical or horizontal implementations of the above categories.
  4. Consulting: organizations offering advisory or legal services around responsible AI implementation.
  5. Open-Source: companies or libraries offering comprehensive bias testing or monitoring capabilities.

While these categories functioned well for their time, they were entirely too simplistic and we discovered far too much overlap between them. Open source offerings were also too hard to keep track of when they were not attached to a for-profit entity (i.e., when they were created and maintained by academic institutions, individuals, departments of Fortune 500 companies, etc.). We pivoted when we realized these simply were not descriptive of the market.

© 2022-2024 EAIDB