A transparency score is not a capability score

The shot

In December 2025, researchers centered at Stanford University's Center for Research on Foundation Models published 'The 2025 Foundation Model Transparency Index', the third annual edition of the Foundation Model Transparency Index (FMTI) following earlier October 2023 and May 2024 editions. The paper lists co-authors from Stanford, UC Berkeley, Princeton University and MIT, and the index's own landing page reports scoring 13 developers — including Google, OpenAI, Anthropic, Meta, IBM, Alibaba, DeepSeek and xAI — on 100 disclosure indicators.

What the documents show

The paper states its purpose directly: the FMTI 'is a measurement instrument specifically designed to measure the transparency of AI companies,' distinct from what it calls the more common practice of 'benchmarking the capabilities and risks of AI as a technology.' A company's score therefore describes how much it discloses, not how capable or safe its models are. The 2025 edition reports average transparency fell to 40.69 out of 100, down from 58 in 2024; IBM scored highest in the index's history at 95, while xAI and Midjourney tied lowest at 14. The authors disclose their own method's limits: for six companies that did not submit a self-reported report — Alibaba, Anthropic, DeepSeek, Midjourney, Mistral and xAI — the team gathered information manually, aided by an AI agent the paper says 'still fall[s] short of completely replacing' human review.

The workflow

The index's own five-step method — designing indicators, selecting companies, gathering information, scoring practices, and engaging companies to improve future disclosure — describes a research and advocacy process, not a product certification. A score depends heavily on cooperation: participation fell from 74% of contacted companies submitting reports in 2024 to 30% in 2025, and companies preparing their own reports score considerably higher than those that do not. The authors also revised the indicators substantially for 2025, so scores across editions are not a simple like-for-like trend line.

What the tool does not change

This is a boundary the authors draw themselves: the paper separates its transparency measurement from evaluating 'the capabilities and risks of AI as a technology,' so scores should not be read as a safety or quality ranking. Nor does an open-weight release guarantee transparency by this measure — DeepSeek, Meta and Alibaba release open models yet score among the more opaque developers on training data and deployment disclosures, while IBM and AI21 Labs combine open releases with high scores.

• Does a cited FMTI score reflect disclosure about training data, compute and deployment impact, or is it being read as a safety or accuracy measure?
• Was the score self-reported by the company, or gathered manually because the company declined to participate that year?
• Which edition's indicator set produced the score, given the authors' own statement that 2025's indicators changed substantially from 2023 and 2024?

The FMTI measures what a company says about itself, not what its models can do. Reading a score without that distinction risks treating a disclosure index as a capability leaderboard, which its own authors say it is not.