评估与评分员

🌐 Evals & Scorers

评估 API 已在新的评分系统上整合，采用了更新的命名规范和配置要求。

🌐 The evaluation API has been consolidated on the new scorers system with updated naming conventions and configuration requirements.

已更改

🌐 Changed

getScorers 到 listScorers

🌐 getScorers to listScorers

getScorers() 方法已重命名为 listScorers()。此更改与整个 API 中的命名约定保持一致，其中复数的 getter 方法使用 list 前缀。

🌐 The getScorers() method has been renamed to listScorers(). This change aligns with the naming convention used across the API where plural getter methods use the list prefix.

要迁移，请将所有对 getScorers() 的调用替换为 listScorers()。

🌐 To migrate, replace all calls to getScorers() with listScorers().

- const scorers = mastra.getScorers();
+ const scorers = mastra.listScorers();

Codemod

你可以使用 Mastra 的 codemod CLI 来自动更新你的代码：

🌐 You can use Mastra's codemod CLI to update your code automatically:

npx @mastra/codemod@latest v1/mastra-plural-apis .

runExperiment 到 runEvals

🌐 runExperiment to runEvals

runExperiment() 函数已重命名为 runEvals()。此更改提供了更清晰的命名，更好地描述了评估功能。

🌐 The runExperiment() function has been renamed to runEvals(). This change provides clearer naming that better describes the evaluation functionality.

要迁移，请将函数调用从 runExperiment 更新为 runEvals。

🌐 To migrate, update function calls from runExperiment to runEvals.

- import { createScorer, runExperiment } from '@mastra/core/evals';
+ import { createScorer, runEvals } from '@mastra/core/evals';
  import { myAgent } from './agents/my-agent';

  const scorer = createScorer({
    id: 'helpfulness-scorer',
    // ...
  });

- const result = await runExperiment({ target: myAgent, scorers: [scorer], data: inputs });
+ const result = await runEvals({ target: myAgent, scorers: [scorer], data: inputs });

Codemod

你可以使用 Mastra 的 codemod CLI 来自动更新你的代码：

🌐 You can use Mastra's codemod CLI to update your code automatically:

npx @mastra/codemod@latest v1/evals-run-experiment .

getScorerByName 到 getScorerById

🌐 getScorerByName to getScorerById

getScorerByName() 方法已重命名为 getScorerById()。评分器现在需要 id 字段，而不是 name。此更改与使用 id 进行实体标识的更广泛 API 模式保持一致。

🌐 The getScorerByName() method has been renamed to getScorerById(). Scorers now require an id field instead of name. This change aligns with the broader API pattern of using id for entity identification.

要进行迁移，请更新方法调用和评分器配置，以使用 id 替代 name。

🌐 To migrate, update method calls and scorer configuration to use id instead of name.

  const scorer = createScorer({
-   name: 'helpfulness-scorer',
+   id: 'helpfulness-scorer',
    // ...
  });
  
- const scorer = mastra.getScorerByName('helpfulness-scorer');
+ const scorer = mastra.getScorerById('helpfulness-scorer');

Codemod

你可以使用 Mastra 的 codemod CLI 来自动更新你的代码：

🌐 You can use Mastra's codemod CLI to update your code automatically:

npx @mastra/codemod@latest v1/evals-scorer-by-name .

评分器配置从 name 更改为 id

🌐 Scorer configuration from name to id

评分者现在需要 id 字段而不是 name。name 字段现在是可选的。这一更改与其他 Mastra 实体保持了一致性。

🌐 Scorers now require an id field instead of name. The name field is now optional. This change provides consistency with other Mastra entities.

要迁移，请更新评分器定义以使用 id 作为必填字段。

🌐 To migrate, update scorer definitions to use id as the required field.

  const scorer = createScorer({
-   name: 'helpfulness-scorer',
+   id: 'helpfulness-scorer',
+   name: 'Helpfulness Scorer', // optional
    // ...
  });

存储分数 API 到 listScoresBy* 模式

🌐 Storage score APIs to listScoresBy* pattern

分数存储 API 已被重新命名，以遵循 listScoresBy* 模式。此更改与更广泛的存储 API 命名规范一致。

🌐 Score storage APIs have been renamed to follow the listScoresBy* pattern. This change aligns with the broader storage API naming conventions.

要迁移，请更新评分查询方法以使用新的命名模式。

🌐 To migrate, update score query methods to use the new naming pattern.

- const scores = await storage.getScores({ scorerName: 'helpfulness-scorer' });
+ const scores = await storage.listScoresByScorerId({
+   scorerId: 'helpfulness-scorer',
+ });

// Also available:
// - listScoresByRunId
// - listScoresByEntityId  
// - listScoresBySpan

预构建评分器导入到 scorers/prebuilt 路径

🌐 Prebuilt scorer imports to scorers/prebuilt path

预构建评分器的导入已合并到单个 @mastra/evals/scorers/prebuilt 路径下，而不再使用单独的 scorers/llm 和 scorers/code 路径。此更改简化了导入操作，并提供了更清晰的预构建评分器组织方式。

🌐 Prebuilt scorer imports have been consolidated under a single @mastra/evals/scorers/prebuilt path instead of separate scorers/llm and scorers/code paths. This change simplifies imports and provides a clearer organization of prebuilt scorers.

要迁移，请更新导入语句以使用新的 scorers/prebuilt 路径。

🌐 To migrate, update import statements to use the new scorers/prebuilt path.

  // LLM-based scorers
- import { createHallucinationScorer } from '@mastra/evals/scorers/llm';
- import { createFaithfulnessScorer } from '@mastra/evals/scorers/llm';
+ import { createHallucinationScorer } from '@mastra/evals/scorers/prebuilt';
+ import { createFaithfulnessScorer } from '@mastra/evals/scorers/prebuilt';

  // Code-based scorers
- import { createContentSimilarityScorer } from '@mastra/evals/scorers/code';
- import { createCompletenessScorer } from '@mastra/evals/scorers/code';
+ import { createContentSimilarityScorer } from '@mastra/evals/scorers/prebuilt';
+ import { createCompletenessScorer } from '@mastra/evals/scorers/prebuilt';

Codemod

你可以使用 Mastra 的 codemod CLI 来自动更新你的代码：

🌐 You can use Mastra's codemod CLI to update your code automatically:

npx @mastra/codemod@latest v1/evals-prebuilt-imports .

UIMessage 到 MastraDBMessage 的得分器消息类型

🌐 Scorer message types from UIMessage to MastraDBMessage

评分器的输入和输出类型现在使用 MastraDBMessage[] 而不是 UIMessage。此更改使评分器与数据库持久化的消息格式保持一致，从而在整个框架中实现一致性。

🌐 Scorer input and output types now use MastraDBMessage[] instead of UIMessage. This change aligns scorers with the database-persisted message format for consistency across the framework.

要进行迁移，请更新评分器实现以使用 MastraDBMessage 类型，并通过嵌套的 content 结构访问消息内容。

🌐 To migrate, update scorer implementations to use MastraDBMessage types and access message content through the nested content structure.

  import type {
    ScorerRunInputForAgent,
    ScorerRunOutputForAgent
  } from '@mastra/core/evals';

- // ScorerRunInputForAgent uses UIMessage[]
- const inputMessages: UIMessage[] = run.input.inputMessages;
+ // ScorerRunInputForAgent now uses MastraDBMessage[]
+ import type { MastraDBMessage } from '@mastra/core/agent';
+ const inputMessages: MastraDBMessage[] = run.input.inputMessages;

将消息内容结构转换为嵌套格式

🌐 Message content structure to nested format

工具调用和文本内容现在通过嵌套的 content 对象结构访问，而不是作为消息的平面属性。这一变化提供了更好的类型安全性，并与数据库消息格式保持一致。

🌐 Tool invocations and text content are now accessed through a nested content object structure instead of being flat properties on the message. This change provides better type safety and aligns with the database message format.

要进行迁移，请通过 message.content.toolInvocations 访问工具调用，通过 message.content.content 访问文本，或使用 getTextContentFromMastraDBMessage() 辅助工具。

🌐 To migrate, access tool invocations via message.content.toolInvocations and text via message.content.content or use the getTextContentFromMastraDBMessage() helper.

+ import { getTextContentFromMastraDBMessage } from '@mastra/evals';
+
  const run = await scorer.run(testRun);

  // Accessing text content
- const text = message.content;
+ const text = getTextContentFromMastraDBMessage(message);
+ // or directly: message.content.content

  // Accessing tool invocations
- const toolCalls = message.toolInvocations;
+ const toolCalls = message.content.toolInvocations;

已移除

🌐 Removed

遗留评估代码

🌐 Legacy evals code

@mastra/core 中已移除旧版评估代码。这包括旧版评估指标、评分/评审模块以及基于钩子的自动评估代码。此更改通过移除过时的评估方式简化了代码库。

🌐 Legacy evals code has been removed from @mastra/core. This includes legacy evaluation metrics, scorer/judge modules, and hook-based automatic evaluation code. This change simplifies the codebase by removing outdated evaluation approaches.

要进行迁移，请使用 @mastra/core/evals 或 @mastra/evals 中新的 evals/scorers API。

🌐 To migrate, use the new evals/scorers API in @mastra/core/evals or @mastra/evals.

- // Legacy evals APIs
+ import { createScorer, runEvals } from '@mastra/core/evals';
+ 
+ const scorer = createScorer({
+   id: 'my-scorer',
+   // Use new scorer API
+ });

代理 TMetrics 泛型参数

🌐 Agent TMetrics generic parameter

TMetrics 泛型参数已从 AgentConfig 和 Agent 构造函数中移除。度量/评分器现在使用评分器 API 配置，而不再是 Agent 类型系统的一部分。此更改简化了 Agent 的类型签名。

🌐 The TMetrics generic parameter has been removed from AgentConfig and the Agent constructor. Metrics/scorers are now configured using the scorers API instead of being part of the Agent type system. This change simplifies the Agent type signature.

要进行迁移，请移除 TMetrics 泛型参数，并使用评分器 API 配置评分器。

🌐 To migrate, remove the TMetrics generic parameter and configure scorers using the scorers API.

- const agent = new Agent<AgentId, Tools, Metrics>({
+ const agent = new Agent<AgentId, Tools>({
    // ...
  });

与 Evals 相关的类型导出

🌐 Evals-related type exports

已删除多个与评估相关的类型导出，包括 DeprecatedOutputOptions、Metric 以及处理器选项类型。这些类型现在是内部的，或者已被新的评分器 API 取代。此更改减少了 API 的表面面积。

🌐 Several evals-related type exports have been removed including DeprecatedOutputOptions, Metric, and processor option types. These types are now internal or have been replaced by the new scorers API. This change reduces API surface area.

要进行迁移，请删除对这些已移除类型的引用，并使用新的评分器 API。

🌐 To migrate, remove references to these removed types and use the new scorers API.

- import type {
-   DeprecatedOutputOptions,
-   Metric,
-   LanguageDetectorOptions,
-   ModerationOptions,
- } from '@mastra/core';
+ // Use new scorers API types
+ import type { Scorer } from '@mastra/core/evals';

createUIMessage 测试助手

🌐 createUIMessage test helper

createUIMessage() 测试辅助已被移除，取而代之的是 createTestMessage()。新的辅助工具可以创建带有嵌套内容结构的 MastraDBMessage 对象，并支持可选的工具调用。此更改使测试工具与新的消息格式保持一致。

🌐 The createUIMessage() test helper has been removed and replaced with createTestMessage(). The new helper creates MastraDBMessage objects with the nested content structure and supports optional tool invocations. This change aligns test utilities with the new message format.

要进行迁移，将 createUIMessage() 调用替换为 createTestMessage()，并更新为使用 MastraDBMessage 类型。

🌐 To migrate, replace createUIMessage() calls with createTestMessage() and update to use MastraDBMessage types.

- import { createUIMessage } from '@mastra/evals';
+ import { createTestMessage } from '@mastra/evals';

  // Creating test messages
- const message = createUIMessage({
-   id: 'test-1',
+ const message = createTestMessage({
+   id: 'test-1', // optional, defaults to 'test-message'
    role: 'user',
    content: 'Hello',
    toolInvocations: [], // optional
  });