Skip to main content

MastraScorer

MastraScorer 类是 Mastra 中所有评分器的基类。它提供了一个标准的 .run() 方法用于评估输入/输出对，并支持通过 preprocess → analyze → generateScore → generateReason 的执行流程进行多步骤评分工作流。

🌐 The MastraScorer class is the base class for all scorers in Mastra. It provides a standard .run() method for evaluating input/output pairs and supports multi-step scoring workflows with preprocess → analyze → generateScore → generateReason execution flow.

注意： 大多数用户应使用 createScorer 来创建评分器实例。不建议直接实例化 MastraScorer。

如何获取 MastraScorer 实例
Direct link to 如何获取 MastraScorer 实例

🌐 How to Get a MastraScorer Instance

使用 createScorer 工厂函数，它会返回一个 MastraScorer 实例：

🌐 Use the createScorer factory function, which returns a MastraScorer instance:

import { createScorer } from "@mastra/core/evals";

const scorer = createScorer({
  name: "My Custom Scorer",
  description: "Evaluates responses based on custom criteria",
}).generateScore(({ run, results }) => {
  // scoring logic
  return 0.85;
});

// scorer is now a MastraScorer instance

.run() 方法
Direct link to .run() 方法

🌐 .run() Method

.run() 方法是执行评分器并评估输入/输出对的主要方式。它通过你定义的步骤处理数据（预处理 → 分析 → 生成分数 → 生成理由），并返回包含分数、理由和中间结果的完整结果对象。

🌐 The .run() method is the primary way to execute your scorer and evaluate input/output pairs. It processes the data through your defined steps (preprocess → analyze → generateScore → generateReason) and returns a comprehensive result object with the score, reasoning, and intermediate results.

const result = await scorer.run({
  input: "What is machine learning?",
  output: "Machine learning is a subset of artificial intelligence...",
  runId: "optional-run-id",
  requestContext: {
    /* optional context */
  },
});

.run() 输入
Direct link to .run() 输入

🌐 .run() Input

input:

any
Input data to be evaluated. Can be any type depending on your scorer's requirements.

output:

any
Output data to be evaluated. Can be any type depending on your scorer's requirements.

runId:

string
Optional unique identifier for this scoring run.

requestContext:

any
Optional request context from the agent or workflow step being evaluated.

groundTruth:

any
Optional expected or reference output for comparison during scoring. Automatically passed when using runEvals.

.run() 返回
Direct link to .run() 返回

🌐 .run() Returns

runId:

string
The unique identifier for this scoring run.

score:

number
Numerical score computed by the generateScore step.

reason:

string
Explanation for the score, if generateReason step was defined (optional).

preprocessStepResult:

any
Result of the preprocess step, if defined (optional).

analyzeStepResult:

any
Result of the analyze step, if defined (optional).

preprocessPrompt:

string
Preprocess prompt, if defined (optional).

analyzePrompt:

string
Analyze prompt, if defined (optional).

generateScorePrompt:

string
Generate score prompt, if defined (optional).

generateReasonPrompt:

string
Generate reason prompt, if defined (optional).

步骤执行流程
Direct link to 步骤执行流程

🌐 Step Execution Flow

当你调用 .run() 时，MastraScorer 会按以下顺序执行定义的步骤：

🌐 When you call .run(), the MastraScorer executes the defined steps in this order:

  1. 预处理（可选）- 提取或转换数据
  2. 分析（可选）- 处理输入/输出和预处理数据
  3. generateScore（必填）- 计算数值得分
  4. generateReason（可选）- 提供评分的原因

每个步骤都会接收前一步的结果，从而让你构建复杂的评估流程。

🌐 Each step receives the results from previous steps, allowing you to build complex evaluation pipelines.

使用示例
Direct link to 使用示例

🌐 Usage Example

const scorer = createScorer({
  name: "Quality Scorer",
  description: "Evaluates response quality",
})
  .preprocess(({ run }) => {
    // Extract key information
    return { wordCount: run.output.split(" ").length };
  })
  .analyze(({ run, results }) => {
    // Analyze the response
    const hasSubstance = results.preprocessStepResult.wordCount > 10;
    return { hasSubstance };
  })
  .generateScore(({ results }) => {
    // Calculate score
    return results.analyzeStepResult.hasSubstance ? 1.0 : 0.0;
  })
  .generateReason(({ score, results }) => {
    // Explain the score
    const wordCount = results.preprocessStepResult.wordCount;
    return `Score: ${score}. Response has ${wordCount} words.`;
  });

// Use the scorer
const result = await scorer.run({
  input: "What is machine learning?",
  output: "Machine learning is a subset of artificial intelligence...",
});

console.log(result.score); // 1.0
console.log(result.reason); // "Score: 1.0. Response has 12 words."

整合
Direct link to 整合

🌐 Integration

MastraScorer 实例可以用于代理和工作流步骤

🌐 MastraScorer instances can be used for agents and workflow steps

有关定义自定义评分逻辑的详细信息，请参见 createScorer 参考。

🌐 See the createScorer reference for detailed information on defining custom scoring logic.

目录