Evaluate a prompt

task australia Updated Sep 10, 2026 3 min read evaluate-prompt.md
Raw file servicenow.com

Use the AI Skill Kit evaluation tools to measure how your skill prompts perform against a dataset, using automated metrics and human feedback.

Before you begin

Role required: sn_skill_builder.admin

Procedure

  1. Navigate to All > AI Skill Kit > Home.

  2. Select the skill that you want to evaluate.

  3. Select the Prompt performance tab.

  4. Select the Evaluation runs tab.

  5. Create a dataset from a table or data collection.

Method

Steps

Create a dataset from a table

  1. Give the dataset a name and description.
  2. Select Table.
  3. Find the table that you want to use.
  4. Select the maximum number of records that you want to use.
  5. Add conditions.
  6. Select Generate Preview.
  7. Select the mappings.
  8. Select Create.

Create a dataset from a data collection

  1. Give the dataset a name and description.
  2. Select Data Collection.
  3. Select a data collection that you created in AI Data Kit.
  4. Select Generate Preview.
  5. Select the mappings.
  6. Select Create.
6. Select the add icon \[Omitted image "icon-nask-add.png"\] Alt text: for **Evaluation Runs**.
  1. Give the evaluation run a name and description.

  2. Select one or more prompts that you want to evaluate.

  3. Select Save & Next.

  4. Select a dataset.

  5. Select Save & Next.

  6. Expand the Quality tab.

  7. Select the metrics that you want to evaluate.

    Evaluation method Metric Description
    Human Human Feedback Human evaluation is the default option available for all prompt executions that generate a response. You can rate the response with a thumbs up or thumbs down, based on your satisfaction. You also have the option to provide more detailed feedback to explain your evaluation choice.
    Automated Correctness The correctness metric assesses the generated response's accuracy, completeness, pertinence, and writing quality relative to the given instruction. This metric helps to check that the text accurately reflects the instruction, covers all important points, remains relevant, and is well written.
    Automated Correctness with Golden Response The correctness with golden response metric uses a predefined reference to assess the generated response's accuracy, completeness, pertinence, and writing quality relative to the given instruction. This metric helps to check that the text accurately reflects the instruction, covers all important points, remains relevant, and is well written. Use this metric when a reference response is available.
    Automated Faithfulness The faithfulness metric assesses whether a generated response accurately reflects the information and context provided in the given instruction. This metric helps to check that the text contains no hallucinations, fabricated facts, or unsupported conclusions, maintaining alignment with the source material.
  8. Select Save & Next.

  9. Review the evaluation choices that you made.

  10. Select Save & Evaluate.

  11. Give a human evaluation.

    1. Select Human evaluation.

    2. Select a record to use in the evaluation.

    3. Expand the prompt and read the result.

    4. Select the thumbs up or thumbs down icon [Omitted image "icon-nask-thumbs.png"] Alt text: to give your evaluation.

    5. Add more information and select Submit.

Parent Topic:Using AI Skill Kit

Related topics

Create a skill

Create a prompt

Use prompt assistance

Test a prompt

Finalize and publish a skill

Activate a skill

Call a custom skill from a script

Search documentation
Search document titles and file paths on the australia branch