Resource Library · November 2026

AI Vendor Evaluation Scorecard

Fifty-two criteria, six must-pass gates, and a weighted total. Build is an option. Evidence first; a demo is not a score.

25 pages40 min readIntermediateCompanion workbook included

By AsymiLink Meta · Published · Updated

Request the guide and workbook

Summary

Fifty-two criteria, six must-pass gates, and a weighted total. Build is an option. Evidence first; a demo is not a score. Frameworks excerpted on this page: Six must-pass gates (V19–V24); How the card is meant to be used. A 25 pages November 2026 document, with a companion workbook.

  • Fifty-two criteria, in six dimensions, weighted to 100 percent. Six must-pass gates sit in front of the total. A failed gate is “Do not proceed,” unless a named executive accepts residual risk in writing.
  • Evidence first. A demo, a logo or a brochure sentence is not evidence of production quality.
  • Score a Build option and a reasoned “do nothing” on the same card, on the same three-year basis.
  • We are vendor-neutral: a recommendation can be a tool you already own, a new product, a custom build, or no build at all.

A crawlable plain-text copy of this page is also at /resources/ai-vendor-evaluation-template.txt. The canonical URL is https://asymilinkmeta.com/resources/ai-vendor-evaluation-template.

Written for Procurement, security, operations and the sponsors of an AI purchase or a build.

What's in this scorecard

This 25-page scorecard is a way to choose that is not a reaction to a demo. Most AI purchases are decided in a room where a product has just been shown at its best. The questions that should have come first — what the job is, whether your data will be used to train a model, whether you can leave — are asked later, or not at all. This scorecard is the work that should happen before, during and after that room.

It is fifty-two criteria, in six dimensions, with a default weight for each, summing to 100 percent; and six must-pass gates that a total is not allowed to outvote. Evidence first: a demo, a logo or a brochure sentence is not evidence of production quality. Build is an option, scored on the same card, on the same three-year basis, alongside a reasoned “do nothing.” Changing a weight after a demo needs a reason, written down.

It is written for procurement, security, operations, and the sponsors of a purchase or a build. It assumes a defined job, not a shopping trip. We are vendor-neutral: a recommendation can be a tool you already own, a new product, a custom build, or no build at all, and we explain why. A companion workbook holds the 52 criteria, the weights, the gates, the questions and an illustrative evaluation of four hypothetical options. Yellow cells are inputs. Dimension and overall scores, and a gate result, are formulas. A completed workbook is a recommendation tool, not a certificate.

Gates first, then the total

Apply the six gates first. They live in the security and data-handling dimension (V19–V28): training use, agreed in the contract or forbidden; encryption at rest and in transit, on the path the data actually takes; identity — single sign-on or equivalent, least privilege, no shared vendor logins, vendor access named and revocable; an audit trail you can export; export and deletion of prompts, outputs, embeddings, logs, configurations and fine-tunes, in documented formats; and residency and subprocessors — a list, a jurisdiction, a notice of change, and a path if a location is not acceptable.

A failed gate is “Do not proceed,” unless a named executive accepts residual risk in writing, with the residual risk named and a date to look again. It is not a note in the margin. A pattern of accepted fails is a different risk appetite, and should be described as such, not as a successful evaluation.

The remaining dimensions cover strategic fit, capability and evidence, integration depth, support and operations, and total cost, commercial terms and exit — including a three-year view, lock-in named, and Build on the same card. How to read the result is a recommendation, not a ranking: the number serves the recommendation. The LLM security paper and the offline guide in this library are companions for the path and the hosting decision. This scorecard will not stand in for them.

Six must-pass gates (V19–V24)

These are the ones a total is not allowed to outvote. A vendor default is not an agreement.

  1. V19

    Training use. Agreed in the contract, or forbidden. A vendor default is not an agreement.

  2. V20

    Encryption. At rest and in transit, on the path the data actually takes, including logs, backups and subprocessors.

  3. V21

    Identity. Single sign-on or equivalent; least privilege; no shared vendor logins; vendor access named and revocable.

  4. V22

    Audit trail. Access and automated or model actions, in a form you can export.

  5. V23

    Export and deletion. Prompts, outputs, embeddings, logs, configurations and fine-tunes, in documented formats, with a deletion path.

  6. V24

    Residency and subprocessors. A list, a jurisdiction, a notice of change, and a path if a location is not acceptable.

If a gate is failed and someone still wants to proceed, that is an executive decision, in writing. It is not a courtesy.

How the card is meant to be used

Fifty-two criteria, weighted 100 percent. Build is on the same card. Evidence first.

  1. A defined job

    If the job is not frozen, later scores are a reaction to a presentation. Strategic fit asks whether the product, the build or the choice to leave things as they are is judged against a named outcome.

  2. Evidence, not a demo

    Every score has evidence. A demo, a logo or a brochure sentence is not evidence of production quality.

  3. Build, and do nothing

    Score a Build option and a reasoned “do nothing” on the same card, on the same three-year basis. A purchase that cannot be compared to building, or to stopping, is not finished.

Get the guide and workbook

Enter your name, work email and company. The files appear on this page immediately. A backup download link is also emailed when email delivery is available.

By requesting this resource you agree that AsymiLink Meta may contact you about it. See our Privacy Policy and Terms of Use.

Frequently asked questions

Can a high total outvote a failed gate?

No. The six must-pass gates sit in front of the total. A failed gate is “Do not proceed,” unless a named executive accepts residual risk in writing, with the residual risk named and a date to look again.

Is a demo enough evidence to score a criterion?

No. A demo, a logo or a brochure sentence is not evidence of production quality. Every score has evidence.

Should a custom build be on the same card?

Yes. Score a Build option and a reasoned “do nothing” on the same card, on the same three-year basis. We are vendor-neutral: a recommendation can be a tool you already own, a new product, a custom build, or no build at all.

What is in the companion workbook?

The 52 criteria, the weights, the gates, the questions, and an illustrative evaluation of four hypothetical options. Yellow cells are inputs. Dimension and overall scores, and a gate result, are formulas.