How to Run a GEO Platform Trial

A useful GEO platform trial follows one complete work cycle: configure representative questions, collect answers, inspect the evidence, create a justified improvement, review the result and test the handoff your team will actually use. Give shortlisted vendors the same requirements and preserve failures and unknowns alongside successful demonstrations.

The trial should establish workflow fit and evidence quality. A short trial cannot prove that the tool causes more citations, represents all buyer conversations or will improve revenue. Keep those longer-term outcome questions separate from the immediate buying decision.

This protocol is a reusable evaluation template. The example case set and thresholds are editorial choices for an illustrative team, not industry standards or a report of vendor performance.

Step 1: Write the decision and gates

Use a short charter that names the work, owner, budget constraint and evidence required. For example:

Decision: Can our two-person growth team use this platform to find
and prepare one accurate, actionable improvement each week?

Mandatory gates:
- Inspect raw answers and exact attached source URLs.
- Understand metrics, collection scope and failed requests.
- Export the evidence needed for an editorial brief.
- Review proposed changes before publication.

Preferred:
- Preserve our topic labels in reporting.
- Reduce manual copying into our existing workflow.

Not evaluated in this trial:
- Population market share.
- Guaranteed future citation growth.
- Coding-agent installation success unless separately tested.

Set the gates before the demos. Otherwise an impressive feature can distract from an unresolved requirement. The platform evaluation checklist provides a broader menu; choose the subset your team actually needs.

Name the person who can accept each gate. The buyer, analyst, editor and technical reviewer may be the same person in a small company, but the responsibilities should remain explicit.

Step 2: Prepare a fixed case set

Use a small but varied set you can review completely. For an illustrative developer-tool business, twelve questions could cover the following jobs:

Include questions your buyers genuinely ask. A deliberately impossible riddle may reveal something interesting about a model while saying little about your procurement decision. Keep the exact wording stable across candidates and version the inventory.

Separate questions that name your product from unbranded ones. If one vendor suggests alternatives, preserve them as an additional exploratory set instead of silently changing the common set. Record locale and relevant settings with each case.

Step 3: Create a small truth and ambiguity pack

Prepare current primary evidence for claims you can verify: plan scope, deployment options, supported versions, authentication requirements or a documented limitation. Link the exact source and note its checked date.

Also include ambiguous cases for the reviewer. A negative but accurate description should not be labeled false. A mention without a recommendation should not automatically pass as positive selection. An owned article can be cited without naming its publisher.

Do not feed the expected answer into every collection prompt. That changes the task from observing an independent answer to asking a system to repeat supplied facts. Keep the fact pack available to reviewers and to any explicitly separate correction workflow.

The truth pack is not an excuse to mark every difference from company messaging incorrect. Distinguish factual assertions from opinions and preserve unresolved cases. A fair trial can reveal that your own documentation is incomplete.

Step 4: Record the actual collection setup

For each candidate, capture the package, supported surfaces used, run frequency, locale, question count and collection period. Note whether the data comes from an interface, an API or another disclosed method. Use model/version identifiers only where actually exposed.

Do not assume two products collect the same thing because their dashboards display the same provider logo. If the methods differ, the trial may still compare workflow usefulness, but it cannot treat the answer differences as a controlled vendor accuracy contest.

Track attempts and failures. Keep timeouts, empty results and not-assessable responses visible. Ask how the platform retries and whether the record preserves those attempts. A tool can look unusually favorable if missing or difficult results disappear from the denominator.

Use a setup record:

Candidate and package:
Trial dates:
Question-set version:
Surfaces and disclosed collection methods:
Language/locale:
Cadence and planned repetitions:
Expected observations:
Successful collections:
Failed or not-assessable collections:
Changes to setup during trial:

Anthropic's evaluation guide distinguishes a task from a trial and recommends preserving the evidence needed to grade outcomes. Apply that discipline without assuming its agent-specific examples define a universal GEO test size.

Step 5: Review evidence before looking at scores

Select a common sample of observations from each candidate, including at least one surprising result and one difficult factual case. Inspect the raw answer and sources before reading the dashboard's summary label where feasible.

Ask the reviewer to label mention, recommendation, source citation and factual accuracy independently. Then compare those labels with the platform's definitions and results. A disagreement may reflect an error, a different rubric or evidence the reviewer missed. Record the reason instead of treating every mismatch as a defect.

Peec's documentation distinguishes visibility, position and sentiment. This illustrates why similar-looking charts can describe different things. Require each candidate to explain its own calculation and how it handles cases outside a simple positive mention.

Where practical, remove vendor branding from exported review records. This reduces the influence of dashboard design on the judgment, although it does not make the trial perfectly blind. Preserve the original records and mapping privately so findings remain traceable.

Step 6: Inspect one citation all the way through

Choose a material claim and open the exact attached URL. Determine whether the source supports the claim, contradicts it, does not address it or cannot be assessed. Preserve redirects and source dates where relevant.

Check whether the platform makes this process easy. Can the analyst move from the headline finding to the answer and source without searching manually? Can the evidence travel into a brief? Does the system present inferred influence as if it were proven provenance?

Do not demand that a vendor expose private model internals it does not have. Demand accurate representation of the evidence it does expose. The goal is an honest, useful investigation path.

Use the source-tracing procedure for a claim-level review. This step prevents an attractive aggregate report from hiding a source that fails to support the recommendation.

Step 7: Complete one real work item

Select one actionable finding and create the resulting brief, page change or documentation improvement. Keep the task bounded enough that the responsible reviewer can finish it during the trial.

A strong work item identifies the reader, missing or incorrect information, supporting source, intended change and acceptance test. It should not simply instruct the writer to add keywords or publish another generic article.

Tools can support different amounts of this work. Profound's current product description includes proposed and drafted marketing actions alongside answer insights. The trial should establish how a specific action moves through your actual review path, rather than assuming all products stop at reporting or all automation produces a usable result.

Include one constraint that requires judgment: a planned feature must remain described as planned, a benchmark must retain its environment, or an API example must use the correct permission scope. Record whether the output preserves the constraint and how much correction is needed.

Step 8: Test rejection and revision

Do not test only the happy path where every draft is accepted. Ask the reviewer to reject a specific claim or request a factual revision. Then observe how the correction returns to the writer or system and whether it survives the next draft.

For a publishing integration, use a safe draft destination or an explicitly authorized test page. Verify who can approve and publish, and confirm that a rejection does not accidentally trigger publication. If the normal workflow is manual export, test that export and the effort required to preserve tables, sources and metadata.

The artifact does not need to go live to evaluate this handoff. A ready-to-publish draft can establish much of the workflow. Do not create unnecessary external changes merely to make the trial appear more complete.

Record manual work as part of the result. Copying evidence, rebuilding formatting, chasing permissions and correcting source notes are real operating costs even if the software itself ran quickly.

Step 9: Score gates and effort separately

Use evidence states rather than an unexplained overall number:

Keep an effort log beside the gates. Record configuration, investigation, drafting, review and handoff time. If two candidates both pass mandatory gates, the effort difference can inform the decision. If one fails a mandatory requirement, a lower apparent cost does not erase the failure.

Avoid assigning a score to an unavailable feature demonstration as if its absence were proven. Mark it unresolved and state what evidence would settle it. Written confirmation may resolve a package question; an actual task may still be necessary to resolve workflow behavior.

An illustrative trial result

Suppose two fictional candidates both return usable answers for most of the fixed cases. Candidate A provides complete source exports but requires the editor to publish manually. Candidate B stages drafts in the CMS, but its export omits the answer text your analyst needs to preserve.

Your charter says portable answer evidence is mandatory and direct publishing is preferred. Candidate A can pass the mandatory gates while carrying a measurable manual handoff cost. Candidate B remains unresolved until it supplies a suitable export or an acceptable supported alternative.

Now suppose the reviewer finds that Candidate A's draft misstates a version requirement. The useful result is not an immediate declaration that the product is inaccurate overall. Test the revision path, record the review time and determine whether the corrected artifact meets the gate. A recurring failure under a realistic task may justify rejection; one corrected mistake still needs transparent reporting.

This example is a decision exercise, not a disguised review of real vendors. Its purpose is to show how requirements, evidence and effort interact without reducing them to a decorative score.

Step 10: Write the buying decision

Conclude with one of three outcomes: proceed, decline or gather specific missing evidence. Include the package and setup evaluated, passed gates, failures, unresolved limitations, expected internal work and next review date.

If proceeding, preserve the question-set version and baseline records so onboarding does not erase what the trial established. Confirm final commercial terms separately from the trial environment; a demonstrated feature may belong to a different package than the quote.

If declining, identify the material mismatch rather than criticizing unrelated features. If more evidence is required, make the request concrete: an actual export, a role-permission demonstration, a corrected artifact or clarification of collection scope.

The operating-model comparison helps translate the findings into team fit. For subsequent visibility outcomes, use a separate measurement baseline rather than claiming the procurement exercise proved citation growth.

How long should the trial run?

Long enough to complete the relevant workflow and inspect variation in the chosen cases within the available trial terms. A week can reveal workflow problems, but it may be insufficient for broader outcome inference. State the limitation rather than extending conclusions beyond the evidence.

Should we let each vendor choose the prompts?

Keep a common set for comparison. Vendor suggestions can form a separate exploratory group and may improve your inventory, but should not replace the shared cases mid-trial without a recorded change.

Do we need to publish content during the trial?

Not necessarily. A reviewed draft and tested handoff may answer the buying question. Publishing introduces additional authorization and outcome variables. Only include it when it is necessary, safely scoped and explicitly authorized.

Sources and review date

Sources checked September 17, 2026. The trial protocol and fictional scenarios are Jam's editorial method, not findings about the named products.

• Anthropic evaluation guidance, published January 9, 2026: tasks, trials, graders and outcome evidence.
• Peec documentation: an example of platform-specific metric definitions.
• Profound product description: current example of action workflows accompanying answer insights.

Continue reading

• GEO Platform Evaluation Checklist for Technical Teams
• Choose an AI Visibility Tool for Your Team
• How to Measure AI Visibility

Explore GEO with Jam

See how Jam approaches AI visibility research and content improvements for developer-tool teams.

Explore Jam for GEO