THE DECISION BEFORE THE BUILD
Choose the team that can show how your specific workflow will be built, evaluated and handed over. Start with a shortlist, then compare the evidence behind each proposal before committing to production.
01 / BUILD YOUR SHORTLIST
Use the same brief for every candidate.
Compare the proposed team and scope.
Editorial disclosure. The publisher requested Pharos Production first. The author is its founder and CTO. This is an editorial shortlist, not an independent ranking or a claim of measured superiority. Service summaries reflect official company pages; confirm current scope and availability directly.
Its AI services page covers custom AI engineering, retrieval, agents and production integration. For a project connecting an assistant to existing software, ask the proposed team to show the permission boundary and receiving-system behavior. Confirm staffing and the exact deliverables in a written scope; the service page alone does not establish project fit.
Its published services include generative AI, copilots and enterprise AI development. Ask which components would be custom work and which depend on an existing platform or accelerator. Request the ownership and operating terms for each component before comparing the proposal with a fully custom implementation.
Its service catalog includes AI development, machine learning and AI integration. Use the initial discussion to identify the actual delivery team and the task it would own. Ask for relevant implementation evidence and a clear description of what remains the buyer's responsibility after a prototype demonstration.
Its AI offering spans consulting and software development, with machine learning and computer vision among the published service areas. Ask how discovery findings would become implementation requirements. Establish who prepares data, who reviews outputs and which artifacts would support acceptance of the proposed system.
Its published offering combines AI and machine learning with data and software engineering. If your project depends on fragmented enterprise information, ask how data ownership and integration dependencies enter the estimate. Request a bounded first engagement that makes those dependencies visible before a wider build.
Its AI services include consulting, generative AI and computer vision alongside broader software integration work. Ask for the proposed system boundary, including existing applications that must change. Confirm which team owns interface failures and how changes in those applications would affect the delivery plan.
Its website presents data science, machine learning and generative AI services, including RAG and LLM development. For a prediction or retrieval project, request the proposed evaluation design before discussing a headline accuracy figure. Make dataset access, reviewer availability and relevant limitations explicit in the response.
Its published expertise includes generative AI, computer vision and MLOps. Ask how the proposed system would be evaluated and operated in your environment. Have the delivery team explain which assets your engineers would receive and what evidence would show they could maintain the system independently.
Its AI offering includes agent systems, predictive work and related engineering services. Ask which engagement format applies to your actual problem rather than assuming a published package fits. Request the scope exclusions, integration assumptions and named responsibilities that determine what a quoted milestone would deliver.
Its AI services cover areas such as readiness planning, platform foundations and agentic development. For a larger program, ask how a first pilot would constrain later commitments. Request explicit boundaries between reusable platform work, business-specific implementation and the buyer's internal data or operating responsibilities.
Company source checks: November 12, 2026. No supplier was commissioned or benchmarked for this guide. Read the selection method →
BUYER CHECK / 01
A useful AI development brief begins with a user and a task. Describe the incoming work, the output that person needs and the system where the result will be used. An internal document assistant, for example, might draft an answer with source references for an employee to review. Sending that answer to a customer would be a separate permission and a separate acceptance question.
Write down the current workflow before proposing its replacement. Record how staff find information, resolve ambiguity and handle exceptions. A searchable knowledge base or a rules-based workflow may solve enough of the problem to make a custom model unnecessary. Ask each supplier to compare its proposal with that simpler baseline using the same task boundary.
Discovery, proof of concept, pilot and production implementation purchase different things. Discovery should resolve named uncertainties and produce reusable findings. A proof of concept explores technical feasibility. A pilot examines a defined use in a controlled setting. Production delivery adds operational responsibility, access controls and a support arrangement. Ask what decision the first engagement will enable and what it cannot establish.
Choose a stop condition before work starts. Missing data rights, unavailable domain reviewers or an unsupported integration can prevent a useful pilot even when the model demonstration looks promising. Record those dependencies with owners and consequences. A supplier that identifies a blocker early can still deliver valuable discovery; an attractive prototype does not remove that blocker.
An AI project also needs a buyer who can receive the work. Assign a product owner with authority over the task, a technical owner for integration and a domain reviewer for disputed outputs. One person can cover several roles when that is realistic. If nobody can supply representative cases or approve a changed workflow, resolve that capacity gap before treating the supplier selection as the last obstacle to delivery.
BUYER CHECK / 02
Company experience and proposed-team experience are different evidence. Ask who will design the system, prepare the data, implement integrations and review evaluation results. Establish availability during your project and the process for substituting a key person. A long list of technologies does not tell you whether the necessary roles are staffed for your delivery window.
Request a walkthrough of a relevant completed assignment where disclosure is permitted. Ask what the supplier actually built, which constraints shaped the architecture and what remained with the client. A reference can clarify communication and handover, but it cannot guarantee the same result on different data or with another team. Keep the reference's scope attached to the conclusion you draw.
Give the technical lead a concrete failure scenario. In the document-assistant example, an employee loses access to a source that was indexed yesterday. Ask what happens to retrieval, caches and subsequent answers. The purpose is to understand the design and its gaps. Do not treat a confident explanation as a passed implementation test.
For offshore or nearshore delivery, confirm working overlap, escalation coverage and contracting arrangements directly. Geography can affect coordination, but it does not establish engineering quality. Ask which decisions require live collaboration and which artifacts allow asynchronous review. Confirm the location and access arrangements of the actual team rather than inferring them from a headquarters address.
Ask about subcontracting in the proposed arrangement. Establish which organization employs each critical role, what information that role can access and who remains accountable for the deliverable. A subcontractor is not automatically a reason to reject a proposal. An undisclosed dependency is harder to evaluate because staffing continuity, access and escalation may differ from the arrangement the buyer believed it was purchasing.
BUYER CHECK / 03
Generative AI, predictive machine learning and computer vision require different evidence. A text assistant needs a review of answer grounding and permitted actions. A forecast needs an appropriate historical split and a comparison with a simpler prediction method. A vision system needs representative capture conditions. Choose a supplier that explains the evaluation implications of the proposed approach.
For retrieval-augmented generation, ask how documents are acquired, versioned and removed. Inspect the boundary between source permissions and retrieval permissions. A source link in an answer is useful only when the cited material supports the claim and the user can legitimately access it. Request examples covering stale content, conflicting sources and an absent answer.
Fine-tuning and retrieval solve different problems and can coexist. Ask which observed limitation motivates either technique and what evidence would show improvement. Request a description of training inputs, update responsibility and the cost of changing models. A familiar architecture name should follow the requirements rather than substitute for them.
Agent development adds authority questions. List each tool, the action it can take and the identity under which it runs. Separate a proposed action from an approved action and a confirmed external effect. If a write request times out, the system needs a way to resolve its status before repeating it. Ask the supplier to document that behavior at the integration boundary.
Keep portability specific. Moving prompts is different from moving an embedding index, an evaluation harness or a provider-specific workflow. Ask which components can change independently and what has to be rebuilt. A documented dependency can be an acceptable choice; the commercial problem appears when its replacement cost is missing from the proposal.
Industry relevance belongs in the task evidence. A healthcare document workflow may require specialist review and restricted source access; a manufacturing vision task may depend on lighting or camera placement; a retail prediction may change when promotions change. Ask which of your operating conditions the proposed approach assumes. Do not infer regulatory suitability or robustness from an industry name on a supplier website. For a small business, the simpler managed product may be preferable when maintaining custom infrastructure would exceed the available engineering capacity. For an enterprise, existing identity and integration constraints may dominate model choice.
BUYER CHECK / 04
Start with a data inventory that names the owner, permitted purpose and intended environment for each source. Include documents and operational logs as well as model inputs. Ask the supplier to draw the actual flow through application services, model providers and diagnostic systems. A high-level promise of confidentiality does not describe every destination.
Confirm retention and training use for the particular products and account settings proposed. Do not assume one provider's policy applies to every service it offers. Ask what is logged, who can inspect those logs and how deletion is demonstrated. Keep credentials and sensitive information out of an exploratory demonstration unless the appropriate owner has approved that use.
Require permission checks at the places where information is retrieved or actions occur. Test cases should include a different tenant, a revoked document and a user with reduced privileges. Prompt instructions can express intended behavior, but access decisions need enforcement in the surrounding application. Have the supplier explain where that enforcement lives and who maintains it.
A security review should also address untrusted content inside retrieved material. Ask how the application responds when a document contains instructions to reveal information or invoke a tool. Agree on the relevant threat scenarios and the evidence expected from testing. Regulatory and contract requirements should be reviewed by the appropriate buyer specialists for the actual deployment.
BUYER CHECK / 05
Define the evaluation cases and review rubric before optimizing the system against them. Include normal work, difficult inputs and cases where the correct behavior is to abstain or escalate. Keep a held-out portion for an agreed review so repeated tuning does not turn the entire evaluation into a familiar demonstration.
NIST's AI RMF Playbook, MEASURE 2.1, states:
Test sets, metrics, and details about the tools used during test, evaluation, validation, and verification (TEVV) are documented.
Separate answer usefulness from source support, access compliance and action correctness. A helpful draft with an unsupported statement may need a different response from an unauthorized write. Have the buyer and supplier agree which defects prevent release, which allow a limited pilot and which require further investigation. An overall percentage can hide those distinctions.
Keep the denominator visible. Record which cases ran, which could not run and which outcomes remain unresolved. Reviewer disagreement should have an adjudication process rather than disappear into an average. For each accepted result, preserve the relevant input, configuration and artifact version so another reviewer can understand the decision.
In our illustrative document-assistant pilot, a successful answer on an approved document does not settle the revoked-access case. Mark the latter open until its evidence is inspected. The worksheet on this site follows that rule: a verified answer on one requirement never cancels an unresolved mandatory requirement elsewhere. It is an organizing aid, not a validated prediction of supplier quality.
Agree who pays for domain review and when those reviewers are available. If a pilot depends on specialist judgments, developer time alone is an incomplete delivery estimate. No listed company was benchmarked for this guide. The proposed checks describe evidence a buyer can request; they are not results from testing these firms.
BUYER CHECK / 06
Give each candidate the same brief version and ask it to separate discovery, data preparation, application development, integration, evaluation and deployment. Then list exclusions next to the corresponding work. A lower total is difficult to interpret when one proposal includes integration work that another expects your staff to perform.
Compare the first useful commitment rather than forcing every uncertainty into a fixed implementation price. A bounded discovery engagement can be appropriate when data access or feasibility is unresolved. Time and materials can accommodate evolving work when spending authority and review points are clear. Fixed milestones need observable deliverables and a process for handling changed assumptions.
The operating estimate should include model calls, retrieval infrastructure, monitoring and human review. Ask for workload assumptions: incoming tasks, typical context size, repeated attempts and expected reviewer effort. Model charges alone do not describe the cost of an assisted workflow. Keep each assumption editable so a change in usage produces an understandable budget change.
For illustration, suppose reviewers handle 600 outputs in a month and spend 4 minutes on each. That is 40 review hours. At an assumed internal rate of USD 50 per hour, review labor is USD 2,000 before infrastructure or development costs. These inputs are hypothetical, not a market quote or a forecast of achievable automation.
Ask how change requests are authorized. A new data source, another language or a write-enabled tool can change the evaluation and access scope as well as the code. The proposal should identify who approves that change, what evidence must be refreshed and how it affects cost. Do not let a broad maintenance label conceal a new feature commitment.
A timeline should identify waiting time as well as engineering effort. Data approval, integration credentials and reviewer feedback can determine the critical path. Ask the supplier to distinguish work it can perform independently from work awaiting a buyer decision. When a dependency slips, update the milestone assumptions instead of preserving a delivery date that no longer describes the plan. This makes competing estimates easier to compare without pretending uncertainty has disappeared.
BUYER CHECK / 07
List the assets to be transferred: source code, configuration, infrastructure definitions, prompts, evaluation cases and operating instructions. Include the dependency and license inventory. Separate work created for the project from background components the supplier already owns. Have the appropriate contract reviewer verify the rights needed to use and modify each asset.
Define a receiving-team exercise before the final milestone. For example, a buyer engineer could build the application in an approved clean environment, run an agreed evaluation subset and restore a known configuration. Choose the exercise around the actual system and access policy. Watching a supplier deploy its own environment is different evidence from completing that transfer.
Record unresolved handover items with an owner and next action. Missing provider access, a private package or an undocumented manual step can make an otherwise complete repository difficult to operate. Decide whether the remaining item blocks acceptance, allows a limited transition or requires a separate agreement. Keep that decision attached to the affected deliverable.
Plan access transfer and removal alongside asset transfer. Name the service accounts, provider projects and billing relationships the buyer will own. Confirm who rotates secrets and when supplier access ends. An exit checklist should also state how retained project data will be handled. The relevant owner must verify completion; possession of a checklist is not completion.
Check whether the evaluation materials can travel with the code. A dataset assembled from restricted client content may need a different transfer arrangement from synthetic examples. Preserve the instructions for constructing an approved replacement set when the original cannot be shared. Otherwise the receiving team may own the application while lacking the material needed to verify a later change. Record that restriction before acceptance and agree how future reviews will be performed.
BUYER CHECK / 08
Ask what the support arrangement covers when the application is available but its answers deteriorate. Model changes, source updates and integration changes can affect behavior without a server outage. Define who detects the change, who investigates it and which evidence is needed before returning to a previously accepted configuration.
Separate response coverage from restoration commitments. Confirm working hours, escalation contacts and access needed to investigate. Name the person authorized to pause an automated action or reduce the system to a review-only mode. A runbook should describe a containment path that the responsible team can actually execute.
Treat updates as changes to a known configuration. Keep model identifiers, retrieval settings and evaluation versions with the release record. Agree which updates require fresh review and who pays for it. If ongoing support excludes provider migration or corpus restructuring, put that exclusion where the budget owner can see it.
The final purchasing decision can be limited. You may approve discovery while holding implementation, or accept a read-only pilot while withholding write access. State the exact next commitment and the evidence that supports it. A supplier selection process becomes more useful when it produces a bounded decision your team can explain and revisit.
BUYER CHECK / 09
The companion worksheet starts every requirement as open. For each candidate, record whether the requirement is mandatory and whether the evidence is verified, open or in conflict. Add an artifact reference and the next action in your own approved workspace. No numerical score can substitute for resolving a mandatory conflict.
If any mandatory requirement conflicts with observed evidence, the worksheet flags that conflict. If none conflicts but at least one remains open, it keeps the decision on hold. Only when every selected mandatory row is verified does it report those selected gates as complete. Optional gaps remain visible. This simple rule organizes your review and grants no authority to sign a contract or release software.
Download a separate register for each supplier and preserve the brief version used for comparison. Revisit the register when the team, scope or architecture changes. The useful output is a record of what was inspected, what remains unknown and what you are prepared to commission next.
Written by Dmytro Nasyrov PhD, software architect with 24 years of production experience. Dmytro is the founder and CTO of Pharos Production. He works on production software architecture for FinTech, AI, Web3 and blockchain systems.