Key takeaways
- Choose one initial use case and agree the pilot's decision criteria before launch.
- Separate readiness testing from measuring commercial impact.
- Review qualification, CRM handoffs, buyer outcomes and costs against a fair comparison.
The demonstration sounds convincing. The agent answers project questions, handles an interruption and arranges a callback. A pilot dashboard later shows calls and appointments.
Should you expand the service?
For a residential developer, that decision needs evidence of what happened after the conversation. Were the requirements captured correctly? Did the sales team receive the next action? Did the buyer follow through?
Evaluate an AI pre-sales pilot by checking operational readiness, qualification quality, reliable CRM handoffs, buyer progress and cost. Agree those criteria before launch so the results can support a clear decision.
What decision should the pilot answer?
Choose one initial use case and define the change you want to test.
For fresh enquiries, the question might be whether faster contact produces more useful sales conversations. For database reactivation, it might be whether older enquiries yield current requirements and actionable callbacks.
Assess those campaigns separately. Their lead age, contactability and starting intent can differ.
Agree the project, lead sources, languages, eligibility rules and human follow-up responsibilities. Record the current process and baseline using the same definitions you plan to use during the pilot.
Define acceptable quality and the outcome that would justify expansion. Also agree what would require a pause or correction, such as invented project facts or repeated missed handoffs.
Is the system ready for real enquiries?
Readiness testing checks whether the workflow operates correctly. Commercial evaluation checks whether it improves the business outcome. Both deserve attention.
Before launch, test realistic situations: an unclear budget, a change of language, an interruption, a human request, an unavailable salesperson and a question outside approved project information.
Check callback changes, duplicate enquiries and failed CRM updates too. A polished call can still leave an incomplete sales task.
NIST's AI Risk Management Framework supports evaluation in the intended context, documented testing, human oversight and monitoring during operation. It offers useful evaluation principles rather than a real estate conversion benchmark. NIST AI RMF.
Begin with supervised review and retain representative test situations the agent was not tuned against. Record material changes to scripts, project information and the workflow.
Which results should you measure?
Follow the enquiry from eligibility to its verified outcome. Keep failed attempts and incomplete handoffs visible.
A technical call status needs interpretation. Twilio, for example, documents that a completed call can connect to a person, an IVR or voicemail. Completion alone does not establish a buyer conversation. Twilio call-status documentation.
Use an agreed scorecard:
| Measure | What to verify |
|---|---|
| Eligible enquiries | The complete assigned group, with exclusions explained. |
| Buyer conversations | Actual conversations, separated from attempts and voicemail. |
| Qualification quality | Requirements and decisions checked against agreed project criteria. |
| CRM handoffs | Correct record, usable context, responsible owner and saved next action. |
| Follow-through | Whether the promised callback or other action happened. |
| Buyer outcomes | Confirmed appointments, verified attendance and later bookings recorded separately. |
| Errors and rework | Incorrect information, missed actions and the effort needed to correct them. |
| Operating cost | The agreed cost scope, including human review and follow-up. |
An independent reviewer should check qualification decisions using the developer's rules. Include leads marked unsuitable or placed into later follow-up. Reviewing only successful handoffs can hide missed opportunities.
Explain every rate's denominator and observation period. Attendance among scheduled appointments answers a different question from attended visits among all eligible enquiries. Report the underlying counts alongside rates when reviewing the pilot.
How can you make the comparison fair?
Where feasible, compare the proposed service with your existing process over the same period. Keep project, source, lead age, language and staffing conditions comparable. The comparison group should continue receiving the normal service.
Random assignment can reduce selection bias. Keep repeat enquiries from the same buyer in the same group to reduce overlap. Avoid giving the AI the easiest leads while the existing team handles a different mix.
HM Treasury's evaluation guidance explains why randomised or well-designed comparison groups support attribution more strongly than a simple before-and-after comparison. Those general principles can inform a commercial pilot. Impact evaluation guidance.
When random assignment is impractical, use a carefully matched comparison and describe the remaining uncertainty.
Separate initial tuning from the agreed evaluation period. Record changes during evaluation so the results remain interpretable.
What should the cost assessment include?
Agree whether the assessment includes setup, managed service fees, calling and messaging, CRM add-ons, review and sales follow-through. Explain how one-time costs are treated.
For a chosen outcome, calculate cost using the same definition and scope for both groups. A cheaper call is useful only in the context of the workflow it supports.
Staff time saved may create capacity for better sales conversations. It becomes cash savings only when spending actually falls. Check whether the saved time was used and whether additional review or rework absorbed it.
Track later bookings where possible. A sale following an AI call is relevant evidence, but that sequence alone does not establish that AI caused it.
When should you expand, retest or stop?
Set the pilot's length and volume around the decision, baseline and outcome delay. A short test can show that calls and handoffs work while leaving commercial impact uncertain.
Expand when quality is acceptable, delivery is dependable and the observed benefit supports the operating cost. Fix and repeat when a specific issue prevents a fair assessment. Continue observing when outcomes have not matured. Pause when serious errors remain unresolved.
A buyer's request for a person can be a successful handoff. Evaluate whether that person received the context and followed through, rather than treating every human involvement as a failure.
Escalay builds and operates Speed to Lead and Database Reactivation services for residential developers. Agree the pilot's scope and responsibilities around your existing CRM and sales team. Our CRM integration guide explains what reliable delivery should include.
Keep the review grounded in traceable enquiry records, checked conversations and verified outcomes. That evidence gives the developer a practical basis for deciding what to run next.
Plan a pilot around a clear business decision
Discuss the scope, CRM handoffs and evaluation criteria for an Escalay pre-sales pilot.
