Engineering a bilingual AI voice agent for loan lead qualification_
A Hindi-English voice agent that answers and places calls, handles interruption, captures loan intent, and returns structured lead data to the sales workflow.
- AI Voice Agent
- Hindi Voice AI
- SIP Telephony
- Lead Qualification
- FinTech Automation
Loan enquiries arrived faster than agents could qualify them, especially outside working hours. A scripted IVR could route calls but could not hold a useful conversation or reliably capture name, city, loan type, and requested amount.
Parallaxis combined SIP telephony, streaming audio, voice-activity detection, speech recognition, constrained conversation logic, text-to-speech, and structured CRM hand-off in a latency-aware call pipeline.
The system creates a repeatable first conversation for inbound and outbound leads, captures qualification fields in a structured form, and hands uncertain or high-intent cases to human sales staff with call context intact.
Overview
The project began with a narrow business question: can every new loan enquiry receive a useful first conversation, even when a sales representative is unavailable? The answer required more than connecting a language model to a phone number.
Parallaxis engineered a bilingual Hindi-English voice workflow for inbound and outbound qualification. The agent introduces the service, understands the caller’s request, captures a small set of lending fields, handles interruptions, and produces a structured lead record for follow-up.
Why a phone conversation is different
Voice exposes delays that users tolerate in chat. Speech must be detected quickly, partial audio must be transcribed, the response must stay short enough to sound natural, and text-to-speech must stop when the caller interrupts. A failure at any one layer makes the whole agent feel unresponsive.
Financial conversations also require restraint. The agent qualifies interest and gathers context; it does not invent eligibility decisions, promise approval, or replace the lender’s verification process.
The real-time call pipeline
Calls enter through Asterisk or FreeSWITCH over SIP. Voice-activity detection separates speech from silence, streaming speech recognition produces conversational text, and a constrained dialogue layer decides the next prompt or field to confirm. The response is synthesized and returned to the active call.
Barge-in support cancels playback when the caller starts speaking. Call state, retries, no-answer outcomes, and completion events are recorded through telephony callbacks so the sales workflow reflects what actually happened.
Bilingual collection without a rigid IVR
The agent can continue in Hindi or English and tolerate common code-switching. Instead of demanding every field in a fixed order, it extracts name, city, loan category, and approximate amount from the natural conversation, then asks only for information that is missing or ambiguous.
Values are confirmed before hand-off. This is particularly important for numbers, locations, and loan amounts, where a plausible transcription can still be commercially wrong.
Safety and human escalation
Conversation rules keep the agent within lead qualification. Uncertain requests, complaints, explicit requests for a person, and cases requiring policy judgment are escalated. Recordings and transcripts can be retained according to the client’s consent and data-retention requirements rather than being enabled indiscriminately.
Designed for operational use
The architecture separates telephony, speech providers, conversation logic, and CRM integration. This allows latency or language quality to be improved at one layer without rebuilding the complete system, and supports controlled concurrency as call volume grows.
Project highlights
- Inbound and outbound SIP call handling
- Hindi-English conversational flow
- Voice-activity detection and barge-in
- Streaming speech recognition and text-to-speech
- Structured capture of name, city, loan type, and amount
- Confirmation of ambiguous values
- Call-status and failure callbacks
- CRM-ready lead hand-off
- Human escalation boundaries
- Provider-independent voice pipeline