How to Build an AI Voice Agent from Scratch | ElevenLabs + n8n Tutorial

Summary

The video is a step-by-step tutorial on building an inbound AI phone agent with ElevenLabs and automating post-call data handling with n8n.

The creator begins by demonstrating an AI education assistant answering a live call for a coaching center. The assistant discusses available courses, fees, study modes, and EMI options, then ends the call when the caller says goodbye.

Building and Configuring the Voice Agent

1. Create an agent in ElevenLabs: Start with a blank agent and give it a name.
2. Set its instructions: Use the system prompt to define the agent’s role, knowledge, personality, and response style. The tutorial demonstrates ElevenLabs’ AI-assisted prompt generation, with the option to edit the result manually.
3. Choose voice and language settings: Select a voice, set a default language, and enable language detection so the agent can respond in the caller’s language.
4. Choose an LLM: The speaker describes the LLM as the agent’s “brain” and recommends Gemini for this example, citing its relative cost efficiency on the platform.
5. Set the greeting and interruptions: Customize the opening message and enable interruptible responses, allowing the agent to stop speaking when the caller talks.
6. Add a knowledge base: Provide information about the business and its courses by entering text, uploading a file, or using a URL. The example information was drafted with ChatGPT.
7. Enable tools and expressive behavior: Turn on language detection, call-ending capability, and “skip turn” behavior for callers who ask the agent to wait. The tutorial also demonstrates expressive mode for warmer, more human-sounding delivery.
8. Review calls: ElevenLabs’ call analysis provides transcripts and summaries, which can help monitor customer interactions and satisfaction.

Connecting a Phone Number

The tutorial connects the agent to a phone number through Twilio. It covers creating or logging in to a Twilio account, buying a number, entering the required account credentials in ElevenLabs, and assigning the number to the agent.

Once the agent is made public, it can answer inbound calls on that number. The speaker notes that Twilio offers trial credit, but a real business may incur costs.

Automating Post-Call Data with n8n

The second part of the tutorial sends call information to a Google Sheet after each call:

1. Create an n8n workflow with a Webhook node and copy its test URL.
2. Configure an ElevenLabs post-call webhook to send a POST request to that URL.
3. Add a Google Sheets node, connect a spreadsheet, and create columns for the caller’s mobile number, call time, and summary.
4. Map the incoming call data to the spreadsheet fields and test the workflow.
5. Activate the workflow so future call details are added automatically.

The speaker also mentions possible extensions, such as sending WhatsApp messages, payment links, appointment-related follow-ups, or feedback requests. These are presented as future possibilities rather than features built in the tutorial.

Tutorial Highlights

• Guide: Create and configure an ElevenLabs voice agent from scratch.
• Guide: Connect the agent to a phone number using Twilio.
• Guide: Send post-call information through an ElevenLabs webhook to n8n and Google Sheets.
• Feature demonstrations: Language detection, caller interruptions, call ending, expressive speech, call transcripts and summaries, and “skip turn.”
• Future topic: The video focuses on inbound calls. The creator says a follow-up tutorial on outbound calls may be released if the video reaches 2,000 views.

Main speaker/source: The presenter from Manish Digital Academy. The live-call demonstration also features the ElevenLabs AI education assistant.