Transform raw voice recordings and phone conversations into structured business intelligence, real-time intent classification, diarized transcripts, and automated CRM action items.
| 🔗 GitHub Repository | 🌐 Live Web Application |
|---|---|
| https://github.com/bikram73/VoiceDesk_AI | https://voice-desk-ai.netlify.app/ |
📖 Description |
🚀 Section |
|---|---|
View the project features and capabilities. 👉 |
 |
View the technologies, frameworks, and programming languages used. 👉 |
 |
Explore the project's folder and file organization. 👉 |
 |
Follow the installation steps and local development setup. 👉 |
 |
Understand how confidence scores are calculated and interpreted. 👉 |
 |
View the available REST API endpoints and usage examples. 👉 |
 |
Explore the complete system architecture, AI workflow, processing pipeline, data flow, deployment design, and technical decisions. 👉 |
 |
Review implementation details, AI pipeline, performance metrics, benchmarking, validation strategy, privacy, testing, and technical specifications. 👉 |
 |
Review processing speed, latency, and performance benchmarks. 👉 |
 |
Understand the current limitations and known failure cases of the AI extractor. 👉 |
 |
- Live Microphone Capture: High-definition audio recording directly from the browser with simulated visualizer waveforms and automated duration tracking.
- Batch Audio File Upload: Drag-and-drop support for
.wav,.mp3,.m4a,.flac, and.oggaudio files up to 50MB. - 5 Preset Audio Presets: 1-click loading for Dental Booking, Enterprise Sales, Billing Dispute, HVAC Emergency, and Legal Consult audio files.
- Audible Speech Playback & Downloads: Audible dialogue preview with animated speech synthesis and instant audio file download buttons.
- 5 Realistic Transcript Presets: Immediate testing without requiring microphone permissions.
- Zero-Latency Neural Speech-to-Text: Converts raw telephony speech into timestamped dialogue turns with automated speaker diarization (
Callervs.AI Receptionist). - Structured Business Entity Extraction:
caller_name,company_name,phone,emailservicerequested,products_mentionedappointment_date,meeting_time
- Intent Classification: Classifies calls across 10 business intents (Appointment Booking, Sales Inquiry, Billing Issue, Technical Support, Complaint, etc.).
- Urgency & Priority Triaging: Labels each call with Low, Medium, High, or Critical priority badges.
- Sentiment & Emotional Analysis: Real-time sentiment rating (Happy, Interested, Neutral, Frustrated, Angry, Urgent) with percentage confidence scoring.
- Executive Summaries & Action Items: Generates 2-sentence executive highlights, detailed context overviews, and concrete follow-up instructions for receptionists.
- 100% Client-Side Sandboxing: Analyzed calls and data records are persisted directly inside the browser's
localStorage(voicedesk_calls_v1). - Initial Clean State: Starts cleanly empty with zero placeholder clutter until you analyze calls or load demo data.
- Instant Search & Multi-Filter: Filter calls in real-time by intent, priority, or free-text keywords.
- Data Export & Portability: Export all stored call sessions to CSV or structured JSON with a single click.
| Layer | Technology / Tool | Purpose |
|---|---|---|
| Frontend Framework | React 19 | Core Single Page Application component architecture |
| Language | TypeScript 5.8 | End-to-end static type safety and interface definitions |
| Styling Engine | Tailwind CSS v4 | Utility-first styling with modern CSS variables |
| Bundler & Dev Server | Vite 6 | Lightning-fast HMR and optimized production bundling |
| AI Neural Core | Google Gemini 3.8 Flash (@google/genai) |
Multimodal speech processing, entity extraction & reasoning |
| Backend API Proxy | Express 4.21 + Node.js 22 | Secure API proxying and base64 audio normalization |
| Icons & Typography | Material Symbols + Inter & Poppins | Enterprise typography and crisp UI iconography |
| State & Persistence | React Context API + LocalStorage | Real-time state management and zero-leak offline storage |
| Deployment & Hosting | Netlify | Continuous deployment, CDN edge delivery & SSL |
| Code Repository | GitHub | Version control & open-source repository |
VoiceDesk_AI/
├── 📄 ARCHITECTURE.md # Comprehensive system architecture & data pipeline
├── 📄 TECHNICAL_REPORT.md # Benchmark specifications & testing metrics
├── 📄 README.md # Master documentation and setup guide
├── 📄 package.json # NPM dependencies, scripts, and engine versions
├── 📄 tsconfig.json # TypeScript configuration
├── 📄 vite.config.ts # Vite bundler plugins and Tailwind CSS config
├── 📄 server.ts # Express backend proxy for Gemini 3.8 Flash API
├── 📄 index.html # Main HTML entry with fonts and 2026 meta tags
├── 📄 metadata.json # AI Studio applet capabilities & permissions
├── 📁 public/ # Public assets, logos, and favicon
└── 📁 src/
├── 📄 main.tsx # Application bootstrap entry point
├── 📄 index.css # Tailwind CSS v4 directives & custom utilities
├── 📄 App.tsx # Top-level page router (Home, Analyzer, Dashboard, etc.)
├── 📄 types.ts # TypeScript definitions for CallAnalysis & Filters
├── 📁 context/
│ └── 📄 CallSessionContext.tsx # LocalStorage sync, session state & call statistics
├── 📁 services/
│ └── 📄 voiceAnalysis.ts # Gemini API client, audio preprocessing & sample presets
└── 📁 components/
├── 📄 Navbar.tsx # Top navigation bar with active badges
├── 📄 HomeView.tsx # Product landing page & feature highlights
├── 📄 VoiceAnalyzerView.tsx # Voice recording, upload zone, presets & live analyzer
├── 📄 DashboardView.tsx # Analytics cards, call detail viewer, table & export
├── 📄 AboutProjectView.tsx # Technical blueprint, 2026 roadmap & stack specs
└── 📄 SettingsView.tsx # AI models, audio ingestion & LocalStorage controls- Node.js: v18.0.0 or higher (Node.js 22 LTS recommended)
- NPM or Bun / Yarn / PNPM
- Google Gemini API Key: Obtainable from Google AI Studio
-
Clone the Repository:
git clone https://github.com/bikram73/VoiceDesk_AI.git cd VoiceDesk_AI -
Install Dependencies:
npm install
-
Configure Environment Variables: Create a
.envfile in the project root:cp .env.example .env
Add your Gemini API key:
GEMINI_API_KEY=your_gemini_api_key_here PORT=3000
-
Start the Development Server:
npm run dev
Open your browser and navigate to
http://localhost:3000. -
Build for Production:
npm run build npm start
-
Deploy to Netlify:
- Live Application: https://voice-desk-ai.netlify.app/
- Push code to
mainbranch on GitHub:https://github.com/bikram73/VoiceDesk_AIto trigger automated builds.
VoiceDesk AI calculates sentiment, priority, and entity confidence using a calibrated probabilistic score between 0% and 100%:
0% ──────────────────── 40% ──────────────────── 75% ──────────────────── 100%
│ │ │ │
🔴 Critical / Frustrated 🟠 Urgent / Unresolved 🟡 Neutral / Standard 🟢 High / Interested
- 90% - 100% (High Confidence / Positive): Caller exhibits clear, decisive intent (e.g., explicit confirmation of date, time, and service).
- 70% - 89% (Moderate Confidence / Interested): General inquiries, standard consultations, and pricing discussions.
- 40% - 69% (Neutral / Undetermined): Ambient audio, partial background chatter, or unresolved customer questions.
- 0% - 39% (High Urgency / Escalation): Billing disputes, emergency repairs, compressor failures, or expressions of frustration.
Processes an audio file (base64) or raw text transcript and returns structured call intelligence.
Content-Type: application/json{
"audioBase64": "data:audio/wav;base64,UklGRi...",
"mimeType": "audio/wav",
"transcriptText": "Caller: Hi, my name is Sarah Jenkins. I want to book an appointment tomorrow at 10 AM...",
"fileName": "dental_call.wav"
}curl -X POST http://localhost:3000/api/analyze \
-H "Content-Type: application/json" \
-d '{
"transcriptText": "Caller: Hello, I am John Smith from Apex Dental. I would like to book a dental checkup tomorrow at 11:00 AM. My phone is 987-654-3210."
}'{
"id": "CALL-8912",
"caller_name": "John Smith",
"company_name": "Apex Dental",
"phone": "987-654-3210",
"email": "N/A",
"intent": "Appointment Booking",
"priority": "High",
"service": "Dental Checkup",
"appointment_date": "Tomorrow",
"meeting_time": "11:00 AM",
"follow_up_needed": true,
"callback_requested": true,
"products_mentioned": ["Dental Checkup"],
"sentiment": "Interested",
"sentiment_score": 92,
"short_summary": "John Smith called to book a dental checkup for tomorrow at 11:00 AM.",
"detailed_summary": "John Smith from Apex Dental requested a morning consultation slot at 11:00 AM and provided his phone contact for confirmation.",
"next_action": "Call customer at 987-654-3210 to confirm the 11:00 AM appointment slot.",
"transcript": [
{ "speaker": "Caller", "text": "Hello, I am John Smith from Apex Dental. I would like to book a dental checkup tomorrow at 11:00 AM.", "timestamp": "00:04" }
],
"date_time": "Oct 28, 2026 • 10:42 AM",
"duration": "01:25",
"file_name": "dental_call.wav"
}| Operation | Latency (P50) | Latency (P95) | Throughput |
|---|---|---|---|
| Audio Ingestion & Buffer Prep | 12 ms |
28 ms |
Instantaneous |
| Gemini 3.8 Flash Inference | 720 ms |
1,140 ms |
Concurrent |
| Entity Extraction & JSON Parsing | 4 ms |
8 ms |
In-memory |
| LocalStorage Commit & State Sync | 2 ms |
5 ms |
Zero-blocking |
| CSV / JSON Export Generation | 6 ms |
15 ms |
Client-side |
- Audio File Size: Maximum upload buffer limit is currently 50MB per individual file.
- Heavy Background Noise: In recordings with severe ambient noise (> -10 dB SNR), phoneme diarization accuracy may decrease; noise suppression is recommended in Settings.
- Dialect Discrepancies: Complex non-standard domain jargon (such as proprietary medical codes) requires explicit phonetic spelling in transcript mode for 100% extraction precision.
Repository: https://github.com/bikram73/VoiceDesk_AI • Deployment: https://voice-desk-ai.netlify.app/
Made with ❤️ using React 19, TypeScript, and Gemini 3.8 Flash.