Enterprise-Grade LLM Gateway for Secure AI Adoption
Bringing enterprise security and governance to Large Language Model deployments
Organizations adopting AI face a critical challenge: How do you harness the power of LLMs while maintaining enterprise security, compliance, and cost control?
TrustedAI solves this by providing a secure, observable, and governable gateway between your organization and LLM providers. No more shadow IT, ungoverned API keys, or compliance nightmares.
- 🔒 Security Gaps: Direct API access bypasses enterprise security controls
- 💸 Cost Overruns: No visibility into usage patterns or cost attribution
- 📊 Compliance Blind Spots: Lack of audit trails and usage monitoring
- 🏢 Governance Challenges: Scattered API keys and ungoverned access
- 🔧 Developer Friction: Complex integration patterns for enterprise features
- SSO-Only Authentication: OIDC/SAML integration with Okta, EntraID, Keycloak
- Zero Local Passwords: Eliminate credential vulnerabilities
- Session Management: Secure token-based access with easy revocation
- Multi-Tenant Architecture: Organization-level isolation and controls
- Real-Time Analytics: Usage patterns, cost tracking, and performance metrics
- OpenTelemetry Integration: Seamless monitoring stack integration
- Detailed Audit Trails: Full request/response logging for compliance
- Cost Attribution: Per-user, per-model billing and quota management
- Dual-Plane Design:
- Control Plane: Management APIs, user interface, and administration
- Data Plane: High-performance LLM request routing and processing
- Multi-Provider Support:
- Frontend APIs: OpenAI, Anthropic (Gemini planned)
- Backend Routing: OpenRouter with planned support for Bedrock, Vertex AI
- Database-Driven: PostgreSQL-backed configuration and state management
- Modern Web UI: React/TypeScript dashboard with real-time updates
- ConnectRPC APIs: Type-safe, high-performance API layer with HTTP/2
- Comprehensive Testing: Unit, integration, and acceptance test suites
- Docker Compose: Simple local development setup
We're actively developing these enterprise-critical features:
- 🛡️ AI Guardrails: Content filtering, safety controls, and policy enforcement
- 📝 Prompt Management: Centralized prompt templates and version control
- 🎯 Advanced Routing: Load balancing, failover, and A/B testing
- 📊 Enhanced Analytics: Custom dashboards and reporting
- 🔗 More Integrations: Bedrock, Vertex AI, Azure OpenAI Service
- Mise for tool management
- Docker & Docker Compose for local services
git clone https://github.com/MadsRC/trustedai.git
cd trustedai
# Install tools and dependencies
mise install
mise run dev/bootstrap.sh# Start PostgreSQL, Keycloak, and OTEL Collector
docker compose up -d
# Wait for services to be ready (especially database)
docker compose ps# Start the backend (control plane + data plane)
DATABASE_URL="postgres://postgres:postgres@localhost:5432/postgres" \
LLMGW_BASE_URL="http://localhost:5173" \
go run cmd/trustedai/main.go
# In another terminal, start the frontend
cd frontend
npm run devOpen http://localhost:5173 and sign in using:
- SSO Provider:
http://localhost:8080/realms/testrealm01 - Admin Console: http://localhost:8080/admin (admin/admin)
- IAM Overview - Identity and Access Management
- Model Aliasing - Model routing and aliases
- Testing Guide - Running tests and contributing
- Development Setup - Developer instructions
┌─────────────────┐ ┌─────────────────┐
│ React App │ │ Control Plane │
│ (Port 5173) │◄──►│ (Port 9999) │
└─────────────────┘ └─────────────────┘
│
▼
┌─────────────────┐
│ Data Plane │
│ (Port 8081) │
└─────────────────┘
│
▼
┌─────────────────────────┐
│ LLM Providers │
│ OpenAI │ Anthropic │... │
└─────────────────────────┘
Control Plane: User management, configuration, analytics, and web UI Data Plane: High-performance LLM request routing and response handling
We welcome contributions! TrustedAI is in active development and we're looking for:
- 🐛 Bug Reports: Help us identify and fix issues
- 💡 Feature Requests: Share your enterprise AI governance needs
- 🔧 Code Contributions: Check our issues for good first contributions
- 📖 Documentation: Help improve our guides and examples
- Fork the repository
- Create a feature branch:
git checkout -b feature/amazing-feature - Follow our conventions: Read CLAUDE.md for coding standards
- Test your changes:
mise run test:unit - Format code:
mise run format - Lint code:
mise run lint - Commit with conventional commits:
feat: add amazing feature - Open a Pull Request
- Go: 1.24.4+
- Node.js: 24.2.0+
- PostgreSQL: 17+
- Docker: For local development
TrustedAI is licensed under the GNU Affero General Public License v3.0.
We chose AGPL-3.0 because we believe enterprise AI infrastructure should be transparent, auditable, and community-driven. This ensures that improvements to TrustedAI benefit everyone in the ecosystem.
TrustedAI is currently in Alpha. We're actively developing core features and welcome feedback from enterprise teams tackling AI governance challenges.
- ✅ Core Architecture: Control/Data plane separation
- ✅ Authentication: SSO integration with OIDC
- ✅ Multi-Provider: OpenAI, Anthropic, OpenRouter support
- ✅ Observability: Usage tracking and metrics
- 🚧 Guardrails: In development
- 🚧 Prompt Management: Planned
- 🚧 Advanced Routing: Planned
- GitHub Issues: Report bugs and request features
- Discussions: Share ideas and ask questions
Ready to bring enterprise security to your AI deployment?
⭐ Star this repo if TrustedAI solves a problem you're facing!