AI-Adaptive Onboarding Engine

An AI-driven onboarding system that converts static training into personalized role-readiness pathways.

The app accepts a role + resume, auto-parses candidate profile and projects, computes skill gaps, generates a personalized roadmap, and then drives learning through dedicated module pages with embedded resources, tests, and score-based recalibration.

Hackathon Requirements Coverage

• Intelligent Parsing: resume + JD parsing with skill extraction, alias matching, and experience-level inference.
• Editable Profile Loop: parser pre-fills candidate details and users can correct skills/levels before path generation.
• Dynamic Mapping: skill-gap analysis and prerequisite-aware adaptive pathway generation.
• Continuous Adaptation: score-driven reconfiguration of the pathway after each assessment cycle.
• Module-Centric UX: each module opens as its own learning page with resources + tests.
• Functional Interface: web UI for text/file uploads (PDF/TXT/MD) and roadmap visualization.
• Grounding and Reliability: all recommendations are selected from a fixed internal learning catalog only.
• Reasoning Trace: stepwise trace returned by API and rendered in UI.

Tech Stack

• Frontend: Next.js (App Router), React, Tailwind CSS, shadcn/ui
• Backend: Next.js route handlers + Python ML microservice (FastAPI)
• Parsing: pdf2json for PDF text extraction
• Model Layer: optional Groq LLM (llama-3.1-8b-instant) for richer resume profile extraction when GROQ_API_KEY is set
• Adaptive Logic: custom TypeScript engine in lib/adaptive/engine.ts
• ML Pipelines: role classifier training (TF-IDF + Logistic Regression), RL module policy training (Q-learning)
• Vision/OCR: pdfplumber + pytesseract (ML service endpoint /vision/parse)

Core Logic (Skill-Gap Analysis)

1. Input ingestion

• Users can paste text or upload resume/JD files.
• PDF and text files are normalized and sanitized.

2. Skill extraction

• Resume and JD are scanned against a role skill catalog.
• Alias-aware matching maps variants (for example reactjs, node, postgresql) to canonical skills.
• Proficiency level is inferred using context signals (keywords + years of experience).

3. Candidate edit and validation

• Parsed candidate profile (name, years, skills, projects) is user-editable.
• Final planning uses edited values, not raw extraction only.

4. Gap computation

• For each JD-required skill, compute gap = targetLevel - currentLevel.
• Keep only non-zero gaps and prioritize by gap magnitude.

5. Adaptive pathing algorithm (original implementation)

• Select modules that best close prioritized gaps.
• Enforce prerequisites by recursively expanding required dependency modules.
• Topologically sort selected modules to ensure teach-before-use ordering.
• Group into progressive phases: Foundation, Core Build, Role Readiness.

6. Assessment and reconfiguration

• Generated diagnostic questions are mapped to missing skills.
• Low scores trigger easier/foundation reinforcement; strong scores push advanced progression.

7. Web resources

• Missing skills are mapped to external resources grouped by type: documentation, video, coding, paper, practice.

8. Reasoning trace

• Return deterministic trace events for role detection, extraction, gap computation, pathing, and final assembly.

Repository Structure

study_board/
├── app/
│   ├── page.tsx                                  # Product landing/dashboard
│   ├── intake/page.tsx                           # Step 1: resume + role intake
│   ├── analysis/page.tsx                         # Step 2: known vs missing skill analysis
│   ├── plan/page.tsx                             # Phase/module roadmap page
│   ├── assessment/page.tsx                       # Step 4: adaptive tests + reconfiguration
│   ├── plan/module/[moduleId]/page.tsx          # Module learning + test page
│   ├── catalog/page.tsx                          # Grounding catalog view
│   ├── methodology/page.tsx                      # Adaptive algorithm explainer
│   └── api/adaptive-onboarding/
│       ├── parse/route.ts                        # Parse resume/JD -> editable profile
│       ├── generate/route.ts                     # Generate adaptive path from edited profile
│       ├── reconfigure/route.ts                  # Reconfigure path from assessment scores
│       ├── module/[moduleId]/route.ts            # Module-specific resources + tests
│       ├── module/[moduleId]/complete/route.ts   # Module completion + pace recalibration
│       ├── code-eval/route.ts                    # Free local code evaluator
│       └── analyze/route.ts                      # Backward-compatible combined endpoint
├── components/adaptive/adaptive-onboarding-workbench.tsx
├── lib/adaptive/
│   ├── catalog.ts                                # Fixed course + skill catalog
│   ├── resources.ts                              # External resource mapping by skill
│   ├── assessment.ts                             # Skill diagnostics and scoring
│   ├── file-text.ts                              # PDF/text extraction utilities
│   ├── rl-policy.ts                              # Runtime RL policy loader
│   └── engine.ts                                 # Parsing + gap + pathing logic
├── ml/
│   ├── train_rl_module_priority.py               # Q-learning training script
│   ├── module_catalog.json                       # RL action space
│   ├── role_skill_catalog.json                   # RL state space
│   └── rl_module_priority.json                   # Trained policy output
├── ml_service/
│   ├── app/main.py                               # FastAPI inference service
│   ├── app/parser.py                             # Skill/project extraction logic
│   ├── app/model.py                              # Role classifier runtime
│   ├── app/vision.py                             # OCR/PDF text extraction
│   ├── scripts/download_datasets.py              # Kaggle dataset download
│   ├── scripts/build_training_data.py            # Train data preparation
│   ├── scripts/train_role_classifier.py          # Role model training
│   ├── scripts/evaluate_pipeline.py              # Metrics generation
│   └── Dockerfile
├── docker-compose.yml                            # Deploy web + ML service
├── Makefile                                      # One-command workflows
├── middleware.ts                                 # Restricts UI to onboarding routes
├── docs/
│   ├── DEMO_SCRIPT.md                            # 2-3 minute demo flow
│   └── HACKATHON_5_SLIDES.md                     # 5-slide presentation content
├── Dockerfile
└── README.md

Local Setup

Prerequisites:

• Node.js 18+
• npm

Required and optional env (.env.local):

# Resume/JD enrichment models
GEMINI_API_KEY=your_gemini_key
GROQ_API_KEY=optional_fallback_key

# Python ML microservice endpoint
ML_SERVICE_URL=http://localhost:8001

# Optional future model providers (not required by current code)
OPENAI_API_KEY=
TOGETHER_API_KEY=
HUGGINGFACEHUB_API_TOKEN=

Install and run:

npm install
npm run dev

Open http://localhost:3000.

Full Deployment (Web + ML Service)

docker compose up --build

Services:

• Web app: http://localhost:3000
• ML service: http://localhost:8001/health

Reinforcement Learning Training (Separate Step)

This project includes a Q-learning trainer that learns role-wise module priorities and writes:

• ml/rl_module_priority.json

Run training:

npm run train:rl

How it works:

• State: remaining skill gaps for a role.
• Action: next module from internal catalog.
• Reward: gap reduction minus time cost, plus completion bonus.
• Policy usage: when ml/rl_module_priority.json exists, /api/adaptive-onboarding/generate uses it automatically.

If no RL policy exists, the app falls back to deterministic heuristic pathing.

Recommended Models (Local + Open Source)

You can run this platform as-is, then upgrade model quality incrementally:

1. Resume/JD parsing and normalization

• Local: Qwen2.5-7B-Instruct or Llama-3.1-8B-Instruct via Ollama/vLLM.
• Hosted open-source providers: Groq, Together, Fireworks with same model families.

2. Embeddings for skill matching and retrieval grounding

• Local: bge-large-en-v1.5 or nomic-embed-text-v1.5.
• Store vectors in pgvector, Qdrant, or Milvus.

3. Adaptive assessment/question generation

• Local: Mistral-7B-Instruct or Qwen2.5-14B-Instruct for stronger reasoning.
• Add strict JSON schema outputs for stable scoring pipelines.

4. OCR/document parsing (if resume PDFs are noisy/scanned)

• Keep current pdfplumber + pytesseract.
• Optional upgrade: PaddleOCR for better table/scan handling.

5. Coding assessment execution

• Current local evaluator works for baseline.
• Production hardening: self-host Judge0 CE in isolated containers.

Dataset and Training Pipeline (ML Service)

1. Download datasets (requires Kaggle credentials):

npm run ml:download

2. Build train corpus + train role classifier:

npm run ml:train

3. Evaluate and store metrics:

npm run ml:evaluate

Model artifact:

• ml_service/models/role_classifier.joblib

Metrics artifact:

• ml_service/models/metrics.json

API Usage

Endpoint:

• POST /api/adaptive-onboarding/parse
• POST /api/adaptive-onboarding/generate
• POST /api/adaptive-onboarding/reconfigure
• GET /api/adaptive-onboarding/module/:moduleId
• POST /api/adaptive-onboarding/module/:moduleId/complete
• POST /api/adaptive-onboarding/code-eval
• POST /api/adaptive-onboarding/analyze (legacy combined mode)

ML service endpoints:

• GET /health
• POST /parse
• POST /vision/parse

Supported input modes:

• multipart/form-data with fields: resumeText, jdText, resumeFile, jdFile
• application/json with fields: resumeText, jdText

Example parse request:

curl -X POST http://localhost:3000/api/adaptive-onboarding/parse \
  -H 'Content-Type: application/json' \
  -d '{"resumeText":"3 years React and Node.js...","jdText":"Looking for full stack engineer with TypeScript, testing, cloud"}'

Example generate request:

curl -X POST http://localhost:3000/api/adaptive-onboarding/generate \
  -H 'Content-Type: application/json' \
  -d '{"resumeText":"3 years React and Node.js...","selectedRole":"software_engineer","editedProfile":{"candidateName":"Alex","totalYearsExperience":3,"professionalSummary":"Full stack engineer","skills":[{"skill":"react","level":3,"evidence":"Built dashboards"}],"projects":[]}}'

Docker

Build and run:

docker build -t adaptive-onboarding-engine .
docker run --rm -p 3000:3000 adaptive-onboarding-engine

Datasets and Model Compliance

This project can be trained/evaluated with public datasets; cite sources when used.

Suggested datasets (from challenge prompt):

• Resume Dataset: https://www.kaggle.com/datasets/snehaanbhawal/resume-dataset/data
• O*NET releases: https://www.onetcenter.org/db_releases.html
• Jobs and Job Descriptions: https://www.kaggle.com/datasets/kshitizregmi/jobs-and-job-description

Transparency notes:

• Current prototype uses rule-based extraction + deterministic adaptive mapping.
• Current parser supports Gemini (gemini-1.5-flash) with fallback to Groq Llama (llama-3.1-8b-instant).
• If you add embeddings or more models (Llama, BERT, Mistral, etc.), document exact versions and prompts.

Free Coding Evaluation

• Built-in free evaluator: /api/adaptive-onboarding/code-eval (local Python/Node execution with timeout).
• Fully open-source hosted option: self-host Judge0 CE (no paid tier required).

Internal Evaluation Metrics (Recommended)

• Skill Extraction Precision/Recall on labeled resume-JD pairs.
• Gap Closure Coverage: percent of high-priority gaps addressed by generated pathway.
• Prerequisite Validity Rate: percent of pathways with no missing prerequisites.
• Training Time Reduction: hours avoided vs static curriculum baseline.
• Role Readiness Coverage: weighted completion ratio of JD-required competencies.

Submission Assets

• Demo video guide: docs/DEMO_SCRIPT.md
• 5-slide deck content: docs/HACKATHON_5_SLIDES.md
• Dataset/model compliance notes: docs/DATASETS.md
• Deployment runbook: docs/DEPLOYMENT.md
• ML pipeline runbook: docs/ML_PIPELINE.md

License

MIT