I live in Seattle, WA, and work on AI agents in Visual Studio Code and GitHub Copilot at Microsoft. My work spans agent products, evaluation, and agent observability. You can find it at @zhichli, primarily across VS Code and the GitHub Copilot SDK, and my personal work at @b1tank.

Website · Writing · LinkedIn · X

I led the VS Code evaluation engineering workstream and created the evaluation harness and associated agentic analysis and reporting framework (public showcase: What 50,000 agent runs taught us), as well as VSC-Bench, a benchmarking suite for VS Code and its extensions. I worked closely with model providers, including Microsoft AI, OpenAI, and Anthropic, to produce accurate, meaningful evaluations, and with the community to advance observability in VS Code.

Before joining the VS Code core team, I worked on the GitHub Copilot Applied Science team, focusing on coding data synthesis, model training pipelines, and recommendation systems (public display). Earlier, I helped build cloud services including Azure Datadog and Azure Managed Grafana, where I was a founding engineer. Across these roles, I developed broad experience in full-stack application development, cloud infrastructure, operations, and observability.

My path into software began in scientific computing during my PhD in chemical engineering, where I built machine learning models in MATLAB and aerosol models in Fortran and C++. Those experiences shaped my appreciation for performance, correctness, and clarity.

Having worked in both academia and the AI and software industries, I believe:

“AI can save us work, but it can also distance us from the learning that work provides. Curiosity is fundamental to human nature. We want to learn more, not less - and AI should be wielded to advance education, not deprive us of opportunities to learn.”

— My bet in the AI era

That is why I began building an educational site for my dad, a longtime mathematics teacher, and learning projects for myself to catch up on foundational technologies that still matter.

I also enjoy basketball 🏀 and billiards 🎱. I’m a Stephen Curry and Golden State Warriors fan, and I admire Curry’s humility, work ethic, and competitive drive.

My public collection of reusable agent skills is available at b1tank/skills. Other tools I have built are shown below.

• Agent monitoring in VS Code — The public guide to collecting and inspecting traces, logs, and metrics from agent sessions.
• What 50,000 agent runs taught us — A co-authored analysis of repeated evaluation runs across models, prompts, and agent behavior.
• Enterprise OpenTelemetry controls for VS Code agents — Policy-backed telemetry export controls for VS Code agent sessions in enterprise-managed environments.
• VS Code agent OpenTelemetry walkthrough — The official GenAI observability walkthrough uses VS Code agent instrumentation I worked on, visualized in the Aspire Dashboard.

• VS Code — Agent Host OpenTelemetry, trace context, content controls, and model telemetry.
• VS Code Copilot Chat — GenAI instrumentation, local SQLite trace storage, OTLP export, and agent activity signals.
• Grafana — Fixed OpenTelemetry Jaeger context propagation and added configurable outbound user-agent headers.
• Dagger container-use — Native Windows support, a Git-notes hang fix, and command documentation.
• Pi coding agent — Clarified and reconciled coding-agent keybinding behavior in the public documentation.