Peter Bhabra

Software Engineer | AI/ML & Distributed Systems

Summary

Software engineer with technical leadership experience in AI infrastructure and distributed systems. Currently leading inference platform development at Doubleword. Previously built high-availability data platforms processing 1M+ files daily at CMR Surgical.

Experience

Member of Technical Staff - 2023 to Present

Technical lead for AI inference platform development. Drive architecture decisions and technical direction for generative AI products.

  • Drive technical direction and quarterly planning. Manage sprint processes, delivery timelines, and cross-functional initiatives.
  • Developed a production inference engine supporting real-time and batch workloads with custom scheduling and resource management.
  • Engineered an AI gateway achieving 450x throughput improvement over existing solutions through connection pooling and request batching.
  • Architected a resilient, cost-effective batched inference platform for non-realtime workloads.
  • Led client engagements and deployed production LLM APIs and RAG applications across multiple environments.
  • Built a Rust-based inference engine using Candle for stable diffusion models, demonstrating low-level ML systems expertise.
  • Created the internal DevOps ecosystem spanning release publishing through client deployments.
  • Managed Kubernetes infrastructure including self-hosted clusters for CI/CD. Built Helm charts, custom operators, and deployment automation.

Graduate Software Engineer - 2021 to 2023

Worked within an agile, high-performing team delivering scalable data platforms using AWS serverless microservice architecture.

  • Built and maintained data platforms processing over one million files per day
  • Reduced AWS monthly costs from $60,000 to $35,000 (42% reduction) within one month
  • Built HTTP Live Streaming and fine-grained permissions systems from scratch using Open Policy Agent
  • Led team projects including facilitating collaboration sessions, breaking down ideas into executable stories, and quarterly planning
  • Worked with telemetry and video data to deliver customer insights with frontend teams
  • Utilized CI/CD techniques with TeamCity and GitlabCI for reliable releases across multiple deployment environments
  • Developed AWS serverless applications using microservice architecture

Education

Durham University

Master of Science Scientific Computing and Data Analysis - First Class Honours

Durham University

Bachelor of Science Physics - First Class Honours