[Remote] Principal Production Engineer, Database Infrastructure
Note: The job is a remote job and is open to candidates in USA. GitHub is a leading platform for agentic software development and secure software delivery. The company is seeking a Principal Production Engineer to scale its data platform, improve database reliability, lead design reviews, and develop SDKs and tooling for resilient, high-performing systems.
Responsibilities
- Partner with product and feature teams by leading design reviews and shaping how they model, access, and scale their data so the applications they build are performant, available, and operable at GitHub's scale
- Design and ship SDKs, client libraries, and the tooling applications are built on
- Set reliability strategy for GitHub's database platforms across multiple systems, defining SLOs and operational standards
- Write technical documentation and advocate for the health and quality of the systems the team builds
- Participate in an on-call rotation and respond to incidents as needed
- Develop and design plans for disaster recovery, load shedding, and regional failover
Skills
- * 11+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python
- * OR Associate's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 10+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python
- * OR Bachelor's Degree in Computer Science or related field AND 9+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python
- * OR Master's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 7+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python
- * OR Doctorate in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 5+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, Go, Ruby, Rust, or Python
- * OR equivalent experience
- * 5+ years experience operating large-scale distributed systems in production, including participation in an on-call rotation
- * Excitement about building, operating, and maintaining resilient, scalable systems that impact a global community of users with the ability to break down complex systems into manageable components
- * A track record of partnering with product and feature teams and materially changing the reliability and scalability of what they ship
- * Ability to influence engineering decisions and proactively engage in system design conversations
- * Experience running stateful services on managed cloud data stores, specifically Azure Cosmos DB and Azure SQL Database, or equivalents such as DynamoDB, Aurora, or Cloud Spanner with a focus on partition and index design, consistency and isolation tradeoffs, throughput provisioning, and hot-partition diagnosis
- * Experience diagnosing and resolving application-level scalability problems: N+1 query patterns, hot partitions, unbounded fan-out, cache stampedes, and data access patterns that don't scale
- * A track record of building internal platforms, SDKs, or developer tools adopted across an engineering organization, written in production-grade Go, Python, Ruby, or Rust
- * Deep familiarity with the failure modes of large-scale systems, both in the application and in the platform beneath it e.g. cascading failures, retry storms, thundering herds, partial outages, throttling and quota limits, control plane outages, noisy neighbors, and the patterns that mitigate them
- * Experience leading large-scale cloud migrations of live, high-traffic services e.g. dual-write and backfill strategies, traffic shifting, correctness verification, and rollback under load
- * Effective communication skills and willingness to pair on problems, brainstorm in public, and enthusiastically engage with your teammates in group problem solving
Benefits
- Certain roles are eligible for benefits and additional rewards, including annual bonus and stock.
- Certain roles also have the opportunity to earn sales incentives based on revenue or utilization, depending on the terms of the plan and the employee's role.
- Remote-first work arrangement
- Generous learning and growth opportunities
- Excellent benefits to support you, wherever you are
Company Overview
Company H1B Sponsorship