Service Reliability Senior Administrator - Riot Operations Center

The Riot Operations Center (ROC) manages the 24x7 monitoring and response components of Riot's player-facing services. We are the first line of defense when things go wrong with any of our live services. We leverage technical familiarity with best-practice processes to rapidly remediate incidents. The team helps to create and mentor other Riot teams on best practice in alerting, monitoring, and operational processes.

As a Service Reliability Senior Administrator, you will work closely with the Live Operations team and Riot globally to establish and maintain a high-performing and highly available game service for players. You will monitor and support all aspects of LIVE production environments, development environments, and general system needs. Your technical skills and grasp of system integration will help you diagnose and communicate potential issues to Rioters and the community, improving the quality of the player experience. You will be a craft expert in operational and triaging skills. You can also be involved in projects which help contribute to improving overall service quality in the incident management and observability problem spaces.

Responsibilities:

• Triage and investigation of live incidents 
• Execute technical return to service actions in a fast-paced, distributed systems environment specifically microservices to quickly restore service and protect player experience
• Monitor the health of Riot’s distributed services using observability tools, identify gaps with alerting, runbook steps, processes or tools 
• Runbook execution and maintenance to keep documentation up to date
• Onboarding new team members
• Provide support, coordination during major launches, events and release deployments
• Contribute to project work with some guidance to develop automation scripts, utilities and new processes to continuously improve the incident management process
• Document details of incident response as needed to identify problems and improve overall incident management/response
• Participate in post-incident RCA meetings as required

Required Qualifications:

• Computer Science/IT Systems/Information Technology diploma or equivalent 
• 2+ years of Service Reliability Administration or equivalent role (System Analyst, System Administrator/Engineer, Live Operations, Network Administrator, NOC Engineer etc)
• Experience with incident management and have good understanding of ITIL processes
• Familiarity with the core concepts of operating systems, networking, SDLC and Agile methodologies

• Good troubleshooting skills with triaging incidents in a high-capacity, high-availability and highly distributed environment 

• Experience with the following tools/platforms:
• Monitoring solutions eg: Datadog, NewRelic, Nagios, Elastic Search, Grafana
• Event management tools eg: BigPanda, Moogsoft
• ITIL-based Ticketing systems eg: ServiceNow, JIRA

Desired Qualifications:

• Computer Science/IT Systems/Information Technology degree or equivalent 
• Understand relational databases like MySQL, CI/CD pipelines, especially Jenkins
• Experience working on deployments in a live environment is a plus
• Experience working in container-based ecosystems like docker and with a container scheduler like Kubernetes, Amazon EKS/ECS or GKE
• AWS Cloud Services experience/certification/training or equivalent, Linux+, Network+, or equivalents
• Experience building automation scripts/utilities/jobs using either Python, Go, Powershell, JavaScript or Bash
• Familiarity with Site Reliability Engineering (SRE) principles and best practices

For this role, you'll find success through craft expertise, a collaborative spirit, and decision-making that prioritizes your fellow Rioters, who are the customers of your work. Being a dedicated fan of games is not necessary for this position!

Our Perks: 

We offer medical and dental plans that cover you, your spouse/domestic partner, and children. Life insurance, plus short-term and long-term disability coverage are also available. Riot will support your retirement benefits with access to a company pension, provide travel allowance, and double down on your donations of time and money to non-profit charitable organizations. Balance between work and personal life is encouraged with open paid time off, access to a spectrum of wellbeing activities and a play fund so you can broaden and deepen your personal relationship with games.

Let's Thrive Together:

Because together we are better

It's our policy to provide equal employment opportunities for all applicants and members of Riot Games, Inc. We know that fresh and varied perspectives will make us better at what we do, so however you identify and whatever background you bring with you, we’re excited to hear from you. Don’t be discouraged if you feel you don’t fully meet every single one of the requirements for a particular role, there’s always room for growth at Riot. If you spot a role that will make you want to jump out of bed in the morning, we are waiting to hear from you!