AWS DevOps Competency

AWS Partner has a company overview presentation that sets the stage for customer conversations about their AWS DevOps capabilities and showcases AWS Partner’s demonstration capabilities.

Presentation contains information about the AWS Partner’s AWS DevOps capabilities, including AWS specific differentiators, e.g., what is unique about the AWS Partner’s practice that can only be accomplished leveraging AWS.

Overview presentations contain:

• Company history
• Office locations
• Number of employees
• Customer profile, including number, size, and industries of customers
• Overview of DevOps practice
• Notable AWS projects

Please provide the following as evidence:

• Delivery of presentation by a business development executive at the beginning of the validation session. This should be limited to 15 minutes.

AWS Partner has internal mechanisms for maintaining their consultants' expertise on DevOps-related AWS services and tools.

Please provide the following as evidence:

• List of internal and/or external AWS-focused education events lead by AWS Partner staff (e.g. formal training, lunch and learns, meetups, user groups, etc.) in last 12 months.
• Resources provided by AWS Partner to staff for ongoing AWS skills development

AWS Partner must describe how DevOps opportunities are identified, how their sellers are trained to identify and sell those opportunities, and specific demand generation/lead generation efforts associated to their AWS DevOps practice.

Please provide the following as evidence:

• A description on how the AWS Partner engages with customers, their internal sellers, and AWS sellers if applicable.

AWS Partner must describe how and when they engage with AWS sellers and AWS Solutions Architects.

Please provide the following as evidence:

• A verbal description for how and when they engage AWS sellers or AWS Solutions Architects on an opportunity or in the form of a demonstration of the AWS Opportunity Management tool in AWS Partner Central with sales qualified opportunities submitted (sales qualified = budget, authority, need, timeline, and competition fields completed).

AWS Partner must have a process to ensure that there are sufficient DevOps trained personnel to effectively support customers.

Please provide the following as evidence:

• An established training plan including on-boarding processes that identify job roles (sellers, solutions architects, project managers) and required training paths
• A verbal description of methods used to allocate required resources to DevOps projects

AWS Partner has processes for working with customers to determine and define expected outcomes associated with the projects.

Please provide the following as evidence:

• Project deliverable templates or other resources used for project scoping and definition

AWS Partner has processes to determine scope of work with specific criteria defining customer project with expected deliverables.

Please provide the following as evidence:

• Project templates or other resources(e.g. RACI Matrix) used for project scoping and definition

AWS Partner has standard Statement of Work (SOW) templates for DevOps projects that can be customized to customer needs.

Please provide the following as evidence:

• Default SOW template

AWS Partner assigns Project Manager to each project to ensure project remains on time and within budget.

Please provide the following as evidence:

• Documentation to show that Project Managers were assigned to each of the 4 customer example projects.

AWS Partner has processes to document, manage, and respond to requests for changes to the project scope.

Please provide the following as evidence:

• Documentation of change management practices

AWS Partner has a customer acceptance process.

Please provide the following as evidence:

• Example customer training documents
• SOW language describing handoff responsibilities and acceptance criteria

AWS Partner implements customer satisfaction checkpoints as part of the project plan.

Please provide the following as evidence:

• Project plan and customer satisfaction results for milestone-defined checkpoints

AWS Partner is aware and has read the DevOps Use case fit and technical calibration guide.

AWS Partner evaluates customer’s internal organizational structure as basis for providing organizational recommendations to move to DevOps approaches and methodologies.

Evidence must be in the form of documented approach to evaluate internal organizational structures as part of project risk assessment.

AWS Partner has a methodology and processes to recommend organizational changes, including identification of plans for various departments (Legal, Finance, HR, and Sales) and addressing the importance of executive sponsorship.

Evidence must be in the form of organization change recommendations provided to customers and description of outcomes of those projects.

Developers and systems administrators should have an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion.

AWS Partner has methodologies leveraging templated infrastructure provisioning for repeatable deployments. This can include using AWS CloudFormation or leveraging a Technology Partner solution to provision, secure, connect, and run solution infrastructure.

Evidence must be in the form of CloudFormation templates or Technology Partner equivalent leveraged for the four (4) submitted customer case studies with description as to how they communicate the need and use of these solutions for repeatable deployments.

AWS Partner has process to automatically collect software inventory, apply OS patches, create system images, and configure Windows and Linux operating systems. If EC2 Systems manager is not leveraged, Partner can show documented processes and provide technical demonstration for how configurations and infrastructure updates are managed, and the methods used to automate these functions

Evidence must be in the form of technology demonstration and process documentation provided to customer as part of their DevOps conversion.

Partner has processes and methodologies to conduct AWS resource inventory, configuration history, and configuration change notifications to enable security and governance. If AWS Config is not used, AWS Partner can show documented processes and provide technical demonstration of how policies are managed, and the methods used to automate these functions.

Evidence must be in the form of technology demonstration and process documentation provided to customer as part of their DevOps conversion.

AWS Partner has processes to build, test, and deploy code every time there is a code change, based on pre-defined release process models. AWS Partner may leverage AWS CodePipeline or equivalent.

Evidence must be in the form of process documentation describing software release process and examples of how AWS Partner teaches customer how to build software release models. must be in the form of process documentation describing software release process and examples of how AWS Partner teaches customer how to build software release models.

AWS Partner has processes to compile source code, run tests, and produce software packages that are ready to deploy. As part of this process, AWS Partner has processes that support provisioning, managing, and scaling servers according to needs. AWS Partners may leverage AWS CodeBuild or equivalent.

Evidence must be in the form of process documentation describing development and testing processes and examples of how AWS Partner teaches customer how to accomplish the same.

Evidence must be in the form of demonstration and/or detailed description of how AWS Partner sets up source control systems for customers transforming to DevOps approaches

AWS Partner has evaluated and has methodologies to build and deploy microservices architectures leveraging containers or serverless computing.

Evidence must be in the form of documentation of AWS Partner’s approach and philosophy leveraging serverless computing and/or containers with customer examples if/where applicable.

AWS Partner has methodologies leveraging essential cloud and network monitoring services such as Amazon CloudWatch.

Evidence must be in the form of established monitoring, customer metrics, logs, alarms, and dashboards with description for how/why these items are recommended to customers transforming to DevOps approaches.

AWS Partner has methodologies leveraging AWS X-Ray to Analyze and debug production, distributed applications.

Evidence must be in the form of detailed description of use with at least one customer example. If AWS X-Ray is not leveraged, AWS Partner can describe appropriate use cases and alternative ways this is accomplished

AWS Partner has methodologies for API and Activity usage tracking via AWS CloudTrail to record AWS API calls and deliver log files.

Evidence must be in the form of two (2) of four (4) customer examples with description for how/why these items are recommended to customers transforming to DevOps approaches.

AWS Partner has deep understanding of and leverages AWS Elastic Beanstalk as an orchestration platform to handle deployments from capacity provisioning, load balancing, auto-scaling to application health monitoring. If an alternate such as Heroku is leveraged, AWS Partner to describe reasons for leveraging alternative(s).

Evidence must be in the form of demonstration and description of PaaS as part of AWS Partner’s DevOps strategy.

AWS Partner ensures customers understand AWS security processes and technologies as outlined here.

Evidence must be in the form of onboarding and educational documents provided to customers that specifically cover customer security considerations in the AWS Partner’s environment. In addition, evidence must be provided that the AWS Partner helps encourage “shift-left” of security practices, i.e., incorporating security guidelines and best practices early in development and as an ongoing part of the development and operations process.

AWS Partner must submit architecture diagrams depicting the overall design and deployment of its AWS Partner solution on AWS as well as any other relevant details of the solution for the specific customer in question.

The submitted diagrams are intended to provide context to the AWS Solutions Architect conducting the Technical Validation. It is critical to provide clear diagrams with an appropriate level of detail that enable the AWS Solutions Architect to validate the other requirements listed below.

Each architecture diagram must show:

• All of the AWS services used
• How the AWS services are deployed, including virtual private clouds (VPCs), availability zones, subnets, and connections to systems outside of AWS.
• Elements deployed outside of AWS, e.g. on-premises components, or hardware devices.
• how design scales automatically - Solution adapts to changes in demand. The architecture uses services that automatically scale such as Amazon S3, Amazon CloudFront, AWS Auto Scaling, and AWS Lambda.
• how design has high availability with multi-AZ or multi-region deployment. When intentional tradeoffs have been made (e.g. to optimize cost in favor of high availability), please explain the customer's requirements.

Please provide the following as evidence (required for all provided customer examples):

• An architecture diagram depicting the overall design and deployment of your solution on AWS.
• Explanation of how the major solutions elements will keep running in case of failure.
• Description of how the major solutions elements scale up automatically.

AWS expects all Services Partners to be prepared to create AWS accounts and implement basic security best practices. Even if most of your customer engagements do not require this, you should be prepared in the event you work with a customer who needs you to create new accounts for them.

Establish internal processes regarding how to create AWS accounts on behalf of customers when needed, including:

• When to use root account for workload activities
• Enable MFA on root
• Set the contact information to corporate email address or phone number
• Enable CloudTrail logs in all region and protect CloudTrail logs from accidental deletion with a dedicated S3 bucket

Please provide the following as evidence:

• Documents describing Security engagement SOPs which met all the 4 criteria defined above. Acceptable evidence types are security training documents, internal wikis, or standard operating procedures documents.
• Description of how Secure AWS Account Governance is implemented in one (1) of the submitted customer examples.

Define standard approach to access customer-owned AWS accounts, including:

• Both AWS Management Console access and programmatic access using the AWS Command Line Interface or other custom tools.
• When and how to use temporary credentials such as IAM roles
• Leverage customer's existing enterprise user identities and their credentials to access AWS services through Identity Federation or migrating to AWS Managed Active Directory

Establish best practices around AWS Identity and Access Management (IAM) and other identity and access management systems, including:

• IAM principals are only granted the minimum privileges necessary. Wildcards in Action and Resource elements should be avoided as much as possible.
• Every AWS Partner individual who accesses an AWS account must do so using dedicated credentials

Please provide the following as evidence:

• Security engagement Standard Operation Procedure (SOP) which met all the 2 criteria defined above. Acceptable evidence types are: security training documents, internal wikis, standard operating procedures documents. Written descriptions in the self-assessment excel is not acceptable.
• Description of how IAM best practices are implemented in one (1) of the submitted customer examples.

AWS Partner has defined metrics for determining the health of each component of the workload and provided the customer with guidance on how to detect operational events based on these metrics.

Establish the capability to run, monitor and improve operational procedure by:

• Defining, collecting and analyzing workload health metrics w/AWS services or 3rd Party tool
• Exporting standard application logs that capture errors and aid in troubleshooting and response to operational events.
• Defining threshold of operational metrics to generate alert for any issues

Please provide the following as evidence:

• Standardized documents or guidance on how to develop customer workload health KPIs with the three components above
• Description of how workload health KPIs are implemented in (1) of the submitted customer examples.

Create a runbook to document routine activities and guide issue resolution process with a list of operational tasks and troubleshooting scenarios covered that specifically addresses the KPI metrics defined in OPE-001.

Please provide the following as evidence:

• Standardized documents or runbook met the criteria defined above.

Deployments are tested or otherwise validated before being applied to the production environment. For example, DevOps pipelines used for the project for provisioning resources or releasing software and applications.

Use a consistent approach to deploy to customers including:

• A well-defined testing process before launching in production environment
• Automated testing components

Please provide the following as evidence:

• A deployment checklist example or written descriptions met all the criteria defined above.

Establish internal processes regarding how to secure traffic within VPC, including:

• Security Groups to restrict traffic between Internet and Amazon VPC
• Security Groups to restrict traffic within the Amazon VPC
• Network ACL to restrict inbound and outbound traffic
• Other AWS security services to protect network security

Please provide the following as evidence:

• Written descriptions/documents on network security best practices met the criteria defined above.
• Description of how network security is implementation in one (1) of the submitted customer examples.

Establish internal processes regarding a data encryption policy used across all customer projects

• Summary of any endpoints exposed to the Internet and how traffic is encrypted
• Summary of processes that make requests to external endpoints over the Internet and how traffic is encrypted
• Enforcing encryption at rest. By default you should enable the native encryption features in an AWS service that stores data unless there is a reason not to.

All cryptographic keys are stored and managed using a dedicated key management solution

Please provide the following as evidence:

• Data encryption and key management policy met the criteria defined above.
• Description of how data encryption is implementation in one (1) of the submitted customer examples.

Changes to infrastructure are automated for customer implementation

• Tools like AWS CloudFormation, the AWS CLI, or other scripting tools were used for automation.
• Changes to the production environment were not done using the AWS Management Console.

Please provide the following as evidence:

• Written description of deployment automation and an example template (e.g., CloudFormation templates, architecture diagram for CI/CD pipeline) met the criteria defined above.

Incorporate resilience discussion and advise a RTO&PRO target when engaging with customer. Customer acceptance and adoption on RTO/RPO is not required.

• Establish a process to establish workload resilience including:
• RTO & RPO target
• Explanation of the recovery process for the core components of the architecture
• Customer awareness and communication on this topic

Please provide the following as evidence:

• Descriptions or documents on workload resilience guidance met the three criteria defined above
• Description of how resilience is implementation in one (1) of the submitted customer examples including reasons for exception when RTO&RPO is not defined

Determine solution costs using right sizing and right pricing for both technical and business justification.

Conducted TCO analysis or other form of cost modelling to provide the customer with an understanding of the ongoing costs including all the following 3 areas:

• Description of the inputs used to estimate the cost of the solution
• Summary of the estimates or cost model provided to the customer before implementation
• Business value analysis or value stream mapping of AWS solution

Please provide the following as evidence:

• Description of how to develop cost analysis or modeling with the critical components defined above
• Cost analysis example in one (1) of the submitted customer examples. Acceptable evidence types are: price calculator link, reports or presentations on business values analysis

The DevOps Competency is intended to identify and promote AWS Partners who help customers transform their software development and infrastructure management practices to deliver high quality applications and services at high velocity. As a result, the following are guidelines for valid DevOps projects. All customer examples must meet these criteria:

• We are looking for customer use cases that involve transformation from traditional hardware/software development to AWS Cloud-based Infrastructure as Code leveraging AWS design best practices for AWS Services.
• We are looking for use cases that involve business transformation in addition to technical process transformation.
• 3 rd party tools for Configuration Management, Container Services, and CI/CD are acceptable, but the solution must leverage AWS compute, storage, or network services and essential cloud management toolsets such as AWS CloudWatch for resource monitoring and AWS CloudTrail for logging in conjunction with those services to qualify for this program. (In other words, AWS has to play a key role either in DevOps specific services leveraged (AWS CodePipeline, AWS CodeBuild, AWS CodeDeploy, AWS CodeStar, AWS CodeCommit, Amazon Elastic Container Service, AWS Lambda, AWS CloudFormation, AWSOpsWorks, AWS Config, Amazon EC2 Systems Manager, AWS Elastic Beanstalk, AWS X-Ray) OR solution must leverage AWS Infrastructure services with like applicability of AWS DevOps services via 3rd Party toolsets approved in the Technology Category of the AWS DevOps Competency).

Customer examples must describe customer impact as a result of the move from traditional models to a DevOps approach leveraging AWS.