Aws basic account-level hardening

Cloudformation template to enable basic AWS account level security: Cloudtrail, AWS Config, Cloudwatch Alarms on security events, etc.

The very first thing you need to do while building your AWS infrastructure is to enable and configure all AWS account-level security features such as CloudTrail, CloudConfig, CloudWatch, IAM, etc. To do this, you can use my Amazon AWS Account level security checklist and how-to or any other source. To avoid manual steps and to be aligned with the SecuityAsCode concept, I suggest using a set of the CloudFormation templates which will provide the following functionality:

• configures CloudTrail according to the new best practices (KMS encryption, validation, etc)
• configures AWS Config service and creates a basic set of the CloudConfig rules to monitor best practices
• implements Section 3 (Monitoring) of the CIS Amazon Web Services Foundations Benchmark.

Launch template now: 

Global Security stack template structure:

security.global.yaml - parent template for all nested templates to link them together and control dependency between nested stacks.

cloudtrail.clobal.yaml - nested template for Global configuration of the CloudTrail

awsconfig.global.yaml - nested template for Global AWS Config Service configuration and config rules.

cloudwatchalarms.global.yaml - nested template for Global CloudWatch Logs alarms and security metrics creation. Uses FilterMap to create different security-related filters for ClouTrail LogGroup, corresponding metrics, and notifications for suspicious or dangerous events. You can customize a filter on a per-environment basis.

Input parameters:

• CFtemplateBucketURL: URL of the CloudFromation templates to use (Normally in the s3 bucket). This parameter will be prepopulated with value: https://s3.amazonaws.com/secureincloud.ca/aws/

• Bucket4Logs : Name of the new bucket that will be created to collect cloudtrail and config logs

• LogRetentionDays : Amount of days to store the logs in S3 bucket. Default 365 or 1 year

• AWSAccountName : AWS Account nickname(purpose). User-Friendly name(purpose) of your AWS account. Will be used in the names of the CloudWatch Alarms.

• InfosecEmail : Email of the infosec team to send security-related alerts from the CloudWatch Alerts

• DevOpsEmail : Email of the DevOps team to send operations-related alerts from the CloudWatch Alerts

AWS Managed Config Rules deployed by template:

• iam-user-no-policies-check Description: Checks that none of your IAM users have policies attached. IAM users must inherit permissions from IAM groups or roles
• root-account-mfa-enabled Description: Checks whether the root user of your AWS account requires multi-factor authentication for console sign-in.
• s3-bucket-public-read-prohibited Description: Checks that your S3 buckets do not allow public read access. If an S3 bucket policy or bucket ACL allows public read access, the bucket is noncompliant.
• s3-bucket-public-write-prohibited Description: Checks that your S3 buckets do not allow public write access. If an S3 bucket policy or bucket ACL allows public write access, the bucket is noncompliant.
• restricted-ssh Description: Checks whether security groups that are in use disallow unrestricted incoming SSH traffic.
• iam-password-policy Description: Checks whether the account password policy for IAM users meets the specified requirements.

AWS CIS Checks covered by the template (implemented via CloudWatch Alert mechanism ):

• AWS CIS 3.01 Ensure a log metric filter and alarm exist for unauthorized API calls
• AWS CIS 3.02 Ensure a log metric filter and alarm exist for Management Console sign-in without MFA
• AWS CIS 3.3 Ensure a log metric filter and alarm exist for usage of Root account
• AWS CIS 3.4 Ensure a log metric filter and alarm exist for IAM policy changes
• AWS CIS 3.5 Ensure a log metric filter and alarm exist for CloudTrail configuration changes
• AWS CIS 3.6 Ensure a log metric filter and alarm exist for AWS Management Console authentication failures
• AWS CIS 3.7 Ensure a log metric filter and alarm exist for disabling or scheduled deletion of customer-created CMKs
• AWS CIS 3.8 Ensure a log metric filter and alarm exist for S3 bucket policy changes
• AWS CIS 3.9 Ensure a log metric filter and alarm exist for AWS Config configuration changes
• AWS CIS 3.10 Ensure a log metric filter and alarm exist for security group changes
• AWS CIS 3.11 Ensure a log metric filter and alarm exist for changes to Network Access Control Lists (NACL)
• AWS CIS 3.12 Ensure a log metric filter and alarm exist for changes to network gateways
• AWS CIS 3.13 Ensure a log metric filter and alarm exist for route table changes
• AWS CIS 3.14 Ensure a log metric filter and alarm exist for VPC changes

Custom checks implemented via CloudWatch Alert mechanism:

• Custom: Alarms when a large number of sensitive (Start. Stop, Terminate, Reboot Instance) operations are performed in the short time period
• Custom: Alarms when a large number of Instances are being terminated
• Custom: Alarms when a volume is a force detached from an Instance
• Custom: Alarms when VPC traffic flow is created or deleted

Some important notes:

Many Cloud Security professionals might suggest that using CloudWatchAlarm as security detective control is a bit outdated. AWS has way more robust native mechanisms now like 10th of managed AWS  Config Rules, CIS, PCI, and AWS best practices standards (and associated checks) for the Security Hub. 

In addition, you can leverage nice 3d part tools like Splunk or Sumologic to have way more sophisticated detective controls.

This is true... But each of these mechanisms has significant extra costs associated. AWS Security becomes quite expensive when you leverage AWS Config Rules or Security Hub at scale. Cloudwatch Alarm on the contrary is quite cheap. 

More over CloudWatch Alarm-based security controls rely on the most robust, reliable, and fundamental AWS services and should be used as a 3d layer of your security defense to protect you in case of failure of 3d part or even more complex native AWS security mechanisms. 

Feel free to extend this list with your custom checks as per examples provided in the template and below:

```
rds-change:
  all: '{$.eventName = CopyDB* || $.eventName = CreateDB* || $.eventName = DeleteDB*}'

srt-instance:
  all: '{($.eventName = StopInstances || $.eventName = TerminateInstances || $.eventName
    = RebootInstances)}'

large-instance:
  all: >-
    { (($.eventName = RunInstances) || ($.eventName = StartInstances)) && (($.requestParameters.instanceType
    = *.2xlarge) || ($.requestParameters.instanceType = *.4xlarge) || ($.requestParameters.instanceType
    = *.8xlarge) || ($.requestParameters.instanceType = *.10xlarge)) }

change-critical-ebs:
  prod: >-
    {($.eventName = DetachVolume || $.eventName = AttachVolume || $.eventName
    = CreateVolume || $.eventName = DeleteVolume || $.eventName = EnableVolumeIO
    || $.eventName = ImportVolume || $.eventName = ModifyVolumeAttribute) && ($.requestParameters.volumeId
    = vol-youvol1ID || $.requestParameters.volumeId = vol-youvol2ID)}

create-delete-secgroup:
  all: >-
    {$.eventName = CreateSecurityGroup || $.eventName = CreateCacheSecurityGroup
    || $.eventName = CreateClusterSecurityGroup || $.eventName = CreateDBSecurityGroup
    || $.eventName = DeleteSecurityGroup || $.eventName = DeleteCacheSecurityGroup
    || $.eventName = DeleteClusterSecurityGroup ||  $.eventName = DeleteDBSecurityGroup}

secgroup-instance:
  all: '{$.eventName = ModifyInstanceAttribute && $.requestParameters.groupSet.items[0].groupId
    = * }'

cloudformation-change:
  all: '{$.eventSource = cloudformation.amazonaws.com && ($.eventName != Validate*
    && $.eventName != Describe* && $.eventName != List* && $.eventName != Get*)}'

critical-instance:
  prod: >-
    {$.requestParameters.instanceId = i-instance1ID || $.requestParameters.instanceId
    = i-instance2ID || $.requestParameters.instanceId = i-instance3ID || $.requestParameters.instanceId
    = i-instance4ID || $.requestParameters.instanceId = i-instance5ID || $.requestParameters.instanceId
    = i-instance6ID|| $.requestParameters.instanceId = i-instance7ID}

eip-change:
  all: '{$.eventName = AssociateAddress || $.eventName = DisassociateAddress ||
    $.eventName = MoveAddressToVpc || $.eventName = ReleaseAddress }'

net-access
all: >-
    {$.sourceIPAddress != 111.222.3* && $.sourceIPAddress != 111.222.4* && $.sourceIPAddress
    != cloud* && $.sourceIPAddress != AWS* && $.sourceIPAddress != 11.22.33.00
    && $.sourceIPAddress != 11.22.33.01 }

```

Lack of the spelling checks for the AWS IAM API actions and security implications

 AWS IAM policy language used everywhere:

- to define IAM policy itself

- to define resource-based policy like S3 bucket policy

- to define the most important AWS control - SCP (Service Control Policy)

- to define VPC endpoint policies 

Let's take a look at the AWS IAM policy structure:

This policy's vital element is "Action," which is a list of AWS APIs that will be Allowed or Denied by the policy. 

Currently, there are several thousands of AWS actions. A list of all of them could be found here.  

It's extremely easy to make a typo in the action name when you are creating a policy. 

But, AWS will detect and warn you, right? 

Nope!

It might come as a surprise even for the experienced cloud engineers, but AWS does not verify API actions spelling.  

Proof: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_policy-validator.html

Yes, when you are using a policy generator, you can choose from the dropdown list of the available API actions.  

But if you are using CLI, Cloudformation, Terraform, or any SDK, your policy will be accepted as long as a policy syntax and grammar will pass (policy grammar, but not action names or resource ARNs )

So what? Not a big deal? If policy not working, you can troubleshoot it using the policy simulator, right, and find a problem? 

The challenge is that even an AWS native policy simulator will not check API (action names) spelling. It will show if the desired actions are allowed or blocked but will not point you to the simple typo in your policy. 

As long as your policy is for the ALLOW effect, it's not a big deal. You might spend some time troubleshooting and not understanding why access is not granted, but generally, it should be OK, right?

Even in the case of ALLOW, not precisely: 

when you are using Infra as a code, you might make a typo in production-related IAM/SCP/etc. policy and cause quite an outage!

What about policies that suppose to protect, a.k.a DENY effect? Implications, in this case, might be quite catastrophic:

- SCP that implements your AWS Account level preventative controls will allow actions that you think you have blocked, making controls not-exiting.

- IAM policy will not protect against destructive or unsafe actions.

- Resource-based policy might become unintentionally too open.

Luckily, AWS has a second layer of protection ( and implicitDeny ) that, up to a certain extent, will compensate for such mistakes: as long as API call (Action name) is not explicitly Allowed, it will be deemed as implicitDeny. This helps and might save your backend, but not in all cases. Moreover, relying on this it's definitely a bad security practice. 

What could be a solution to the problem we just discussed? 

A process and a tool of the IAM policy validation for the syntax and spelling.

The process is an IAM policy linting that must be done before any deployment or during PR review in your code repo.

As for the tool, it might be home built linting tool (example: extension or rule to the CloudFormation Linter  https://github.com/aws-cloudformation/cfn-python-lint ) or an open-source linting tool that performs IAM action names validation. (example:  https://github.com/duo-labs/parliament)

Note, the tool must be regularly and automatically synced with the latest list of the AWS IAM actions or manually updated to reflect any changes AWS might do to the subj. 

Deployment time security audit using CloudFormation custom resource.

To prevent deployment of the potentially sensitive resources or infrastructure into the AWS account that might not meet current organizational security standard we can use AWS CloudFormation custom resource to perform quick security audit (or kind of sanity check) of the cloud account before processing with deployment.

Why we need this if we can scan/audit account as a part of the CI/CD pipeline? For the cases when deployments are performed manually or to have CI/CD independent "portable" CloudFormation template that has all security checks built-in and not bolt-on.

How it will look like:

1. To you normal CloudFormation template you will add a custom resource.
2. This custom resource it technically speaking a Lambda function that created and called during CloudFormation stack deployment.
3. This Lambda function will perform quick (to meet CloudFormation deployment timeouts restrictions) security audit of the account where template going to be deployed
4. As result of this audit Lambda will return status that will be interpreted by CloudFormation as a resource creation outcome.
5. If AWS environment pass security check - deployment of other resources in you stack continue as usual
6. If AWS environment fail security check - stack deployment will interrupted and rolled back as a result of the custom resource failure. 

I will publish example on such functionality on my Github shorty and will update this post with more details. 

Secure your AWS account using Terrafrom and CloudFormation

This is very updated version of the blog post: http://blog.it-security.ca/2016/11/secure-your-aws-account-using.html

As I mention before:
The very first thing you need to do while building your AWS infrastructure is to enable and configure all AWS account level security features such as: CloudTrail, CloudConfig, CloudWatch, IAM, etc.

Time flies when you're having fun and flies even faster in the infosec world. My templates become outdated and now I'm presenting an updated version of the AWS security automation with following new features:

1. integrated with Terraform (use terraform templates in the folder tf)
2. creates prerequisites for Splunk integration (User, key, SNS, and SQS)
3. configures cross-account access (for multiaccount organizations, adding ITOrganizationAccountAccessRole with MFA enforced)
4. implements Section 3 (Monitoring) of the CIS Amazon Web Services Foundations benchmark.
5. configures CloudTrail according to the new best practices (KMS encryption, validation etc)
6. configures basic set of the CloudConfig rules to monitor best practices

First, my security framework now consists of two main parts: cf (CloudFormation) and tf (Terraform) with Terraform template as a bootstrapper of the  whole deployment.

You can use  Terraform, you can use CloudFormation, but why both ?
Terraform is very quickly evolves, has cross-cloud support and implements some missing in CloudFormation features (like account level password policy configuration, etc); CloudFormation is native for AWS, well supported, and, most important, AWS provides a lot of best practices and solutions in the form of the CloudFormation templates.

Using both (tf and cf) gives me (and you) ability to reuse solutions, suggested and provided by AWS, without rewriting the code, have flexibility and power of terraform and one single interface for whole cloud automation.
No more bucket pre-creation or specific sequence of the CloudFormation deployment - just terraform apply. It will take care of all CloudFormation prerequisites, version control and template updates.
But,  if you wish, at current state you can use only my CloudFormation templates - cf still does all heavy lifting.

The main trick of the Terraform - CloudFormation integration was to tell terrafrom when CloudFormation template is updated to ensure that terraform will trigger cf stack update.
I achieved this using S3 bucket with version control enabled and always updating (just setting template version) security.global.yaml.

This code takes care of Terraform and CloudFormation integration:
# creating Security cloudforation stack

resource "aws_cloudformation_stack" "Security" {
  name = "Security"
  depends_on = ["aws_s3_bucket_object.iam_global", "aws_s3_bucket_object.cloudtrailalarms_global", "aws_s3_bucket_object.awsconfig_global", "aws_s3_bucket_object.cloudtrail_global", "aws_s3_bucket_object.security_global"]
  parameters {
    AccountNickname = "${var.enviroment_name}",
    CompanyName = "${var.company_name}",
    MasterAccount = "${var.master_account}"
  }
  template_url = "https://s3.amazonaws.com/${aws_s3_bucket.CFbucket.bucket}/${var.security_global}?versionId=${aws_s3_bucket_object.security_global.version_id}"
  capabilities = [ "CAPABILITY_NAMED_IAM" ]
  tags { "owner" = "infosec"}
}

And finally deployment steps are:

1. Get code from my git repo:  https://github.com/IhorKravchuk/it-security
2. Switch to tf folder and update terraform.tfvars specifying: your AWS profile name (configured for aws cli using aws configure --profile profile_name); name for the environment (prod, test, dev ..) ; company(or division) name; region and AWS master account ID.
3. terraform init to get aws provider downloaded by terraform
4. terraform plan
5. terraform apply

Secure your AWS account using CloudFormation

      The very first thing you need to do while building your AWS infrastructure is to enable and configure all AWS account level security features such as: CloudTrail, CloudConfig, CloudWatch, IAM, etc..

       To do this, you can use mine Amazon AWS Account level security checklist and how-to or any other source.

        To avoid manual steps and to be align with SecuityAsCode concept, I use set of CloudFormation templates, simplified version of which I would like to share: 

GitHub: https://github.com/IhorKravchuk/it-security

Global Security stack template structure:

security.global.json - parent template for all nested templates to link them together and control dependency between nested stacks.

cloudtrail.clobal.json - nested template for Global configuration of the CloudTrail:

• creates S3 bucket for the logs
• creates CloudTrail-related IAM roles and policies
• creates CloudLog LogGroup
• enables CloudTrail on default region including global events and multi-region feature
• creates SNS ans SQS configuration for easy integration with Splunk AWS app.

cloudtrailalarms.global.json - nested template for Global CloudWatch Logs alarms and security metrics creation. Uses FilterMap to create different security-related filters for ClouTrail LogGroup, corresponding metrics and notifications for suspicious or dangerous events. You can customise filter per environment basis.

Predefined filters are:

• rds-change: RDS related changes
• iam-change; IAM changes
• srt-instance: Start, Reboot, Terminate instance
• large-instance: launching large instances
• massive-operations: massive operations- more 10 in 5 min 
• massive-terminations: massive terminations- more 10 in 5 min 
• detach-force-ebs: force detachment of the EBS volume from the instance
• change-critical-ebs: any changes related to the critical EBS volumes
• change-secgroup: any changes related to the security group
• create-delete-secgroup: creation and deletion of the security group 
• secgroup-instance: attaching security group to the instance
• route-change: routing  changes
• create-delete-vpc: creation and deletion of a VPC
• netacl-change: changes at Network ACL
• cloudtrail-change: changes in the CloudTrail configuration
• cloudformation-change: changes related to the CloudFormation
• root-access: any root access events
• unauthorised: failed and unauthorised operations
• igw-change: Internet Gateway related changes
• vpc-flow-logs: Delete or Create VPC flow logs
• critical-instance: any operation on the critical instances
• eip-change: Elastic IP changes
• net-access: Any access outside of predefined known IP ranges

4 preconfigured notification topics : 

• InfosecEmailTopic, 
• DevOpsEmailTopic
• InfosecSMSTopic
• DevOpsSMSTopic

awsconfig.global.json - nested template for Global AWS Config Service configuration.

• creates S3 bucket for the config dumps
• creates AWS Config-related IAM roles and policies
• creates AWS config delivery channel and schedule config dumps (hourly)
• creates and enables AWS config recorder 
• creates SNS ans SQS configuration for easy integration with Splunk AWS app.

cloudwatchsubs.global.json - nested template for configuring AWS CloudWatch Subscription Filter to extract and analyse most severe CloudTrail events using custom Lambda function:

• creates Lambda function and all requred roles and permissions
• creates Subscription filter as a compilation of the filters from the FilterMap
    

Currently uses following filters and aggregate them to one due to the AWS CloudWatch Logs subscription limitation (only one filter supported):

• critical-instance
• iam-change
• srt-instance
• cloudtrail-change
• root-access
• net-access
• detach-force-ebs
• unauthorised

iam.global.json - nested template for IAM Global configuration: 

• creates Infosec Team IAM Group and managed policy
• creates DevOps Team IAM Group and managed policy
• creates DBA Team IAM Group and managed policy
• creates Self Service Policy  for users to manage API keys and MFA
• creates  ProtectProdEnviroment to protect production environment from destructive actions 
• creates EnforceMFAPolicy to enforce MFA for sensitive operations
• creates EnforceAccessFromOfficePolicy to restrict some operation to office source IPs
• creates DomainJoin role and all required policy to perform automated domain join
• creates SaltMasterPolicy and Role for Configuration Management Tool (in this case - Salt)
• creates SQLDataBaseInstancePolicy and Instance profile example policy
• creates SIEM system example policy
• creates VPC flow log role
• creates and manages SIEM user and API keys
• creates and manages SMTP user (for the AWS SES service ) and API keys

cloudwatchsubs_kinesis.global.json - PoC template (not linked as nested to the security.global.json)  for configuring AWS CloudWatch Subscription Filter to send most severe CloudTrail events to AWS Kinesis stream using subscription filter similar to the cloudwatchsubs.global.json 

Supported features:

Environments and regions: Stack supports unlimited amount of environments with 4 environments predefined (staging, dev, prod, and dr) and use 1 account and 1 region per environment concept to reduce blast radius (if account become compromised)

AWS services used by stack: CloudTrail, AWS Config, CloudWatch, CloudWatch Logs and Events, IAM,  Lambda, Kinesis.

To deploy:

1. Create bucket using following naming convention: com.ChangeMe.EnviromentName.cloudform, replacing ChangeMe and EnviromentName with your value to make it look like this:            com.it-security.prod.cloudform
2. Enable bucket versioning 
3. in the templates  security.global.json and cloudwatchsubs.global.json replace "ChangeMe" with name used in the bucket creation.
4. In the template cloudtrailalarms.global.json modify SNS endpoints for email notification infosec@ChangeMe.com and devops@ChangeMe.com; Add endpoints with mobile phone numbers for SMS notification to appropriate SNS topics if needed.
5. Modify iam.global.json template to adrress you SQL DataBase bucket location (com-ChangeMe-", {"Ref": "Environment"} , "-sqldb/)  and modify any permission if need according to your organisation structure, roles, responsibilities and services.
6. Modify FilterMap in cloudtrailalarms.global.json and cloudwatchsubs.global.json templates make filters work for your infrastructure (Critical Instance IDs, Critical Volume IDs, you ofiice IP range, you NAT gateways, etc) 
7. Zip example Lambda function LogCritical_lambda_security_global.py like LogCritical_lambda_security_global.zip
8. Upload this function into S3 bucket created at step 1 and copy object version (GUI- show version -object properties ) and insert into cloudwatchsubs.global.json template into "LogCriticalLambdaCodeVer" mapping at the appropriate environment (prod, staging ..)
9. Modify "regions" Environments mapping in the iam.global.json, and cloudwatchsubs.global.json templates to specify correct AWS region you are using for the deployment.
10. Upload all *.global.json templates into S3 bucket created at step 1. 
11. Create new CloudFormation stack  using parent security template security.global.json and your bucket name (Example: ttps://s3.amazonaws.com/com.it-security.prod.cloudform/security.global.json ),  call it "Security" and specify environment name you going to deploy.
12. Done!

AWS CloudFormation template security group viewer

        Almost any AWS CloudFormation template are more then long enough. It's OK when you are dealing with different relatively "static" resources but become a big  problem for something way more dynamic like security group.
    This kind of resource you need to modify and review a lot, especially if you cloud security professional.  Reading AWS CloudFromation template JSON manually  makes your life miserable and you can easily miss bunch of security problems and holes.
     My small aws_secgroup_viewer Python program helps you to quickly review and analyse all security groups in your template.

     https://github.com/IhorKravchuk/it-security/blob/master/aws_secgroup_viewer.py

     Supports both security group notations used by CloudFormation: firewall rules inside security group or as separate resources linked to group.