Firewall rule logging

Problem:
     When firewall is managed by NOC team and you are part of infosec it's really hard to maintain reasonable log level at firewall rules.
    Even If decision of log do not log this rule is part  of the change request process and made by infosec you have a risk of having to much logs, having duplicated logs of the same event form different firewalls or spending more time on each request to avoid problem mentioned.
     In all other cases you definitely end up having too much logs that could even overload and slow down you FW device (for me it happened on opendbsd pf firewall when syslog had consumed all memory ).
  Sure thing, you can optimize fw rules and reduce amount of logs - but  it's thankless time consuming process especially if you have many FW devices.
 
 Solution:
    Create special fw rules just for the log purpose. They will be at the begin of the rule list and identical on all your FW devices.

Result:
Logging rules that:

• easy to create (based on network topology and security zones )
• easy to check
• easy to tune 
• easy to distribute and install

Example:

For PF firewall those rules will look like: 

match log inet proto tcp from 145.23.56.15 to 10.156.25.15 port 80

Virtual machines templates. Static or Live?

Virtualization is everywhere. Need a new VM ? - it's easy,  clone from template and spin it up!

But let's take a close look at the classical "static" templates security cons:

- Static disk image file 

- get updated from time to time ( in most cases not often)

- security policies (or hardening) on it updated only after deployment 

- not being monitored by any part of your security monitoring infrastructure (no events coming, no AV or vulnerabilities  scans,  etc.)

Templates are the key component of your infrastructure - you are cloning all you running machines from them.

To keep templates on the same security level you need: spin up a vm from this template, update it, check stability after updates, scan it for vulnerabilities and re-template back (and do not forget to update checksums. You are checking templates' checksums before deployment, aren't you? :-)   ). 

IMHO, the biggest security issue with "static" templates - is the time frame when your recent cloned from template machine boot up and  get updates  being at the same time not fully protected and up to day.  

Instead of using "static" templates, you can use "live" templates - just normal running VMs  (CPU and memory are really cheap now). So ,what the pros:

- part of you live infrastructure

- get updates as soon as updates are issued or/and approved
- get current version of the security(or hardening) policy implemented
- monitored and covered by your security infrastructure(AV scanners, HIDS/NIDS, log management)
- monitored for up time and stability after each update

Script, shown below, being placed in cron.daily will keep your RH based linux  template up to day (for VMware virtualization)

#!/bin/sh

# Doing self-update if we are running on host with template in name and self-destruction on other hosts 

if [[ "$HOSTNAME" == y??0?centos?tmpl* ]]; 
then /usr/bin/yum update -y > /var/log/self_update.log 2>&1
else
/bin/rm -v /etc/cron.daily/self_update > /var/log/self_update.log 2>&1 
fi

# Comparing installed and running kernel and reboot in case if they differ 

LAST_KERNEL=$(/bin/rpm -q --last kernel | /usr/bin/perl -pe 's/^kernel-(\S+).*/$1/' | /usr/bin/head -1)
CURRENT_KERNEL=$(/bin/uname -r)
if [ $LAST_KERNEL != $CURRENT_KERNEL ];
then 
/bin/echo '/usr/bin/vmware-config-tools.pl -d ; /bin/sed -i "/.*vmware.*/d"  /etc/rc.local'  >> /etc/rc.local
/sbin/reboot; 
fi
exit 0

PS. Script also address annoying problem which requires rerun vmware configuration utility after each kernel update to make vmware tools run .
PS2 And self-destruct on non-template hosts.

Shame on you, Actividentity!

Looking for ActivIdentety smart cards, google them and first link pointing to the web site in Drupal install mode

whois activcard.com

Registrant:

Actividentity Corporation

   6623 Dumbarton Circle

   Fremont, CA 94555

   US

   Domain Name: ACTIVCARD.COM

CloudStack IaaS insecure password reset

CloudStack is one of the major and popular IaaS  (Infrastructure as a Servile)  platform.
http://en.wikipedia.org/wiki/Apache_CloudStack

Below small review of the password reset process in the CloudStack.

The purpose of password reset procedure - to allow user during deployment of the VM (virtual machine) template or after this to reset root (administrator password) of VM. Because of the main idea of IaaS to give user ability to help himself  this is one of the key functionality.

From user perspective process looks like:

1. start new VM or click reset password on any stopped VM
2. Get popup with new root password
3. Log in using console, rdp or ssh using new password.

Let's see what behind the scene:

Each network in CloudStack has dedicated router (VR) which doing dhcp, dns, loadbalancing, firewalling and password reset for whole subnet.

On VR we have following components of password reset service:

1. Process listening on port 8080:
 socat -lf /var/log/cloud.log TCP4-LISTEN:8080,reuseaddr,crnl,bind=10.0.146.2 SYSTEM:/opt/cloud/bin/serve_password.sh "$SOCAT_PEERADDR"
it actually waiting for request like : DomU_Request: send_my_password

2. script actually doing the job: /opt/cloud/bin/serve_password.sh

3. and password file: /var/cache/cloud/passwords having all passwords in clear text with filesystem permissions -rw-r--r--
  10.0.146.15=rD7nudcze
 10.0.146.13=jB9kbknvq
 10.0.146.181=saved_password

after each password request VR replace corresponding password in password file  by "saved_password"

On VM template and  VM instance you have script: /etc/init.d/cloud-set-guest-password
This script automatically request root password from VR at each system startup and update it .

The password request procedure is:

1. Client VM parse local  network setting and getting DHCP server IP.

2. Client send clear text  request like wget -q -t 3 -T 20 -O - --header "DomU_Request: send_my_password" $PASSWORD_SERVER_IP:8080

3. If it get password it will use it. If it gets "saved_password" it won't do anything.

Security problems:

1. clear text password storage on VR

2. Clear text password transmission over the network

3. Missing password sever authentication (only by IP)

4. auto-starting password reset service.

Conclusion:

If attacker has access to the one instance into cloudstack network by spoofing password server(VR) IP he will able to compromise other instances int this subnet after their reboot. Having access to VR - will be able to compromise all nodes.

Adding security into Enterprise Linux system-wide authentication mechanism:

Some small modification to default authentication mechanism at RedHat 5/6 based system will increase system security.

First witch from md5 to sha512 hashes: it as easy as executing:

"authconfig--passalgo=sha512 --updateall"

Update your /etc/pam.d/system-auth to the more adequate to the current threats settings:

#vi /etc/pam.d/system-auth

#%PAM-1.0
# User changes will be destroyed the next time authconfig is run.

auth        required      pam_env.so
auth        required      pam_unix.so nullok try_first_pass
auth        sufficient    pam_tally2.so deny=10 onerr=fail unlock_time=1200
# lock account after 10 failed attempt. unlock automatically after 20 minutes.
auth        required      pam_deny.so

account     required      pam_tally2.so
account     required      pam_unix.so
account     sufficient    pam_succeed_if.so uid < 500 quiet
account     required      pam_permit.so

password    required      pam_passwdqc.so min=disabled,disabled,12,8,8 max=32 passphrase=2 match=4 similar=deny random=32 enforce=everyone retry=5
# instead of default pam_cracklib.so switch to more advanced pam_passwdqc.so 
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so

Brief explanation of  pam_passwdqc.so proposed here settings:
min=disabled,disabled,12,8,8  - disallows any length passwords from any single character class, disallows any length passwords from any two character classes, a minimum length of 12 characters for a passphrase  a minimum length of 8 characters for a password from any three character classes, and a minimum length of 8 characters from four character classes.the module will ask for a new password if the user fails to provide a sufficiently strong password
max=32 - maximum allowed password length is 32 characters.
passphrase=2 - passphrase must consist of 2 words.
match=4 -  check if 4 character substring in password will match dictionary worlds. In this case password complexity requirements will be checked against password with this substring removed.
similar=deny - deny new password to  be similar to the old one
random=32 - randomly-generated passphrases will be 32 bit (IMHO byte) long
enforce=everyone - enforce complex passwords
retry=5 - the module will ask 5 times for a new password if the user fails to provide a sufficiently strong password.

SSL certificates commands and tips

Verify a SSL certificate

1. By SSL verification web site: https://www.ssllabs.com/ssltest/index.html
2. By openssl:

$ openssl s_client -showcerts -connect test.domain.net:443

How to create pfx file that includes private key, certificate and certification chain?

$openssl pkcs12 -export -out test.domain.net_2012.pfx -inkey test.domain.net_2012.key -in test.domain.net_2012.crt -certfile thawtesecca.crt -certfile thawteprimca.crt

]How to see information inside CSR?

Normaly CSR looks like:

$cat test.domain.net_2012.csr
-----BEGIN NEW CERTIFICATE REQUEST-----
MIIC6TCCAdECAQAwgaMxCzAJBgNVBAYTAkNBMQ8wDQYDVQQIEwZRdWViZWMxETAP
BgNVBAcTCE1vbnRyZWFsMSIwIAYDVQQKExlSYWRpYWxwb2ludCBTYWZlQ2FyZSBJ
bmMuMQwwCgYDVQQLEwNOT0MxGjAYBgNVBAMUESoucmFkaWFscG9pbnQubmV0MSIw
IAYJKoZIhvcNAQkBFhNub2NAcmFkaWFscG9pbnQuY29tMIIBIjANBgkqhkiG9w0B
OZrroiuiouorpro[pooooo[orrjjjggoipoioZY2toSHheDDEtHga35cPDJUfl1W

ITBFs9G+fji+i0GrS1WmRYUGCeI1+5+48k6rIGim2WwcxYfBofVgZUSaFTydf1nH
icCM8aMi39jWtNCLxlWfBKNNLkNj0zPEPgXmYJIQRE0dUXfCBi68o+7s2pRyCJ7V
H9D8wV9OaBugaOMuPGM8vFtVTGRuC/waFuH/0Avr8SKs4U3Io6ZiViikopgfipii
dCeW6KZzJnAlBmrEuzv0POty5NDvnr2iZYUct15dNN8CheG36pXXrjFgqF6siBXs
lVBU023hBFsCu6foF3pflEmq9zVoVdUBruoepVbaIGziYyXes85C4Iokha9Ym8hE
YG0C/Nd4vp1ICB1gXBf88JoT7c2bgJWFSWpLVHU=
-----END NEW CERTIFICATE REQUEST-----

But if you do the trick:

$ openssl req -in test.domain.net_2012.csr -noout -text

Certificate Request:
   Data:
       Version: 0 (0x0)
       Subject: C=CA, ST=Quebec, L=Montreal, O=USer Inc., OU=SOC, CN=test.domain.net/emailAddress=soc@test.domain.com
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               Public-Key: (2048 bit)
               Modulus:
                   00:a5:83:4e:65:96:36:b6:84:87:85:e0:c3:12:d1:
                   47:21:30:45:b3:d1:be:7e:38:be:8b:41:ab:4b:55:
                   a6:45:85:06:09:e2:35:fb:9f:b8:f2:4e:ab:20:68:
                   a6:d9:6c:1c:c5:87:c1:a1:f5:60:65:44:9a:15:3c:
                   
                   8b:53:3f:6b:64:7e:2e:50:ec:35:1c:71:0f:42:4e:
                   bd:dd
               Exponent: 65537 (0x10001)
       Attributes:
           a0:00
   Signature Algorithm: sha1WithRSAEncryption
       7a:90:48:34:cb:70:47:af:54:a2:c2:f4:5e:5a:4c:57:a3:44:
       9b:6d:04:ec:b8:11:59:fb:32:c9:86:f4:a4:2f:5c:ad:df:cb:

       9d:48:08:1d:60:5c:17:fc:f0:9a:13:ed:cd:9b:80:95:85:49:
       6a:4b:54:75

You can get whole information about CSR you going to send for signature

Check CSR if certificate corresponds to private key

$ diff <(openssl rsa -in test.domain.net_2012.key -modulus -noout) <(openssl req -in test.domain.net_2012.csr -noout -modulus) 

if you get nothing, everything is OK.

Check what inside certificate you got from CA (certificate authority)

$ openssl x509 -in test.domain.net_2012.crt -noout -text

Certificate:
   Data:
       Version: 3 (0x2)
       Serial Number:
           73:68:b7:62:11:51
       Signature Algorithm: sha1WithRSAEncryption
       Issuer: C=US, O=Thawte, Inc., CN=Thawte SSL CA
       Validity
           Not Before: Aug 20 00:00:00 2012 GMT
           Not After : Oct 19 23:59:59 2014 GMT
       Subject: C=CA, ST=Quebec, L=Montreal, O=User Inc., CN=test.domain.net
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               Public-Key: (2048 bit)
               Modulus:
                   00:a5:83:4e:65:96:36:b6:84:87:85:e0:c3:12:d1:
                   47:21:30:45:b3:d1:be:7e:38:be:8b:41:ab:4b:55:
                   a6:45:85:06:09:e2:35:fb:9f:b8:f2:4e:ab:20:68:
                   a6:d9:6c:1c:c5:87:c1:a1:f5:60:65:44:9a:15:3c:

                   8b:53:3f:6b:64:7e:2e:50:ec:35:1c:71:0f:42:4e:
                   bd:dd
               Exponent: 65537 (0x10001)
       X509v3 extensions:
           X509v3 Subject Alternative Name: 
               DNS test.domain.net
           X509v3 Basic Constraints: 
               CA:FALSE
           X509v3 Certificate Policies: 
               Policy: 2.16.840.1.113733.1.7.54
                 CPS: https://www.thawte.com/cps/

           X509v3 Key Usage: critical
               Digital Signature, Key Encipherment
           X509v3 Authority Key Identifier: 
               keyid:A7:A2:83:BB:34:45:40:3D:A1:01:9F:F6:DB

           X509v3 CRL Distribution Points: 

               Full Name:
                 URI:http://svr-ov-crl.thawte.com/ThawteOV.crl

           X509v3 Extended Key Usage: 
               TLS Web Server Authentication, TLS Web Client Authentication
           Authority Information Access: 
               OCSP - URI:http://ocsp.thawte.com
               CA Issuers - URI:http://svr-ov-aia.thawte.com/ThawteOV.cer

   Signature Algorithm: sha1WithRSAEncryption
       34:3f:66:4f:7b:3e:44:f6:88:d8:de:62:3f:93:34:55:fd:7e:

       20:94:60:f8:6e:99:88:73:8f:86:6b:55:38:31:a1:0a:e2:c0:
       f9:22:0f:e7

Verify if certificate you got corresponds to correct private key

diff <(openssl rsa -in test.domain.net_2012.key -modulus -noout) <(openssl x509 -in test.domain.net_2012.crt -noout -modulus) 

if you get nothing, everything is OK.

Checking amount of packets dropped by iptables using zabbix

I really love zabbix  and I using it for monitoring almost everything. Below I describe how to use it for checking amount of packets dropped by iptables .

Why it's nice to monitor amount of dropped packets? It will show you amount of traffic blocked by host firewall and anomalies in the traffic inform you about attack (DDOS) or firewall misconfiguration.
 Sure thing, you can log all dropped packets and analyze packet's log by your SIEM solution.
 I propose simple way to get quick results and increase system protection without significant investment of money or your time.

1. Get dropped packets counters from iptables:
       - if you use default policy DROP for INPUT:
        iptables -L -n -v -x | grep "Chain INPUT" | awk {'print $7'}
     
      - if you use DROP rule:
        iptables -L -n -v -x | grep DROP | awk {'print $2'}

2.  To run iptables command you need to have root privileges, so zabbix agent (which running normally under "zabbix" user account) can't do it without following tricks:
 
Instead of setting SUID flag for whole iptables (very bad idea ),  I propose to create simple C program that will have SUID flag and able to get only amount of dropped packets:

# vi /usr/bin/iptables_drop.c

#include
#include
#include
#include

int main()
{
setuid( 0 );
        system("/sbin/iptables -L -n -v -x | grep 'Chain INPUT' | awk {'print $7'}");
   return 0;
}

compile:

 #gcc iptables_drop.c -o iptables_drop

change permissions: 

#chmod 4111 /usr/bin/iptables_drop

3. Now let's add a zabbix custom parameter in /etc/zabbix/zabbix_agentd.conf:

UserParameter=iptables.block,/usr/bin/iptables_drop

4. Now add corresponding item to zabbix checks list:

The trick here: change "Store value" parameter to "Delta (simpe change)"
it will store difference between counters, so show you new traffic between update interval.

Use this new item to build a Graph and stay informed!