Virtual machines templates. Static or Live?

Virtualization is everywhere. Need a new VM ? - it's easy,  clone from template and spin it up!

But let's take a close look at the classical "static" templates security cons:

- Static disk image file 

- get updated from time to time ( in most cases not often)

- security policies (or hardening) on it updated only after deployment 

- not being monitored by any part of your security monitoring infrastructure (no events coming, no AV or vulnerabilities  scans,  etc.)

Templates are the key component of your infrastructure - you are cloning all you running machines from them.

To keep templates on the same security level you need: spin up a vm from this template, update it, check stability after updates, scan it for vulnerabilities and re-template back (and do not forget to update checksums. You are checking templates' checksums before deployment, aren't you? :-)   ). 

IMHO, the biggest security issue with "static" templates - is the time frame when your recent cloned from template machine boot up and  get updates  being at the same time not fully protected and up to day.  

Instead of using "static" templates, you can use "live" templates - just normal running VMs  (CPU and memory are really cheap now). So ,what the pros:

- part of you live infrastructure

- get updates as soon as updates are issued or/and approved
- get current version of the security(or hardening) policy implemented
- monitored and covered by your security infrastructure(AV scanners, HIDS/NIDS, log management)
- monitored for up time and stability after each update

Script, shown below, being placed in cron.daily will keep your RH based linux  template up to day (for VMware virtualization)

#!/bin/sh

# Doing self-update if we are running on host with template in name and self-destruction on other hosts 

if [[ "$HOSTNAME" == y??0?centos?tmpl* ]]; 
then /usr/bin/yum update -y > /var/log/self_update.log 2>&1
else
/bin/rm -v /etc/cron.daily/self_update > /var/log/self_update.log 2>&1 
fi

# Comparing installed and running kernel and reboot in case if they differ 

LAST_KERNEL=$(/bin/rpm -q --last kernel | /usr/bin/perl -pe 's/^kernel-(\S+).*/$1/' | /usr/bin/head -1)
CURRENT_KERNEL=$(/bin/uname -r)
if [ $LAST_KERNEL != $CURRENT_KERNEL ];
then 
/bin/echo '/usr/bin/vmware-config-tools.pl -d ; /bin/sed -i "/.*vmware.*/d"  /etc/rc.local'  >> /etc/rc.local
/sbin/reboot; 
fi
exit 0

PS. Script also address annoying problem which requires rerun vmware configuration utility after each kernel update to make vmware tools run .
PS2 And self-destruct on non-template hosts.

Shame on you, Actividentity!

Looking for ActivIdentety smart cards, google them and first link pointing to the web site in Drupal install mode

whois activcard.com

Registrant:

Actividentity Corporation

   6623 Dumbarton Circle

   Fremont, CA 94555

   US

   Domain Name: ACTIVCARD.COM

CloudStack IaaS insecure password reset

CloudStack is one of the major and popular IaaS  (Infrastructure as a Servile)  platform.
http://en.wikipedia.org/wiki/Apache_CloudStack

Below small review of the password reset process in the CloudStack.

The purpose of password reset procedure - to allow user during deployment of the VM (virtual machine) template or after this to reset root (administrator password) of VM. Because of the main idea of IaaS to give user ability to help himself  this is one of the key functionality.

From user perspective process looks like:

1. start new VM or click reset password on any stopped VM
2. Get popup with new root password
3. Log in using console, rdp or ssh using new password.

Let's see what behind the scene:

Each network in CloudStack has dedicated router (VR) which doing dhcp, dns, loadbalancing, firewalling and password reset for whole subnet.

On VR we have following components of password reset service:

1. Process listening on port 8080:
 socat -lf /var/log/cloud.log TCP4-LISTEN:8080,reuseaddr,crnl,bind=10.0.146.2 SYSTEM:/opt/cloud/bin/serve_password.sh "$SOCAT_PEERADDR"
it actually waiting for request like : DomU_Request: send_my_password

2. script actually doing the job: /opt/cloud/bin/serve_password.sh

3. and password file: /var/cache/cloud/passwords having all passwords in clear text with filesystem permissions -rw-r--r--
  10.0.146.15=rD7nudcze
 10.0.146.13=jB9kbknvq
 10.0.146.181=saved_password

after each password request VR replace corresponding password in password file  by "saved_password"

On VM template and  VM instance you have script: /etc/init.d/cloud-set-guest-password
This script automatically request root password from VR at each system startup and update it .

The password request procedure is:

1. Client VM parse local  network setting and getting DHCP server IP.

2. Client send clear text  request like wget -q -t 3 -T 20 -O - --header "DomU_Request: send_my_password" $PASSWORD_SERVER_IP:8080

3. If it get password it will use it. If it gets "saved_password" it won't do anything.

Security problems:

1. clear text password storage on VR

2. Clear text password transmission over the network

3. Missing password sever authentication (only by IP)

4. auto-starting password reset service.

Conclusion:

If attacker has access to the one instance into cloudstack network by spoofing password server(VR) IP he will able to compromise other instances int this subnet after their reboot. Having access to VR - will be able to compromise all nodes.

Adding security into Enterprise Linux system-wide authentication mechanism:

Some small modification to default authentication mechanism at RedHat 5/6 based system will increase system security.

First witch from md5 to sha512 hashes: it as easy as executing:

"authconfig--passalgo=sha512 --updateall"

Update your /etc/pam.d/system-auth to the more adequate to the current threats settings:

#vi /etc/pam.d/system-auth

#%PAM-1.0
# User changes will be destroyed the next time authconfig is run.

auth        required      pam_env.so
auth        required      pam_unix.so nullok try_first_pass
auth        sufficient    pam_tally2.so deny=10 onerr=fail unlock_time=1200
# lock account after 10 failed attempt. unlock automatically after 20 minutes.
auth        required      pam_deny.so

account     required      pam_tally2.so
account     required      pam_unix.so
account     sufficient    pam_succeed_if.so uid < 500 quiet
account     required      pam_permit.so

password    required      pam_passwdqc.so min=disabled,disabled,12,8,8 max=32 passphrase=2 match=4 similar=deny random=32 enforce=everyone retry=5
# instead of default pam_cracklib.so switch to more advanced pam_passwdqc.so 
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     required      pam_unix.so

Brief explanation of  pam_passwdqc.so proposed here settings:
min=disabled,disabled,12,8,8  - disallows any length passwords from any single character class, disallows any length passwords from any two character classes, a minimum length of 12 characters for a passphrase  a minimum length of 8 characters for a password from any three character classes, and a minimum length of 8 characters from four character classes.the module will ask for a new password if the user fails to provide a sufficiently strong password
max=32 - maximum allowed password length is 32 characters.
passphrase=2 - passphrase must consist of 2 words.
match=4 -  check if 4 character substring in password will match dictionary worlds. In this case password complexity requirements will be checked against password with this substring removed.
similar=deny - deny new password to  be similar to the old one
random=32 - randomly-generated passphrases will be 32 bit (IMHO byte) long
enforce=everyone - enforce complex passwords
retry=5 - the module will ask 5 times for a new password if the user fails to provide a sufficiently strong password.

SSL certificates commands and tips

Verify a SSL certificate

1. By SSL verification web site: https://www.ssllabs.com/ssltest/index.html
2. By openssl:

$ openssl s_client -showcerts -connect test.domain.net:443

How to create pfx file that includes private key, certificate and certification chain?

$openssl pkcs12 -export -out test.domain.net_2012.pfx -inkey test.domain.net_2012.key -in test.domain.net_2012.crt -certfile thawtesecca.crt -certfile thawteprimca.crt

]How to see information inside CSR?

Normaly CSR looks like:

$cat test.domain.net_2012.csr
-----BEGIN NEW CERTIFICATE REQUEST-----
MIIC6TCCAdECAQAwgaMxCzAJBgNVBAYTAkNBMQ8wDQYDVQQIEwZRdWViZWMxETAP
BgNVBAcTCE1vbnRyZWFsMSIwIAYDVQQKExlSYWRpYWxwb2ludCBTYWZlQ2FyZSBJ
bmMuMQwwCgYDVQQLEwNOT0MxGjAYBgNVBAMUESoucmFkaWFscG9pbnQubmV0MSIw
IAYJKoZIhvcNAQkBFhNub2NAcmFkaWFscG9pbnQuY29tMIIBIjANBgkqhkiG9w0B
OZrroiuiouorpro[pooooo[orrjjjggoipoioZY2toSHheDDEtHga35cPDJUfl1W

ITBFs9G+fji+i0GrS1WmRYUGCeI1+5+48k6rIGim2WwcxYfBofVgZUSaFTydf1nH
icCM8aMi39jWtNCLxlWfBKNNLkNj0zPEPgXmYJIQRE0dUXfCBi68o+7s2pRyCJ7V
H9D8wV9OaBugaOMuPGM8vFtVTGRuC/waFuH/0Avr8SKs4U3Io6ZiViikopgfipii
dCeW6KZzJnAlBmrEuzv0POty5NDvnr2iZYUct15dNN8CheG36pXXrjFgqF6siBXs
lVBU023hBFsCu6foF3pflEmq9zVoVdUBruoepVbaIGziYyXes85C4Iokha9Ym8hE
YG0C/Nd4vp1ICB1gXBf88JoT7c2bgJWFSWpLVHU=
-----END NEW CERTIFICATE REQUEST-----

But if you do the trick:

$ openssl req -in test.domain.net_2012.csr -noout -text

Certificate Request:
   Data:
       Version: 0 (0x0)
       Subject: C=CA, ST=Quebec, L=Montreal, O=USer Inc., OU=SOC, CN=test.domain.net/emailAddress=soc@test.domain.com
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               Public-Key: (2048 bit)
               Modulus:
                   00:a5:83:4e:65:96:36:b6:84:87:85:e0:c3:12:d1:
                   47:21:30:45:b3:d1:be:7e:38:be:8b:41:ab:4b:55:
                   a6:45:85:06:09:e2:35:fb:9f:b8:f2:4e:ab:20:68:
                   a6:d9:6c:1c:c5:87:c1:a1:f5:60:65:44:9a:15:3c:
                   
                   8b:53:3f:6b:64:7e:2e:50:ec:35:1c:71:0f:42:4e:
                   bd:dd
               Exponent: 65537 (0x10001)
       Attributes:
           a0:00
   Signature Algorithm: sha1WithRSAEncryption
       7a:90:48:34:cb:70:47:af:54:a2:c2:f4:5e:5a:4c:57:a3:44:
       9b:6d:04:ec:b8:11:59:fb:32:c9:86:f4:a4:2f:5c:ad:df:cb:

       9d:48:08:1d:60:5c:17:fc:f0:9a:13:ed:cd:9b:80:95:85:49:
       6a:4b:54:75

You can get whole information about CSR you going to send for signature

Check CSR if certificate corresponds to private key

$ diff <(openssl rsa -in test.domain.net_2012.key -modulus -noout) <(openssl req -in test.domain.net_2012.csr -noout -modulus) 

if you get nothing, everything is OK.

Check what inside certificate you got from CA (certificate authority)

$ openssl x509 -in test.domain.net_2012.crt -noout -text

Certificate:
   Data:
       Version: 3 (0x2)
       Serial Number:
           73:68:b7:62:11:51
       Signature Algorithm: sha1WithRSAEncryption
       Issuer: C=US, O=Thawte, Inc., CN=Thawte SSL CA
       Validity
           Not Before: Aug 20 00:00:00 2012 GMT
           Not After : Oct 19 23:59:59 2014 GMT
       Subject: C=CA, ST=Quebec, L=Montreal, O=User Inc., CN=test.domain.net
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               Public-Key: (2048 bit)
               Modulus:
                   00:a5:83:4e:65:96:36:b6:84:87:85:e0:c3:12:d1:
                   47:21:30:45:b3:d1:be:7e:38:be:8b:41:ab:4b:55:
                   a6:45:85:06:09:e2:35:fb:9f:b8:f2:4e:ab:20:68:
                   a6:d9:6c:1c:c5:87:c1:a1:f5:60:65:44:9a:15:3c:

                   8b:53:3f:6b:64:7e:2e:50:ec:35:1c:71:0f:42:4e:
                   bd:dd
               Exponent: 65537 (0x10001)
       X509v3 extensions:
           X509v3 Subject Alternative Name: 
               DNS test.domain.net
           X509v3 Basic Constraints: 
               CA:FALSE
           X509v3 Certificate Policies: 
               Policy: 2.16.840.1.113733.1.7.54
                 CPS: https://www.thawte.com/cps/

           X509v3 Key Usage: critical
               Digital Signature, Key Encipherment
           X509v3 Authority Key Identifier: 
               keyid:A7:A2:83:BB:34:45:40:3D:A1:01:9F:F6:DB

           X509v3 CRL Distribution Points: 

               Full Name:
                 URI:http://svr-ov-crl.thawte.com/ThawteOV.crl

           X509v3 Extended Key Usage: 
               TLS Web Server Authentication, TLS Web Client Authentication
           Authority Information Access: 
               OCSP - URI:http://ocsp.thawte.com
               CA Issuers - URI:http://svr-ov-aia.thawte.com/ThawteOV.cer

   Signature Algorithm: sha1WithRSAEncryption
       34:3f:66:4f:7b:3e:44:f6:88:d8:de:62:3f:93:34:55:fd:7e:

       20:94:60:f8:6e:99:88:73:8f:86:6b:55:38:31:a1:0a:e2:c0:
       f9:22:0f:e7

Verify if certificate you got corresponds to correct private key

diff <(openssl rsa -in test.domain.net_2012.key -modulus -noout) <(openssl x509 -in test.domain.net_2012.crt -noout -modulus) 

if you get nothing, everything is OK.

Checking amount of packets dropped by iptables using zabbix

I really love zabbix  and I using it for monitoring almost everything. Below I describe how to use it for checking amount of packets dropped by iptables .

Why it's nice to monitor amount of dropped packets? It will show you amount of traffic blocked by host firewall and anomalies in the traffic inform you about attack (DDOS) or firewall misconfiguration.
 Sure thing, you can log all dropped packets and analyze packet's log by your SIEM solution.
 I propose simple way to get quick results and increase system protection without significant investment of money or your time.

1. Get dropped packets counters from iptables:
       - if you use default policy DROP for INPUT:
        iptables -L -n -v -x | grep "Chain INPUT" | awk {'print $7'}
     
      - if you use DROP rule:
        iptables -L -n -v -x | grep DROP | awk {'print $2'}

2.  To run iptables command you need to have root privileges, so zabbix agent (which running normally under "zabbix" user account) can't do it without following tricks:
 
Instead of setting SUID flag for whole iptables (very bad idea ),  I propose to create simple C program that will have SUID flag and able to get only amount of dropped packets:

# vi /usr/bin/iptables_drop.c

#include
#include
#include
#include

int main()
{
setuid( 0 );
        system("/sbin/iptables -L -n -v -x | grep 'Chain INPUT' | awk {'print $7'}");
   return 0;
}

compile:

 #gcc iptables_drop.c -o iptables_drop

change permissions: 

#chmod 4111 /usr/bin/iptables_drop

3. Now let's add a zabbix custom parameter in /etc/zabbix/zabbix_agentd.conf:

UserParameter=iptables.block,/usr/bin/iptables_drop

4. Now add corresponding item to zabbix checks list:

The trick here: change "Store value" parameter to "Delta (simpe change)"
it will store difference between counters, so show you new traffic between update interval.

Use this new item to build a Graph and stay informed!

Open Security Self-Test initiative

Hi folks!

I'd like to establish  Open Security Self-Test initiative:

It' will allow any user to make security audit of his gadget (laptop, tablet, smartphone) on free of charge and self-service basis.

Desc: Free wifi APs located in public places connected to special designed security audit server.

• All users nearby will be able connect to this AP (all communication gonna be encrypted, password for connection will be included into AP's SSID )
• After connection to AP, user's workstation security scan will be started and all user's trafic to Internet (or Internet emulation sandbox) will be monitored by NIDS.
• In couple of minutes user will get  report about any security threats found on his workstation and short recommendations how to solve the problems including the links to updates/patches and free/trial anti-viruses.
• All security audits will be done by open-source solutions (some commercial tools could be involved if  vendors will allow such usage)
• Whole solution will work on free of charge basis.

Architecture:

Updated: 

Proposed architecture could use Cloud tendency: in this case  Security Audit Servers will be located in the Cloud and being connected by VPN to distributed across the world WiFi APs. APs will use modified firmware (OpenWrt) and tunnel all the traffic from connected users' gadgets to the Cloud.
It will definitely reduce the operation costs and give a great scalability, flexibility and reliability for whole solution.

1. Access Point (AP)

2. Security Audit Server/s  including:

• Captive portal 
• Firewall
• Vulnerabilities  scanner/s
• Network Intrusion Detection System (NIDS)
• Internet connection or Internet emulation

Current implementation:

Nginx+ Bind+iptables+snort+ nmap +OpenVAS+tcpdump+Scripts


Usage:

1. User  connects to public AP by his laptop, tablet or smartphone. PSK for connection included in broadcasted AP's SSID. Connection encrypted.

User get IP form DHCP and all connection attempt forwarded to Captive portal

2. Users establish connection via https with Open Security Self-Test web server.
Read end-user agreement, Accept it and start the security audit:

3. Vulnerabilities scan runs against user's device:

4.  During vulnerabilities scan and mostly after it user computer get connected to Internet (or Internet emulation). All traffic between user's device and Internet intercepted and monitored by NIDS.

5. Finally, user will get brief (with option to download full) description of security problems found and recommendations how to fix it. Recommendations will include links to vendors' patches, updates and free and trial antivirus software.

During security audit process user will see progress bar and will be able surf Internet.



Current stage: Alfa-testing.  Scheduled for  first implementation on late February, Montreal, QC.

Solution could be implemented on a stand-alone basis in public places or as a part of current free-WiFi access infrastructure. Showing advertisement to a users during the scan will allow using some commercial security audit solution or ,even, make project profitable.