CloudStack is one of the major and popular IaaS (Infrastructure as a Servile) platform.
http://en.wikipedia.org/wiki/Apache_CloudStack
http://en.wikipedia.org/wiki/Apache_CloudStack
Below small review of the password reset process in the CloudStack.
The purpose of password reset procedure - to allow user during deployment of the VM (virtual machine) template or after this to reset root (administrator password) of VM. Because of the main idea of IaaS to give user ability to help himself this is one of the key functionality.
From user perspective process looks like:
1. start new VM or click reset password on any stopped VM
2. Get popup with new root password
3. Log in using console, rdp or ssh using new password.
2. Get popup with new root password
3. Log in using console, rdp or ssh using new password.
Let's see what behind the scene:
Each network in CloudStack has dedicated router (VR) which doing dhcp, dns, loadbalancing, firewalling and password reset for whole subnet.
On VR we have following components of password reset service:
1. Process listening on port 8080:
socat -lf /var/log/cloud.log TCP4-LISTEN:8080,reuseaddr,crnl,bind=10.0.146.2 SYSTEM:/opt/cloud/bin/serve_password.sh "$SOCAT_PEERADDR"
it actually waiting for request like : DomU_Request: send_my_password
socat -lf /var/log/cloud.log TCP4-LISTEN:8080,reuseaddr,crnl,bind=10.0.146.2 SYSTEM:/opt/cloud/bin/serve_password.sh "$SOCAT_PEERADDR"
it actually waiting for request like : DomU_Request: send_my_password
2. script actually doing the job: /opt/cloud/bin/serve_password.sh
3. and password file: /var/cache/cloud/passwords having all passwords in clear text with filesystem permissions -rw-r--r--
10.0.146.15=rD7nudcze
10.0.146.13=jB9kbknvq
10.0.146.181=saved_password
after each password request VR replace corresponding password in password file by "saved_password"
On VM template and VM instance you have script: /etc/init.d/cloud-set-guest-password
This script automatically request root password from VR at each system startup and update it .
The password request procedure is:
1. Client VM parse local network setting and getting DHCP server IP.
2. Client send clear text request like wget -q -t 3 -T 20 -O - --header "DomU_Request: send_my_password" $PASSWORD_SERVER_IP:8080
3. If it get password it will use it. If it gets "saved_password" it won't do anything.
Security problems:
1. clear text password storage on VR
2. Clear text password transmission over the network
3. Missing password sever authentication (only by IP)
4. auto-starting password reset service.
Conclusion:
If attacker has access to the one instance into cloudstack network by spoofing password server(VR) IP he will able to compromise other instances int this subnet after their reboot. Having access to VR - will be able to compromise all nodes.
Hey there - just wanted to reach out with some feedback on your blog post.
ReplyDeleteWe are aware of the issues mentioned above, but consider them a low-level issue. Allow me to explain what I mean by this:
In the Apache CloudStack (ACS) model, we consider the network between hosts and virtual machines to be secure, and not open to 3rd party sniffing. If this part of the model is found to be invalid, then the installation will have all sorts of problems - as will just about any IaaS product out there. The same goes for servers and system VMs that ACS manages. If a malicious user could get access to them, they could modify the configuration database, DHCP servers, routing configurations, firewalls - in general it'd be a very bad thing. The user documentation stresses that systems and networks be hardened prior to ACS installation, and that the sytems are kept secure.
All that said, we are planning on improving this. Some of us were already talking about improving the password reset system to make it more secure and resilient. I'll try to post a functional spec to our wiki by the end of this week, and would love to have your feedback upon the new design.
If you have further concerns we're happy to discuss them on the CloudStack developer's mailing list. If you find further issues, please do bring them to our attention by emailing security@cloudstack.apache.org. Thanks!
John Kinsella
Apache CloudStack PMC member, committer, and security team member
Thanks a lot John for your detailed response. I completely agree with you that system VM should be assumed as a secure place, but network, especially if it shared and available on the Root level IMHO couldn't be considered secured. I have some ideas for instance-VR mutual authentication and will be happy to see new functional spec and discuss it in the mailing list.
ReplyDelete