VMWorld – Day 2 – September 12, 2007
I was at VMWorld last week. Unfortunately, my network connectivity was limited so I was unable to post my notes during the conference.
Keynotes – John Chambers, CEO Cisco
John Chambers message was basically talking about how IT enables business and how to present that case to business executives.
Keynotes– Energy Panel
Quentin Hardy, Panel Moderator, Forbes Magazine
David Brooks, Harvard
Ray Cline, EDS
Amdrew Fanara, EPA
John Gibson, HSBC Bank
ESX Server CPU Scheduling
Andrei Drofeev, VMWare
CPU Scheduling Internals
- SMP Virtual Machines
- Load Balancing & Migrations
- Hyperthreading Support
- User Worlds – VMWare term for whole VM
- Interrupts
- NUMA
Performance Tips
- Idle VMs still receive timer interrupts
- Avoid VM CPU affinity
- SMP – use as few virtual CPUs as possible
- 64 bit guest generally better performance
- Avoid running programs in service console
- Don't fully commit CPU with reservations
ESXTOP Utility
- High %RDY time – VM was ready for CPU time but had to wait
- High %Wait / %Idle – workload is not CPU intensive
Future Directions
- Scalability – # CPUs per host
- Power mgt
- Better multi-core support
VM Infrastructure 3 – Best Practices for Performance
Jeff Buell, Staff Engineer
Deraki Kulkarni, Sr. MTS, VMWare
Sources of Virtualization Overhead
- CPU
- Memory
- Devices
- Resource Management
CPU Performance:
- Keep in mind OS timer interrupts. For awhile, Linx was using a 1000 Hz which was too often. Latest linux is set to 250 Hz which is OK. Windows is 100 Hz.
- 64 bit guests give better performance
- DIsable unused controllers / devices (see KB 1290)
Memory Performance
- Page sharing – pages which are the same between VMs can be shared between VMs
- Memory ballooning – require VMWare tools
- Avoid active memory over commit (eliminate ESX memory swapping)
- Right size guest OS memory
Networking
- Ensure there is enough CPU to process networks
- Use vmxnet network driver from VMWare tools
Storage Performance
- Guest driver setting can affect performance (KB 9645697) – Increase the size of writes
- Recommend using Fibre channel SAN
- NFS or iSCSI – use more CPY than SAN
- Better performance if use Virtual Center to create partitions
- Increase VMs max outstanding disk requests if needed (KB 1268)
Using the Secure Technology Implementation Guide (STIG) with VMWare Infrastructure 3
DISA – Defense Infromation Systems Agency
STIG – general guide for securing systems
Vulnerability Categories
Category I – Worse level. Allows access tot he machine
Category II – Provides information that could lead to access (high potential)
Category III – Provide information that could lead to access
5 STIGs related to VMI3 (http://iase.disa.mil/stigs
- Virtual Computing STIG
- Unix STIG (ESX Server, VMs)
- DB STIG (Virtual Center)
- Win OS STIG (Win VM)
- Web Server STIG and checklist
- Other => SRR scripts, associated checklists, vulnerabilities management systems
Run SRR scripts (system readiness review – ./start-SRR)
See slides for many details of current findings and their meanings.
VMWare Infrastructure 3: Advanced Diagnostic Log Analysis
Mostafa Khalil, VCP, VMWare Product Support Engineer
ESX Server Boot Process: Boot Loader -> initrd -> vmkernel -> vmnix -> /sbin/init ->init scripts -> vmware init scripts
Collecting logs:
- UI via VI Client (right click on item and select export diagnostics data)
- Multiple logs on each ESX server
VMKernel Log:
All events generated by vmkernel (warnings are also written to seperate log)
Logs rotates. All events since last load are also in memory at /proc/vmware/log.
General Log msg: timestamp, hostname, msg source, uptime, <instance>, device, src line #, msg
Translating vmkernel error codes: already listed in msg in ESX 3
Message Log:
Console events, logon events, iSCSI authentication events
hostd.log:
Events done on behalf of various services
vpxa.log
esxcfg-firewall
oldconf files
esxupdate.log
vmkernel-version
Security Architecture Design and Hardening of VMWare Infrastructure 3
Kick Larsen – Engineering – Product Security Officer
Banjot S. Chanana – Product Mgr, Platform Security
Brian Cosker-Swerske – Senior Consultant
Service Console
Ports 902, 80, 443 and 22 open by default
Hardening:
Disable ctrl-alt-del
Require password for single user mode
service console network parameters
login banners
Password complexity
/etc/security/access.conf
enable syslog
Change snmp community string from the default
Securing VMs
Remove unnecessary functions / services
Disable cut and paste