On a sunny day in februari suddenly all hosts in one cluster give the folllowing error: Error: Cannot synchronize host s02esxhyp000003 Cannot complete login due to an incorrect user name or password. Removing the host from the cluster and re-add did not help, neither did restarting the services: service vmware-vpxa restart service mgmt-vmware restart. Every action i do in the cluster gives me the following error "vim.fault.Invalidlogin": Finally good old mister Google came up with the following solution. 1. Disconnect the host from vcenter (do not remove) 2. Run from the ESX console "userdel vpxuser", or connect to the host using the viclient and Delete the vpxuser 3. Reconnect the host in vCenter, it will first give an error about a wrong password and then starts the Add Host Wizard. I came across a nice knowwie when i was troubleshooting this error. The next time you add a host to a cluster watch the "Recent Tasks" that appear in a viclient connected directly to that host. So indeed start another viclient and connect directly to the host, then go back to the viclient connected to vcenter and add the host to a cluster (preferably one with resource pools).

After waiting for 18 agonizing days i finally received the results of my VCAP-DCA exam.

I am very glad to say i passed and can now officially be called a

"VMware Certified Advanced Professional - Datacenter Administrator"

Admit it!; it is a mouth full

 

Next up is the VCAP-DCD exam, i scheduled the Datacenter Design exam for friday the 30th December.

 

 

 

Today i received the following email from vmware about a change to the announced new licensing model for vSphere 5.

On the one hand i think its good that they listen to the community but on the other hand they can ask themselves "Did we think this new licensing model through enough before announcing it?"

On July 12, 2011, VMware announced our new Cloud Infrastructure Suite. The launch featured vSphere 5, the newest version of our flagship product.

As many of you know, as part of this announcement, we introduced changes to the vSphere licensing model in order to align costs with the benefits of virtualization rather than with the physical attributes of individual servers. While our goal was to provide a licensing model based on consumption and value rather than physical components and capacity, we strived to make the new model as non-disruptive as possible.

These changes generated much debate in the blogosphere, in conversations with our partners and customers, and across VMware communities. Some of the discussion had to do with confusion around the changes. We have been watching the blog commentaries carefully, and we have been listening to the partner and customer conversations very intently. A great deal of feedback was provided that examined the impact of the new licensing model on every possible use case and scenario, and equally importantly, reflected our partners’ and customers’ intense passion for VMware.

Our success depends on the active involvement of our channel partners. We are a company built on partner and customer goodwill, and we’ve taken your feedback in earnest. Our primary objective is to do right by our customers, so we are announcing three changes to the vSphere 5 licensing model that address the most recurring areas of your feedback.

Finally, we introduced the vSphere Desktop edition to address vSphere licensing in a desktop environment. The vSphere Desktop edition does not have any vRAM entitlements, and allows customers to purchase vSphere for VDI use case on per user basis. Our price books are being updated and will be distributed shortly.

One of the new features of VMware vSphere is booting the ESXi hosts from PXE. This feature is called Auto-Deploy and gives you the ability to use stateless ESXi servers.

Below is a step-by-step guide to get you started. It assumes you use the vCenter Virtual Appliance. of course auto deploy also works with any other DHCP/TFTP server and auto-deploy itself can be installed on a Windows vCenter server as well.

1. Open the vCenter administration webpage (https://<vcenter>:5480), Go to the services tab and make sure the ESXi services are started. If not click “Start ESXi Services”.
2. Configure DHCPd in /etc/dhcpd.conf. Configure at least the following items.
   1. DNS / GW
   2. Subnet
   3. scope or reservations
   4. next-server
   5. filename: undinonly.kpxe.vmw-hardwired
   6. uncomment or add the keyword autoritative;
   7. My dhcpd.conflooks like this:

option domain-name "vmtopia.lab";
default-lease-time 600;
max-lease-time 7200;
autoritative;

log-facility local7; 
 allow booting;
 subnet 192.168.16.0 netmask 255.255.255.0 {
   range 192.168.16.150 192.168.16.160;
   next-server 192.168.16.100;
   filename "/tftpboot/undionly.kpxe.vmw-hardwired";
 } 

1. bind dhcp to network interface by editing the /etc/sysconfig/dhcpd file
   Add or change this line: DHCPD_INTERFACE = “ANY”
2. Start DHCP: service dhcpd start
3. Make sure the boot files are in /tftpboot. The VCVA comes with the right files.
4. Start atftpd: service atftpd start
5. Open file /tftpboot/tramp to check if hostname is used. If no DNS available replace vcenter hostname for IP address
6. If you want to make sure your hosts can access tftpd even if DNS fails or when hostname is not in DNS you have to edit /etc/hosts.allow. At the bottom of the file change ALL : KNOWN to ALL : ALL
7. Poweron an ESX server. Check if it loads gPXE. As the information screen shows, it won't boot because there is no autodeploy rule associated with that host. The host will keep rebooting so power it off for now
8. The next step is to create autodeploy rules which associate the hosts to a boot image, host profile and a cluster or folder. The only way to create those rules is by using PowerCLI. So install the vSphere5 PowerCLI on a windows box, start it and login to vCenter using the  connect-viserver commandlet
9. Now we have to load an ESXi depot. The default ESXi depot can be downloaded from the Vmware site <<LINK>>. Now load the depot using the following powershell command:
   Add-esxsoftwaredepot <path to depot file>
10. In the default depot there are two image profiles. An image profile is basically a collection of VIB files. A VIB file can contain the base system, Vmware tools, drivers etc. We can show the profiles available in the depot by using:
    get-esximageprofile
11. To make a new ESX server boot from one of the profiles we have to create a rule. A rule has three properties:
    1. Rule name: Just give a descriptive name here like “imageprofilerule”
    2. Item: this is the object you want to associate with a host or group of hosts. In this case it will be an image profile (ESXi-5.0.0-381646-standard)
    3. Pattern: The pattern defines to which hosts this rule will apply. There are a lot of different variables you can put into the pattern like: Vendor, Model, MAC, Asset Tag and IP range. For now we will just use the IP range.
12. Create a new rule:
    new-deployrule -name imageprofilerule -item ESXi-5.0.0-381646-standard -pattern “ipv4=192.16.0.100-192.168.0.200”
13. After creating the rule, the rule has to be added to the active rule set:
    add-deployrule imageprofilerule
    To show the current rule set use: get-deployrule
14. Now autodeploy knows which image to boot. The host will be added to vCenter automatically but we have to tell autodeploy where to put the host (which datacenter or cluster). So let's add another rule:
    new-deployrule -name clusterrule -item cluster1 -pattern “Model=”BL 460c G6”
    Note that i used the server model in the matching pattern here so there will be only one type of machines in the cluster.
15. And add it to the ruleset: add-deployrule clusterrule
16. Now poweron your ESX server and watch the magic happen :)
17. Ater your first ESX server is started you can configure it, create a hostprofile from it and create a rule to make sure all your servers will use that hostprofile. I will discuss host profiles in another post.
    new-deployrule -name profilerule -item <hostprofilename> -ALLservers
    add-deployrule profilerule
    If you don't want all your servers to use the same profile you can also use a pattern here instead  of “allservers”
18. Now poweron all 10000 ESX servers, don't tell your boss you're ready installing all servers, leave the office pretending you need to be in the datacenter and enjoy you afternoon in the pub :)

cheers!

 

Today i was invited by VMware to participate in the VCP5 beta exam program.

The official exam code is:

VCP511: VMware Certified Professional on vSphere 5 (Private Beta - Invite Only)

It is only available for the official RCBETA 5 program members.

I decided i am going to take a shot at it on friday july 22nd. the beta exam periode only runs from july 12 to the 24th so there is not much time to prepare.

 

Last Friday I took a shot at the VMware VCAP-DCA exam.
I kinda knew what to expect, but still it was a toughy!

Its all labs, no multiple choice questions.
You get 4 hours, which in the end isn't a lot, its not extremely difficult but it just is a lot.
There are PDFs at hand but you hardly have time to check them.
There was one time when i could have used the Hardening guide but alas it was nowhere to be found
I think i flunked it, big time, because i ran outta time after only 70% of the questions.
I will get the result within 10 business days, i'll post again when i receive the bad news

Tip:

Immediately skip the questions you do not know.
When there is time left try to do them using the PDFs.

 

13-072011 - Yep, i flunked it.

The number of point wasnot all that bad considering i missed about a third of the questions due to a lack of time.

 

vCenter Server 4.0, VMotion of a VM fails

Error in viclient: Insufficient disk space on datastore

I ran into this error today.

When i checked the datastore i noticed there was still 5GB of free space.

I do not want to say that this is a fair enough amount of free space on a 280GB Datastore, and I did anticipate some possible problems when using snapshots, but I did not expect problems with Vmotion.

I remembered some capacity guidelines from vmware and after some googling i found them here:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003412

In short:

____________________________________

All VMFS (vmfs2, vmfs3) must retain a level of free space to ensure that the virtual machine functions properly. Two important capacity thresholds must be observed:

· 200 Megabyte Capacity: VMware suggests 200 MB as the threshold where ESX system administrators take action to resolve capacity issues,

   VMFS volumes vary widely in size and it is impossible to define a precise percentage, but the risk of unexpected problems become greater at this point.

· 100 Megabyte Remaining: If left unchecked, a VMFS volume grows until only 100 MB of space remains, at which time issues may occur.

  100 MB is defined as a critical threshold at which ESX system administrators must take immediate action to correct the issue because VMware VMFS stops growing at this point.

____________________________________

The odd thing is that these thresholds do not account for preserving the VMotion, DRS and Snapshot capabilities.

I agree that, with these sugested thresholds, VMs will keep running but you lose a lot of functionality when using these capacity thresholds.

 

Cause of the problem:

vCenter Server 4.0 requires enough space to store two full-size copies of the virtual machine swap file simultaneously. If there is not enough space on the datastore, you receive this error.

So the VMotion of a VM with 8GB of memory requires 8GB of free diskspace on the datastore.

I have not checked it, yet, but i can imagine that this is not the case when using vCenter Server 4.1.

This could really cause problems with large memory VMs or when vmotioning multiple VMs sharing the same datastore. 

Solution:

Free up space on the datastore (SVmotion will still work as this does not move the VM, just make sure you have enough space for the temporarily delta files) 

Workaround:

Since only the non-reserved memory uses the swap file, you can temporarily increase the memory reservation of the virtual machine and remove the reservation after the vMotion.

 

If you are into cloud computing you are probably familiar with the company Terremark. In 2009 VMware invested 20 million dollars in terremark in return for a 5% stake.If you were at VMworld back in 2009 you can't have missed all the "hidden" advertisement for this company :)

Today Verizon announced that it will acquire Terremark for 1.4 Billon dollars. That is $19 per share, VMware only payed $5 per share so for VMware that is quite a good return on investment. I am just wondering what they'll do whith the money. Will they invest it in yet another cloud company like rackspace or bluelock so they have some influence on how this business will develop? Or will they spent the money on pushing VMware Desktop virtualization into the marked?

Yesterday I had the strangest errors in the VI-Client when accessing the “Map View” functionality under the “Storage View” tab.

 The following error came when accessing the “Storage View” Tab and clicking “Map”

 

 The following error came when changing the “Map” checkboxes and clicking “Update View”

 

 Well, they look very clear error messages don’t they? J

 It took me quite some googling to find the following:

http://www.red-gate.com/supportcenter/Content?p=SQL%20Dependency%20Tracker&c=SQL_Dependency_Tracker%5Cknowledgebase%5CSDT_VJSLIB_Not_Found.htm

We seemed to be missing a “Microsoft Visual J# Version 2.0 Redistributable Package.”

After downloading and installing the package the errors were gone and I could see the Maps.