vCenter Server Appliance 6.0 in VMware Workstation

For demo’s, presentations, breaking environments or just killing time I have a portable testlab on my notebook. Yes I know there are also options for permanent labs, hosted labs and Hands-on labs for these same purposes. Great places for sure, but that is not really what I wanted to discuss here.

As I am break.. ehhh rebuilding my lab to vSphere 6.0 I wanted to install VCSA 6.0 in VMware workstation. Nice, import a my vmware downloaded VCSA-versionsomething.ova (after e-mail address number ####### registered over there) and we are done! …….. Well not quite.
First the vCenter download contains the OVA, but it is a little bit hidden. The guided installer will not help you here. You will need to mount or extract the downloaded ISO and look for vmware-vcsa in the vcsa/ folder.

VCSA Location
Copy the vmware-vcsa file to a writable location (when just mounted the ISO) and rename vmware-vcsa to vmware-vcsa.ova. And now we can import the ova to VMware Workstation. When the import finishes, do not start the VM yet. Certain values that are normally inserted via the vSphere Client or ovftool are to be appended to the VMX file of the imported VCSA. Open the vmx in the location where you let Workstation import the VM. Append the following lines: = “ipv4” = “static” = “” = “8” = “” = “”
guestinfo.cis.vmdir.password = “vmware-notsecurepassexample”
guestinfo.cis.appliance.root.passwd = “vmware-notsecurepassexample”

Note: Change the net and vmdir/appliance.password options to the appropriate values for your environment.

If not appended when you start the VCSA an error: vmdir.password not set aborting installation is shown on the console (next to root password not set) and network connection will be dropped even if you configure these on the VCSA console (via F2).

Save the VMX file.

And now it is time to let it rip. Start up your engines. And be patience until…lo and behold:

VCSA Running in workstation

And to check if the networking is accepting connections from a server in the same network segment open up the VCSA url in Chrome for example. After accepting the self signed certificate unsecure site (run away!) message you will (hopefully) see:

VCSA in Workstation

Next we can logon to the Web client (click and accept the unsecure connection/certificate) and logon via [email protected] and the password provided in the VMX (in the above example vmware-notsecurepassexample). As a bonus you now know where to look when you forget your lab VCSA password ;-).

VCSA 6 Web Client

(And now I notice the vCenter Operations Manager icon in the Web Client Home screen. Why is this not updated like vRealize Orchestrator 🙂 )




Protecting vCenter services, what is around (comes around)

Depending on your environment there is a need to protect vCenter or some of the services included in the vCenter system. A big question to ask yourself is what kind of downtime can you have according to your service levels and what kind of options do we have or need to have in place?

What will go down if you lose a vCenter component?

Like said this depends on your environment and components using vCenter services to connect to and from. A “plain” server virtualization workload for one company is different than a VD workload in a high demanding organization. The latter probably needs the ability to provision a little more urgent then the first example. Want to deploy a vCOPS vApp or VD Desktop, well wait until your vCenter is back. Using solutions like VMware Data Protection requires an operational vCenter with a functioning vCenter Single Sign-On server to restore a virtual machine. Losing that part of your environment could impact your recovery options seriously. Manage/Edit some VM version 10? How will you do that without vSphere Web client? You can’t. Have a HA or DRS cluster? Well HA will still partially function, it will react with restarts when needed. But to add to the cluster will need vCenter to make this posible. DRS needs vCenter to function in manual or automatic mode. And these are just a few examples.
Important to keep in mind, running VM’s will keep on running and HA will keep on HA’ing, no need to panic there.

Let see which components make up vCenter, a little vCenter architecture to start with.

A “standard” vCenter is made up of the components vCenter SSO (Single Sign-on), Lookup Service, Inventory Service, vSphere Web Client and the vCenter Server (with all of it’s services) itself. Optional services are Dump collector, Syslog Collector and Auto Deploy (and optionally TFTP and PXE DHCP service, but they can be on a separate system so not included in the model as a part). vCenter is also expanded by Update Manager, vCOPS and all sorts.


What are your standard protecting options?

  • Do nothing
    Not advisable, but if you are sure, have a small (just a few hosts and VM’s) environment and have an insight of your environment (or use some scripting to dump your configuration), you could do nothing. You lose part of your services and (in worse case) will have to manually rebuild vCenter and your configuration. You will lose any trending information. Recovery time is typically measured in days, and requires manual intervention.
  • Back-up Restore or Replication.
    Backup and restore should be an essential part of any availability solution, exclamation mark. This provides a recovery method utilizing tape, disk, replication or snapshot technology. This also enables a recovery method when data corruption occurs, depending on the solution that is. If data is corrupt on the primary VM then a replication to the recovery VM can occur after this moment. vCenter VM replication from primary to recovery site should be well monitored (and tested with SRM plans for example). Preferably used on several layers, application and application data (for example databases, certificates, logs, dump locations etc.). Be sure to know your backup  and recovery steps (look in the VMware KB’s for backing up the vCenter Server Appliance services and embedded vPostgres database), document, practice and test them. Recovery time is typically measured in hours or days, and typically requires manual intervention.
  • MS SQL Log shipping – database only
    A simple and cost effective solution. You can use log shipping to send transaction logs from one database (the primary database) to another (the secondary database) on a constant basis. Continually backing up the transaction logs from a primary database server and then copying and restoring them to a secondary database server keeps the secondary database nearly synchronized (depending on your plan) with the primary database. The destination server acts as a cold standby or backup server. Your destination server can also act as primary database for other databases so you will have some sort of active-active instead of a cold standby. Be ware of licensing in this case, log shipping target only or serving database is a different license show! Has to be setup for every database, include your vCenter, Inventory, SSO and such. Recovery time is depending on your plan, but can be minutes or hours. Requires manual intervention to fail over from primary to secondary.
  • SQL mirror / clustering – database only
    Depending on the license of MSSQL these are a more robust solution then the previously mentioned SQL log shipping. These have data replication mechanism in place and have the ability to automatically detect failures and do there fail overs with out manual intervention. Mostly used with a Witness out side the cluster/mirror pair to act as a tie breaker to prevent split brain scenario’s in case of partial failures. Mirroring, clustering has to be setup  for every database, include your vCenter, Inventory, SSO and such. Clustering can also be done per instance with it’s included databases. Oracle will have it’s own clustering, with Oracle RAC for example. Recovery time is typically measured in minutes. No intervention to fail over.
  • Hypervisor HA.
    Hypervisor HA will start your VM after a host failure or VMtools timeout. The time it takes to recover is depending on your amount of free slots, your priority of vCenter vs the other workload and the amount of VM’s needed to restart. Depending on your environment this can take some time to start up. Hypervisor HA will not protect against service failures as it will not monitor any application components, it will also not protect against any data corruption. Hypervisor HA is to be used in conjunction with one or more other protection options. For example a vCenter system on HA and SQL databases on MSSQL Cluster. Recovery time is typically measured in minutes or hours depending on your consolidation ratio and restart settings.
  • App Aware HA.
    If you have the correct edition and have the application aware components in place. Monitors the application and if it goes down, it can be restarted. There is no app aware HA specifically for vCenter yet. But you can protect parts of the applications with app HA, for example MSSQL services. Recovery time is typically measured in minutes or hours.
  • FT
    That is currently a no no. Why did I put it up here? Because it comes up as a question once in a while. FT creates virtual machine “pairs” that run in lock step—essentially mirroring the execution state of a virtual machine. This only protects against host or VM failures. Services that go down or corruption in the application data will be mirrored to the secondary VM.
    FT in vSphere 5.5 is still limited to 1 vCPU, and with a small inventory you still need a minimum of 2vCPU. Same goes for for example a database server these also tend to have more vCPU’s. Yes this has been a issue all along for FT, and we know from following those VMworld sessions demo’s that there is work in progress on multiple vCPU FT, but unfortunately this is not yet released. But a similar technique is next up.
  • vCenter Server Heartbeat
    vCenter Server Heartbeat is a separately licensed vCenter Server plug-in that provides protection of your vCenter system, (physical or virtual). Next to protecting against host failures, heartbeat adds application-level monitoring and intelligence of all vCenter Server components. Heartbeat replicates changes to a cloned virtual machine. The cloned virtual machine can take over when a failure event is triggered.
    imageThe vCenter recovery can be accomplished by restarting the vCenter service, by restarting the entire application, or by the entire failover of the vCenter system. Use in conjuction with a data protection like SQL mirroring to protect against corruption. Recovery time is measured in minutes and requires no manual intervention.
  • Scale out / HA service pair
    Move some of your vCenter services to other components or use multiple same role servers to provide high available and load balanced services. Not all of the vCenter services can be separated this way, but for example SSO can be. Those high availability service are placed behind a third-party network load balancer (for example, Apache HTTPD, vCloud Networking and Security vShield Edge load balancer or load balance appliance like Netscaler).
    imageMove logs to a log insight server, move statistics to vCOPS. Keep vCenter lean and mean.


vCenter Server Heartbeat is a done package for protecting your vCenter server system, but this is at an additional cost. More often you will have some back-end services, like Oracle/MSSQL clustering and back-up restore/replication solutions, already in place or products with a similar need. A combination of protection is the preferred way to utilize those in or to be in place solutions with the need for protection and the allowed recovery/down time. But this is the main thing, know your environment, know how the components interact, know what is needed at which time and know what will be (temporary) unavailable when services are down. Protect against unavailability, corruption and please randomly test to be sure all components are working as expected (even the manual procedures).

And yes sure there will be some other great options out there like a script collection or cold standby solution et al….. but hey isn’t that what the comments section is about. Tell me yours. Share.

– Happy managing your environment!

vCenter 5.1 installation on Windows 2012 fails at SPS – ProtectedStorage none existing service

In the middle of building a VMware Horizon Suite 5.2 lab for a comparison demo (VDI, mobility et al) I was installing vCenter 5.1 on Windows 2012. I can almost be sure I have tried an earlier installation that didn’t show this problem, but hay my memory sometimes plays a little trick on me (I’m used to VCSA in my lab environments). Could be Update 1 as vCenter 5.1 is not supported to run on Windows 2012 (and cue warning not for production only U1 is supported on Windows 2012 (and without the R2)). But vCenter 5.1 without update is in my Horizon Suite evaluation, and this is a lab…

Anyhow, with the installation of SSO en inventory service passed I tried to install the vCenter Server. Nothing new nothing fancy. Only at the SPS (the Profile Drive Storage part) the installer stops and redirects me to the logs. Okay… While checking the logs and system logs the following is registered as event id 7000.:


Okay ProtectedStorage. So fire up service manager via and protectedStorage in dependency of VMware VirtualCenter Server. And no ProtectedStorage service to be found (also no service ).


So power up those register skills and check the dependencies of the VPX service. You can find that at ComputrerHKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesvpxd and then DependOnService key.


Here we see ProtectedStorage, next to Workstation service and MSSQL Express (only if you combine this locally). Remove that ProtectedStorage entry and close registry. You will have to restart to server and on boot up the vCenter Server service is started. (Yes really check in your services ;-))

You will have to rerun the vCenter Server installation to complete the required components. It will start where it left off.

– Enjoy labbing!

Dissecting vSphere 5.5 Enhancements – HA improvement and App HA

With the introduction of vSphere 5.5 there are two mayor HA improvements announced:

– vSphere App HA, on the Intarweb also known as App aware HA; High Availability at the application layer.
– vSphere HA detecting VM antiaffinity rules.

I’ll start with the latter.

HA detecting VM Anti-affinty rules

With vSphere DRS… Hey wait isn’t the subject supposed to be HA… Well yes, but the anti- or affinity rules are DRS rules. So a bit of DRS rule explanation;..these rules helps maintain the order of placements of VM’s on hosts throughout the cluster.  Affinity rules are rules that places VM’s together on certain hosts. Anti-affinity rules are rules that places VM’s separate from those VM’s in the rule. Think of VM’s that are already in a software availability service, such as the nodes of a cluster. You don’t typically want the nodes on one physical host.
With vSphere 5.1 and earlier vSphere HA did not detect a violation of these rules (these rules are unknown to vSphere HA). After a HA failover the VM’s could be place on the same host, after vSphere DRS would kick in and vMotion the VM’s so the anti-affinity rules are satisfied (DRS needs to be in full automated to enable the auto vMotion). Applications with high sensitivity to latency would not like this vMotion and there is a (very slight) moment that HA application clustering service are at higher risk as both VM’s are on the same physical host. A failure of the physical host before the vMotion is completed, would impact a downed service.
In a application cluster service you could also choose to use VM Overrides to disable HA restart for the VM cluster nodes as the application service handles the application HA actions. After a failure you would have to manually get the failed node online (or add a new one) in the application service. But that looses automation…

With vSphere 5.5 HA has been enhanced to conform with the anti-affinity rules. In a case of a host failure the VM’s are brought up accordant to the anti-affinity rules without the need of a vMotion action.This enhancement is configured as an advanced option.

vSphere App HA aka App aware HA

We already have host and VM monitoring, with vSphere 5.5 lifts this to application monitoring. vSphere App HA can be configured to restart an application service when an issue is detected with this service. It is possible to monitor applications as IIS, MSSQL, Apache Tomcat and vCenter. When the application service restart fails App HA can also reset the virtual machine. Service actions can be configured with the use of App HA policies. VM monitoring must be enabled to use application monitoring.

App HA Policies are definitions of the number of times vSphere App HA will attempt to restart a service, the number of minutes it will wait for the service to start, and the options to reset the virtual machine if the service fails to start and to reset the virtual machine when the service is unresponsive. They can also be configured to use other triggers, such as e-mail notifications or vCenter alerts.

When a configured App HA policy is assigned to a specific application service, vSphere App HA is enabled for that service.

Pretty nice.

But what’s needed:

For App HA to work two appliances are needed in the environment (per vCenter), vSphere App HA and vFabric Hyperic. The latter is used by the App HA architecture to monitor applications and is a vFabric Hyperic Server that communicates with vFabric Hyperic agents.
The roles of the both appliances are as follow: the vSphere App HA virtual appliance stores and manages vSphere App HA policies. The vFabric Hyperic appliance monitors applications and enforces the assigned vSphere App HA policies. For monitoring the applications of a VM, vFabric Hyperic agents must be installed inside the VM’s of these applications. These agents are communication brokers for the applications of the VM’s and the vFabric Hyperic appliance.

The vFabric Hyperic agents are supported to be deployed at Linux and Windows os’ses for 32-bit or 64-bit applications. How and what is supported for vSphere 5.5 HA is not yet completely clear (service support for IIS6/7/8, MSSQL 2005/2008/2012, Apache Tomcat, Apache HTTP and vCenter). Following the current vFabric Suite supported OS’ses these include Windows 2003, Windows 2008R2, Red Hat Enterprise Server and Suse Enterprise Linux.



Well. Good Question. App HA is part of the vSphere Enterprise plus edition only. Costs of vSphere 5.5 is expected to be around the current vSphere 5.1 costs. But with what options, constrains and limits…..unknown. The General Availability of vSphere 5.5 is yet unknown.

Separately VMware vFabric Suite is currently available as a one-time perpetual license under which support and subscription (SnS) contracts can be renewed annually – See more at:

How the both are combined at what options/editions/prices keep a look out for further vSphere 5.5 product announcements.

– Exiting. I have the HA BCO5047 – vSphere High Availability – What’s New and Best Practices in my Barcelona schedule to get some more insight at VMworld EU 2013.

Dissecting vSphere 5.5 Enhancements – vCenter Server Appliance ready for lift off

A real concern for implementing the vCenter Server appliance (vCSA) 5.1 in production environments is the environment limits with the use of the embedded database. The embedded database with 5.1 has a limit of 5 managed hosts and 50 managed VM’s. That is not the size of production in a lot of setups.
The maximums could be lifted, but needed an external database service that was limited to only Oracle (and with the 5.5 version still is limited to Oracle as an external database). Not a lot of organizations use Oracle for infrastructure services, so that option is not widely used (or at least haven’t seen that around much). I’ve only used the vCSA in lab / testing environments.

With the introduction of vCSA 5.5 one of the most important parts is a reengineered embedded database (vPostgres). This lifts the configuration maximums of the vCSA to (editted, still production worth though) 100 managed hosts and 3000 managed VM’s. Well that’s more like it, this is production worth.

Table with specification vCSA 5.1 vs vCSA 5.5.

*) Specifications of the minimum requirements are depending on your environment. Don’t expect a difference approach, 2vCPU and memory according to your environment size. Scale down or scale up the standards.

The 5.5 release makes vCSA a production appliance. It is deployed fast and in comparison to a vCenter server does not need a Windows license.

– I’m gonna see a lot more of the vCSA 5.5 in customer environments.

— This post has been edited to change the number of managed hosts to 100 and managed VM’s to 3000. Earlier numbers were unofficial.