You want to train/learn on vCloud Director, test multiple scenarios, develop an application using the vCloud API on the latest VMware cloud product ?
Here is a overall guide to build a complete environment which will allow you to test many features of the product.
Take into account that it’s not meant to be a “light” portable lab (It was built for my everyday use/tests, and such, and even though, you will be able to tune some items like activating/disabling some VMware Clusters & attached Provider vDCs (PvDCs) on the vCloud Director side)
It will include lots of Screenshots of the overall vCloud setup, but I didn’t explain the complete vSphere steps as you can find many on Internet already, so if you are missing something, or cannot get it to work, use the comments for questions :-)
Here is how the vApp looks on my setup, lets see how did I setup everything.
Generally depending on my work I have often 2 vLabs running (1.0, 1.5, 2 x 1.5, etc), and as it’s so simple to deploy, so why should we restrain ourselves ? ;-)
I’m using a Dell T410 for my setup (I clone multiple environment to test VPN features, hybrid clouds setup and do my overall designs/test/crash tests, troubleshooting, vCloud Api code snippets, etc…)
You will need a fair amount of memory to use this lab with all 6x ESXis loaded :
3 x PvDCs : (Representing my different used allocation models : PAYG / Allocation / Reservation)
Here is how the vSphere setup looks like from the vCenter within the vApp.
The vApp setup with many nested ESXi, you will need to ensure your host is properly configured to allow those VMs to run.
Here is the logical diagram of my vApp Setup. (hope this helps to understand each component and it’s inter connectivity)
Reminder : Many best practices are not respected in this design, it was setup for my needs & enables many of my tests, feel free to modify and improve it !
I used 3 differents portgroups for this setup (to simulate different networks) :
- The “External Network” that it my local lab subnet, I can access it from anywhere, and it has access to internet through NAT. (only my vCenter VM, and the dedicated external Nics for the nested ESXi)
- The “vCD Management” portgroup used for all my lab management (ESXi, vCenter internal, vShield, vCD cell(s))
- The “vCDNI Network” portgroup that is used for my unique network pool on this setup (Network Pool portgroup).
Overall VMs Setup
1) vCenter VM
This Virtual Machine hosts multiples applications, the first obvious one is vCenter5, vSphere Client.
It is based on Windows 2008 R2 SP1 x64 with MS SQL Server 2008 R2 Standard Edition.
Some other useful services I recommend are :
- Routing Services
For this specific vApp Lab, I used static IPs, that’s why I didn’t use any DHCP Server.
DNS configuration (Forward Lookup Zones) :
- ESXi hosts
- vCD Cell(s)
- vShield Manager
As the vCenter VM is a Windows 2008 R2 SP1 x64, I’ve added the Routing Services to have a simple access to my isolated environment from the vCenter IP.
The vCenter VM acts as a NAT router, and has 2/3 port forwarding rules :
- HTTPS (TCP 443) to internal static IP of my vCD Cell HTTP vNic (vCloud Director Portal, you could be able to use another port if wanted, but I only connect to the vCenter directly from within an RDP session to that VM)
- SSH (TCP 22) to internal static IP of my vCD Cell HTTP Nic (To have the ability to quickly logon to the cell)
Reminder : This is a vCloud 1.5 Lab, so I used SQL Server, if you are building a 1.x lab, you will need to use an Oracle XE Database that is really simple to setup.
Here are the simple steps to create a database for vCloud Director using MS SQL Server 2008 R2 Standard Edition :
Connect to SQL Management Studio and create a new database with a SQL user that owns it (vcloud in my case).
Here I changed the recovery settings to “Simple“, (just a simple vLab I don’t care about full recovery, I’ll just deploy a new vApp if something goes really wrong)You might want to have the following collation settings, I saw some errors if using some other “funky” settings…
Media ISOs / vApp OVF templates
I’ve added mainly some very small iso files to do my testing, usually Turnkey linux core appliance, and very small vApp (OVF) based on that iso is very useful for fast testing once deployed.
Other tools :
- PuTTY (To connect on the vCloud Director Cell)
- mTAIL (To check the logs coming to your vCenter5 Syslog server)
- Notepad++ (a useful free text editor)
- Java 1.6 (for the upload/download OVF applet)
- Latest Flash version for the vCloud Director UI.
2) ESXi VMs (2 for each PvDC)
I usually deploy 6 x ESXi, with 2 vCPUs and 4Go of ram, 6 x 1Gb vNics [e1000 adapter] each (Yes I know, this may be a lot for a “small setup”)
One thing to notice is when using the vApp bootorder properties you can easily disable or not allow some of ESXi hosts to boot, avoiding eating up more memory then you NEED/HAVE, like in the case you want to work on only 2 x hosts and 1 x PvDC.
3) vShield Manager
vShield Manager is distributed as an Appliance using the OVA format (It will take care of managing the DHCP, Firewall, NAT rules and the VPN features)
The installation process is fairly straitforward, just deploy it into your newly created vApp using the vSphere Client.
- Login using the default credentials (admin/default)
- Elevate privileges using “en” / “enable” command
- Type the Password once more : “default“
- Launch “setup” to go through the network configuration setup
- Login to vShield Manager through a Web Browser (admin/default)
- Select “Settings & Reports“, then enter your vCenter Credentials (the one within your vApp)
- Save (This should take a while, and then on the left pane, you should be able to browse your vCenter Inventory if everything went well)
- Register the vShield Manager vSphere plugin to vCenter.
Log to your vCenter using the vSphere Client, and check the licensing, you should be able to input your vShield Licenses (if you don’t, you will not be able to deploy any vSE device and use the DHCP/Firewall/NAT/VPN features)
3) vCD Cell(s)
I’ve created a basic CentOS Linux ISO to help installing a bare minimum linux “nearly” supported distribution (only Red Hat Enterprise Linux 5.4, 5.5, 5.6 x86_64 is supported at the moment)
Information about my CentOS build
Here is the kickstart I’m using for a vCD Cell build, you can use it to create your own customized RHEL/CentOS ISO.
# I would get killed by my colleagues if I use "French" keyboard here ;-)
# Means during installation will query for network information
# depending on number of nics available.
# Setup root password
rootpw --iscrypted $1$jdxakj8G$JgogpcYE4lrL/5PSC98qi0
authconfig --enableshadow --enablemd5
# Disable Firewall & SELinux
timezone --utc Europe/London
# mbr / disk partitions
bootloader --location=mbr --driveorder=sda --append="rhgb quiet"
clearpart --all --drives=sda
part /boot --fstype ext3 --size=100 --ondisk=sda
part swap --fstype swap --size=2048 --ondisk=sda
part / --fstype ext3 --size=1 --ondisk=sda --grow
%packages --nobase --excludedocs
# pre-requisites packages for vCD
# 3rd party vmware tools from http://www.vmware.com/download/packages.html
# disable SELinux (Disabled with my own method)
# sometimes I had issues with selinux --disabled, so this is the "hard" way.
sed -i -e 's/\(^SELINUX=\)enforcing$/\1disabled/' /etc/selinux/config
My custom CentOS build is about 480Mo, and takes into account Oracle or vCD Cell kickstarts (not same pre-requisites and just installing the minimal packages)
I’ve only shown the vCD kickstart, as MS Sql Server is used for this setup.
HTTP / ConsoleProxy
My cell networking setup :
- eth0 is used for HTTP (192.168.10.21)
- eth1 is used for ConsoleProxy (192.168.10.22)
- default gateway set to vCenter (192.168.10.10)
- DNS server set to vCenter (192.168.10.10)
Once the vCD Cell is installed, I’ve added 2 additional vDisk to it (2 x 100GB, this can be done as an hotplug operation) to use it as a simple NFS Datastore.
Check on linux the availability of 2 news disks using the following command :
You should be able to see new “/dev/sdb“, “/dev/sdc” devices with 100GB size each.
Create Partition/Filesystem on them using the following procedure :
For each disk :
I’ve stripped down the output of many commands, otherwise this step would be HUGE.
Then create a partition with the following commands :
p primary partition (1-4)
Partition number (1-4): 1
First cylinder (1-13054, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-13054, default 13054):
Using default value 13054
Use “p” to display what you create, and once you are sure about it, press “w” to save the changes.
The partition table has been altered!
Calling ioctl() to re-read partition table.
Create a filesystem on the newly created partition.
mke2fs 1.41.3 (12-Oct-2008)
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
6553600 inodes, 26214055 blocks
1310702 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
800 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 23 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
Now you will need to mount this partition into someplace, here is a very quick guide :
Edit “/etc/fstab” and append the following line :
To check if it’s working properly execute the following command :
If you get no errors then you are good to go.
You will need to install the nfs-utils package to be able to create a NFS export.
If you used the Routing Service/NAT on you vCenter VM, you should be able to access internet if your setup allows it.
Ensure name resolution is working properly for inside the vLab setup (*.vlab.loc) and externally, if it doesn’t check that you have something similar to this depending on your DNS setup :
Then execute the following commands :
Ensure the correct services will be starting for next reboots, and start them now
# chkconfig nfslock on
# chkconfig nfs on
# service portmap start
# service nfslock start
# service nfs start
This should enable all services are up & running.
Therefore we need to configure the NFS export now, edit the “/etc/exportfs” file and add the following line :
Reminder: “async” is a option that can drastically speed up nfs operations but can lead to corrupted data, and as it’s “only” a lab I don’t mind using this option.
Once you did this operation for every vDisk you added to your vCD Cell VM, you should be able to see something similar as the screenshot below.
Restart the nfs service to take into account the brand new shares :
Shutting down NFS mountd: [ OK ]
Shutting down NFS daemon: [ OK ]
Shutting down NFS services: [ OK ]
Starting NFS services: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
Now add the newly created datastores to all your nested ESXi hosts (they will be used as your shared storage)
Setting up vCloud Director installation
Once you copied/transferred the vCloud binary installation file, be sure to sure to set the execute bit on it (chmod +x), and then execute it.
When you arrive to the point, where it asks to execute the configuration script say “n” if you didn’t already create your SSL certificate keystore.
This is a mandatory step for the configuration process, so just follow the vCloud Director installation & configuration document that explains well the process of creating them.
Here is the steps to create self-signed certificates (for a lab this is more than enough)
-storetype JCEKS -storepass passwd -genkey -keyalg RSA -alias http
Answer all the questions.
-storetype JCEKS -storepass passwd -genkey -keyalg RSA -alias consoleproxy
Answer all the questions.
Verify your keystore
-keystore certificates.ks -list
During the Installation Wizard it will prompt for you network IP adresses, and then your keystore with your SSL certificates.
Put your certificates in someplace readable by the account that will be created from the installation process (vcloud:vcloud)
Here is a simple way to do it :
# chmod 755 /tmp/certificates.ks
vCloud Director initial configuration
When the certificate keystore is ready you will be able to resume to configuration using the following command :
- Prompts for the 2 IPs for HTTP service, and the console proxy.
- Prompts for the certificates keystore
- Prompts for the syslog server (I used the embedded vCenter5 syslog server)
- Prompts for the vCloud Director database (used MS SQL server)
Now configuration will test the Database connection, and if successful, injects the default schema into it.
It will remind you the public portal IP for the “first-time wizard“, and ask you if it should start the vCloud Director service, answer “y“.
The service takes a while to load, you can check it status through the following command :
Once you get the highlighted section in the bottom of the below screenshot, that means vCloud Director service is up & running.
Clean the temporary keystore location files (it has been imported by vCloud Director in another location)
Reminder : This keystore will be needed if you are creating a multi-cell setup, so make a backup of this file somewhere safe.
Initial vCloud Director Setup Wizard
Go to your HTTP IP address using a browser (https://192.168.10.21), on the below screenshot I was checking something on SSL certificates, so I’ve used my domain names & personal certificates)
You should get the following screen, just click on “Next“.
Here is the license agreement step, have a nice read, and then select “Yes, I accept the terms in the license agreement“, and click “Next“.
Use your evaluation/final license to pass this step, and click “Next“.
A very important step, that will define your initial vCloud local Administrator account, please set it up carefully, then click on “Next”
This step asks you for a System Name & an Installation ID, that will avoid multiple vCloud Director instances to generate same MAC address on your VMs, once done click on “Next”
At least for Part 1, I’ll be explaining the vCloud Director post install setup/settings in more details with some use cases in Part 2.
Leave some comments if you are having trouble following this basic guide to create a vApp containing a vCloud (vCloud in a vApp !)