vSAN Troubleshooting Tools

Before you begin it is important to realize that vSAN is a software based storage product that is entirely dependent on the proper functioning underlying hardware components, like network, storage I/O controller and the individual storage devices. You always need to follow the vSAN Compatibility Guide for all deployments. I often play with things in my homelab, just because you CAN do it, doesn’t mean it’s supported by VMware. 🙂

I’ve recently been playing with stretched clusters in my homelab and have finally had to fix some of the stuff I’ve been playing with. The beauty of the homelab, is I can break anything I want! I had not worked with the Ruby Console at all but it’s an amazing tool to check out.

Many vSAN errors can be traced back to faulty VMkernel ports, mismatched MTU sizes, etc. It’s far more than simple TCP/IP.

Some of the tools you can use to troubleshoot vSAN are:

  • vSphere Web Client
    • The primary tool to troubleshoot vSAN.
    • Provides overviews of individual virtual machine performance.
    • Can inspect underlying disk devices and how they are being used by vSAN.
  • esxcli vsan
    • Get information and manage the vSAN cluster.
    • Clear vSAN network configuration.
    • Verify which VMkernel network adapters are used for vSAN communication.
    • List the vSAN storage configuration.
  • Ruby vSphere Console
    • Fully implemented since vSphere 5.5
    • Commands to apply licenses, check limits, check state, change auto claim mechanisms, etc.
    • See this link for a great introduction to RVC.
  • vSAN Observer
    • This tool is included within the Ruby vSphere Console.
    • Can be used for performance troubleshooting and examined from many different metrics like CPU, Memory or disks.
  • Third Party Tools

I also highly recommend checking out the vSAN troubleshooting whitepaper the Cormac Hogan has written:

vsan-troubleshooting-reference-manual.pdf

Posted in Troubleshooting, vSAN | Leave a comment

vCenter Server HA Deployment/Design Considerations

Check out fellow Indy VMUGer, Adam Eckerle and his great 13 minute vCenter HA discussion on vSphere 6.5. I love the whiteboard Adam!

Posted in HA, vSphere 6.5 | Leave a comment

vSphere Encrypted vMotion Architecture, Performance, and Best Practices

With the rise in popularity of hybrid cloud computing, where VM-sensitive data leaves the traditional IT environment and traverses over the public networks, IT administrators and architects need a simple and secure way to protect critical VM data that traverses across clouds and over long distances.

The Encrypted vMotion feature available in VMware vSphere 6.5 addresses this challenge by introducing a software approach that provides end-to-end encryption for vMotion network traffic. The feature encrypts all the vMotion data inside the vmkernel by using the most widely used AES-GCM encryption standards, and thereby provides data confidentiality, integrity, and authenticity even if vMotion traffic traverses untrusted network links.

Download Whitepaper

Posted in Uncategorized | Leave a comment

vSAN Deepdive videos from VMware Storage

Learn more about vSAN’s value propositions, use cases and customers in these great educational videos.

Posted in Storage, vSAN | Leave a comment

Product Review: Vembu BDR Suite v3.6.0

One of my partners, Vembu, recently approached me to review their new product called Vembu BDR Suite v3.6.0. For those that are unaware, Vembu has been around for awhile (2002) but only recently been inside the VM backup scene. Vembu BDR supports a lot of options for backup. Physical Servers, Hyper-V, desktops, Microsoft Exchange, Office 365, Google Apps, VMware vSphere, SharePoint to name a few. You can backup offsite, onsite or to a cloud like Amazon Web Services.

For this product review, I will document the install process for vSphere. I will be running the tests in my home lab, The lab consists of 3 hosts, 20-25 virtual machines and various storage options whether it be shared or local disks. My primary device for storage is a Synology DS1515. So I’ll be using that to backup my VMs to.

I started by downloading the evaluation copy from here. I will use the Installation Guide posted here.

Installation:

The install was a fairly straightforward process. I installed it on a Windows Server 2008 R2 machine I had running in my lab.

It popped up and asked to install the following prerequisite software.

  • MySQL RDMBS (5.5.40)
  • MySQL Connector (5.2.4)
  • MongoDB (3.2.0)

After clicking next, you can take the defaults or customize the configurations.

After clicking install, it took about 5 minutes and started the services. The installation was a very simple process.

First Login and Backup:

Upon logging into the web portal you are greeted with a configure time zone page and then a page to create a unique ID. Once you configure those options it send you to the product dashboard page. The page was responsive and you can see the various options you can use. I am testing the vSphere functionality so I will add my vSphere 6.5 environment to Vembu.

Once installed browse to the management webpage at https://fqdn:6061.

The default username and password was admin/admin.

Select a time zone for the deployment.

Entering a unique BDR ID.

The Vembu Dashboard.

Adding in my vSphere credentials to Vembu.

After entering in my vSphere credentials, it found all of the hosts in my environment. You can select the VM you’d like to backup and then setup backup frequencies, application aware backups, and additional full backups.

Additional Full Backup options.

Application Aware and GFS Retention Advanced Settings:

The Application and GFS Policy settings are great features. Application awareness helps to optimize the work structure and efficiency of the sub-system in control. Due to this application aware backup is needed in most workloads today.

Application aware backup are used to:

  • Improve RPOs.
  • Maintain consitensy with application aware backups.
  • Save storage space.

Some benefits of the GFS Retention Policies:

  • Reduce the time taken to restore backed up machines and most importantly reduce the size of merged image files in the storage location.
  • Avoid long chains of incremental backups, ensuring safety of backup data and allows you to meet the requirements of your retention policy. Additionally, enabling retention option will remove the requirement for additional full backups.

Check out this post on the Vembu blog for more info in GFS.

Once you select the options you’d like you submit the job. I went to grab a bit to eat and 15 minutes later the job had finished. The backups completed successfully.

Virtual Machine Recovery:

Next up we’ll try to restore the VM we backed up in the prior task. Under the recovery options you have to select from a few restore options.

It is also possible to replicate VMs under the same console, this is configured under Replication.

Under Management you can do things like add storage, add offsite backups along with updating the software.

Selecting which version to restore.

Restoring the VM to vCenter, you can select a new VM name, destination datastore and host.

During the recovery process:

My new RecoveryTEST virtual machine came up with a VM MAC Conflict so the backup pulled that information from my old VM as well which is good to see. One of my minor annoyances happens to be machines that aren’t exactly the same when it comes to backups! Now if you’re planning to run this on the same network as the original, tread carefully here. That will need to be addressed before power on.

In conclusion:

The product itself installed easily and can do many different features within a single console. It’s a powerful backup suite of software. The backup market has a lot of different options available to you but rarely do they have an all in one console like this.

My lab runs on desktop grade hardware and performance was great. The backups and restores all completed quickly with no issues.

Their product pricing is very competitive for this industry.

If you would like to try out the product, register for a free trial at the address below.

https://www.vembu.com/vembu-bdr-suite-download/

Posted in Backup, Product Reviews | Leave a comment

ESXi 6.5 Home Lab / Whitebox Options

I’ve had several people ask if it’s possible to put together a low cost option to build up a VMware ESXi 6.5 white box. I’ve been a long time promoter of the Intel NUCs and the small form factor Shuttle units. Now, the Intel NUCs will run ESXi 6 out of the box, but others have just wondered if there are any other options available. Those boxes still run vSphere 6 perfectly but tend to be a slightly higher price point per dollar. It is possible to put together a cheaper host and I figure I’d look through some various hardware and build out 3 different setups.

As always, if you have other hardware, please share your home lab setups in the comments section!

Goals:

  • Maintain control of costs, build a capable ESXi 6.x host for practice/lab use.
  • 32GB memory.
  • 8 Cores
  • At least 2 network cards.
  • Be able to install ESXi 6.x, right out of the box. No need to inject .vibs or make a custom ESXi image.
  • All builds will install ESXi to the USB stick. No other storage or vSAN used. For a good all around NAS, a Synology DS1515+ is a great unit. Everyone wants to run different types of storage. Build up an NFS target out of an old server if you want! Add a 1TB local disk for approximately $50. There’s just too many options!

Build #1 – Intel Core i7 ASRock Build

A quick note about this build. You could also swap out the i7 for an Intel Xeon E3-1231 V3. Just note that you’ll also have to add an additional graphics adapter because the Xeon does not have integrated graphics. The EVGA GeForce 210 Passive 1024 MB DDR3 PCI Express 2.0 DVI/HDMI/VGA Graphics Card is a good value choice adapter. As of this writing, it’s a price difference of about $30, the i7 being the more expensive option.

Subtotal with HP Quad Port (As of 12.5.2016): $683.97

Subtotal with HP Dual Port (As of 12.5.2016): $669.02

Build #2 – ASRock C2750 Intel Avoton C2750 8 Core Build

This is a unique build because the CPU, storage and network adapters are built into the motherboard and all work out of the box with ESXi 6.x. It’s a low power build as well (approx. 20 watts), so similar to a Shuttle or NUC but has IPMI AND supports up to 64GB memory! The only caveat was that this board needed good airflow, so we’ll be using a larger case.

Subtotal (As of 12.5.2016): $742.76

Build #3 – ASRock AMD AM3+ 8 Core Build

It’s generally known that AMD whitebox builds will use more electricity over an Intel build. You just have to determine how important that is to you. You also typically run into problems with onboard network adapters and drivers out of the box with ESXi. With this build we will add an HP Quad Port gigabit network adapter that will work out of the box to give you a total of 4 network adapter with no driver modifications, however this board uses the Realtek 8111E, and you can follow this guide to add that driver to your ESXi install media. The AMD CPUs will need a graphics adapter as well, so we’ll use a common GeForce 210.

Subtotal (As of 12.5.2016): $469.87

Posted in Home Lab, vSphere 6.5, Whitebox | 6 Comments