Report Changes to DRS/HA settings

Do your vCenter settings get changed when they shouldn't?  This script will email an alert when anything has changed.  

First you need to run once and it'll create "E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-File.txt" and copy that file to "E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-Baseline.txt".

From then on each time the script runs (schedule in Windows Task Scheduler), a new GetCluster-File is created and compared to the baseline file.

___________________________________________________________________

# created by KC on 2015/06/04

#

#

#pre-reqs are:

# 1.  set vCenter server for Connect command

# 2.  verify/create directory/file

# E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-Baseline.txt (Cam)

# F:\scripts\PowerCLI\logs\HA-DRS-GetCluster-Baseline.txt(Basel)

#

#

Add-PSSnapin VMware.VimAutomation.Core

Connect-VIServer vSphere

#

# after valid change to HA/DRS settings in vSphere, copy

# E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-File.txt

# overwriting and replacing:

# E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-Baseline.txt

# to prevent alerts by this script

#

#get email variables from vCenter 

#

$vCenterSettings = Get-View -Id 'OptionManager-VpxSettings'

$emailFrom = ($vCenterSettings.Setting | Where-Object { $_.Key -eq "mail.sender"}).Value

$smtpServer = ($vCenterSettings.Setting | Where-Object { $_.Key -eq "mail.smtp.server"}).Value

#set email variables manaully

$emailRcpt = "email@email.com"

$MailSendingTo = "email@email.com"

#

#set variables for baseline file and new cluster-file

#

$GetClusterFile =  'E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-File.txt'

$GetClusterBaseline = 'E:\scripts\PowerCLI\logs\HA-DRS-GetCluster-Baseline.txt'

$GetClusterMail = 'E:\scripts\PowerCLI\logs\HA-DRS-GetClusterMail.txt'

#

#

#collect cluster config info and send to screen

#

Get-Cluster | Sort-Object -Property Name | 

Select-Object -Property Name,HAEnabled,HAAdmissionControlEnabled,

@{N="AdmissionControlPolicy";E={$_.ExtensionData.Configuration.Dasconfig.AdmissionControlPolicy.GetType().Name}},

DrsEnabled,DrsMode,DrsAutomationLevel

#

#

#collect cluster config info and send to file

#

Get-Cluster | Sort-Object -Property Name | 

Select-Object -Property Name,HAEnabled,HAAdmissionControlEnabled,

@{N="AdmissionControlPolicy";E={$_.ExtensionData.Configuration.Dasconfig.AdmissionControlPolicy.GetType().Name}},

DrsEnabled,DrsMode,DrsAutomationLevel | out-file $GetClusterFile

#

# Get remaining cluster info, send to screen

#

Get-Cluster | Select-Object -Property Name,

@{N="CpuFailoverResourcesPercentage";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.CpuFailoverResourcesPercent}},

@{N="MemoryFailoverResourcesPercentage";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.MemoryFailoverResourcesPercent}} 

#

# Get remaining cluster info, appending to file

#

Get-Cluster | Select-Object -Property Name,

@{N="CpuFailoverResourcesPercentage";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.CpuFailoverResourcesPercent}},

@{N="MemoryFailoverResourcesPercentage";E={$_.ExtensionData.Configuration.DasConfig.AdmissionControlPolicy.MemoryFailoverResourcesPercent}} >> $GetClusterFile

#

# load GetCluster info into variable for email later

$hadrs_getclusterfile = Get-Content $GetClusterFile

#$hadrs_baselinefile = Get-Content $GetClusterBaseline

#

#

$Report = Compare-Object -Ref (Get-Content $GetClusterFile) -Diff (Get-Content $GetClusterBaseline) -Verbose | Out-String 

write-output $Report $hadrs_getclusterfile > $GetClusterMail

#

$Same_Or_Diff = Compare-Object -Ref (Get-Content $GetClusterFile) -Diff (Get-Content $GetClusterBaseline) | Measure | select Count

#

#

$z = $Same_Or_Diff.Count

if ( $z -eq 0)

{

write-host $Same_Or_Diff 

write-host "is Same_Or_Diff"

write-host $z 

write-host "is Count of differences, should be equal to 0" 

write-host "which means no change to HA/DRS config at" 

#$Report = "$MailSender HA/DRS config is unchanged, no action needed for $MailSender"

$MailSender 

$Report = (Get-Content $GetClusterFile | out-string)

#commented email so we only get alerts for problems

#email results

#Send-MailMessage -from $emailFrom -to $emailRcpt -subject "HA/DRS Config Alert OK - not changed" -body $Report -smtpServer $SmtpServer

}

#

else

#

{

#

write-host $Same_Or_Diff 

write-host "is Same_Or_Diff"

write-host $z 

write-host "is Count of differences, should not be equal to 0" 

write-host "A change to HA/DRS config at" $MailSender "needs investigation" >> $GetClusterMail

write-host $Report

write-host is email body

#

#

#email results

$Report = (Get-Content $GetClusterMail | out-string)

#Send-MailMessage -from $MailSender -to kevin.cade@napp.co.uk -subject "Alert: HA/DRS Change Config Change Changed" -body $Report -smtpServer $MailSmtpServer

#prepare email

$emailSubject = "Alert: HA/DRS Change Config Changed!" 

$msgBody = $Report

$smtp = New-Object Net.Mail.SmtpClient -arg $smtpServer

#send email

$smtp.Send($emailFrom,$emailRcpt,$emailSubject,$msgBody)

Send-MailMessage -from $emailFrom -to $emailRcpt -subject "HA/DRS Config Alert changed!" -body $Report -smtpServer $SmtpServer

}

disConnect-VIServer vSphere -confirm:$false

PowerCLI VM Host & DataStore affinity

We name our datastores with a three letter prefix which is the same three first letters of our hosts.  Folks who create VMs often forget to create the DRS rules so this script can be scheduled to run via Task Scheduler to alert us of VMs that need fixing:

____________________________________

### add snapin so PowerShell runs with PoweCLI features

Add-PSSnapin VMware.VimAutomation.Core

Connect-VIServer vSphere

#

$vms = Get-View -ViewType VirtualMachine -Property Name,Datastore,"Runtime.Host",  "Runtime.PowerState" 

### exclude Microsoft SQL Cluster VMs with RDMs that report disks from both sites 

$emailbody = foreach($vm in $vms){ 

    if ($vm.Name -ne "ClusterVM1" -and $vm.Name -ne "ClusterVM2") 

    {

### only report on VMs that are powered on

       if ($vm.Runtime.PowerState -eq "poweredOn") 

       {

### exclude hosts as needed

        $esx = Get-View $vm.Runtime.Host -Property Name

        if ($esx.Name.Split('.')[0] -ne "vh99" )

        {

            $ds = Get-View $vm.Datastore -Property Name

### report those datastores that don't match first three letters of hostname with first three letters of datastore name

            if ($ds | where {$_.Name.Substring(0,2) -notmatch $esx.Name.Substring(0,2)})

            

            {

                      

                       $vm | Select Name,

                        @{N="Host";E={$esx.Name.Split('.')[0]}},

                        @{N="DataStore";E={[string]::Join(',',($ds | %{$_.Name}))}} | out-string

               

            }

        }

        }

    }

} 

### for email alert, get mail info from vCenter database

$vCenterSettings = Get-View -Id 'OptionManager-VpxSettings'

$MailSender = ($vCenterSettings.Setting | Where-Object { $_.Key -eq "mail.sender"}).Value

$MailSmtpServer = ($vCenterSettings.Setting | Where-Object { $_.Key -eq "mail.smtp.server"}).Value

$MailSendingTo = "email@email.com"

### test for VMs with running on wrong host according to datastore

if ($emailbody) 

{

write-host "These VMs are misconfigured and will fail during an outage of either data center, and they will fail with an outage of the link between data centres.  Fix the VMware DRS rule on $MailSender so these VMs run on host and disks in the same data center.  These VMs with disks and hosts spanning two locations are at risk of needless loss of service to our customers:

$emailbody"

$emailbody2 = "These VMs are misconfigured and will fail during an outage of either data center, and they will fail with an outage of the link between data centres.  Fix the VMware DRS rule on $MailSender so these VMs run on host and disks in the same data center.  These VMs with disks and hosts spanning two locations are at risk of needless loss of service to our customers:

$emailbody"

Send-MailMessage -from $MailSender -to $MailSendingTo -subject "alert: VM Host/Disk affinity violations" -body $emailbody2 -smtpServer $MailSmtpServer

}

else

{ 

write-host "ok: host/datastore affinity is correct for all VMs in $MailSender" 

$emailbody2 = "host/datastore affinity is correct for all VMs in $MailSender"

Send-MailMessage -from $MailSender -to $MailSendingTo -subject "ok: no VM Host/Disk affinity violations" -body $emailbody2 -smtpServer $MailSmtpServer

}

disConnect-VIServer vSphere -confirm:$false

________________________________________________

Thanks to LucD for doing the real work of this script.

verifying reload of syslog server

This post is gonna be a bit messy.  I'm trying (again) to start learning about PowerCLI so there are more things that I don't know than things that I do.  No worries, post comments and help me as all I know is Bourne Shell scripting.  (^;

I'm checking my syslog settigns are all consistent and correct.  Of course we can check and change them from vCenter (or new vCenter web client--shudder!), but more than a few hosts are best done by scripts/cli.

I can check this from PowerVCLI:

get-vmhost | Get-VMHostAdvancedConfiguration -Name Syslog.global.logHost

or specific hosts:

Get-VMHostAdvancedConfiguration -Name Syslog.global.logHost -VMHost host1


Name                           Value
----                           -----
Syslog.global.logHost          udp://1.2.3.4:514

and I can even fix it with a script using these commands

Set-VMHostAdvancedConfiguration -Name Syslog.global.logHost -Value 'udp://1.1.1.1:514' -VMHost host1

$esxcli = Get-EsxCli -VMHost host1
    $esxcli.system.syslog.reload()

thanks to this cool script kindly provided by Caleb here.

So, the reloading of the syslog service.  This isn't the usual linux service there's a command for reloading:
from the ESXi host (ssh/putty/DUI session):

~ # esxcli system syslog reload

the VMware kbase article(s) say to check the syslog is running with the good ol' linux ps command:

~ # ps | grep vmsyslogd

8666 8666 vmsyslogd            /bin/python

8667 8666 vmsyslogd            /bin/python

8668 8666 vmsyslogd            /bin/python

But how do you verify whether either of these methods have worked (other than the absence of error messages?).  

Seems /var/log/hostd.log is where this is recorded:

reload issued via PowerCLI:

2014-02-19T11:39:14.045Z [32B81B90 info 'Solo.VmwareCLI'] Dispatch reload

2014-02-19T11:39:14.114Z [32B81B90 info 'Solo.VmwareCLI'] Dispatch reload done

reload issued directly from ESXi linux login session:
2014-02-19T11:52:30.677Z [2A9DEB90 verbose 'Hostsvc.SyslogConfigProvider'] Running '/sbin/localcli system syslog config logger list'
2014-02-19T11:52:30.678Z [2A9DEB90 info 'SysCommandPosix'] ForkExec(/sbin/localcli)  393025
2014-02-19T11:52:30.830Z [2A9DEB90 verbose 'Hostsvc'] Received advanced config change notification

Can you shed any light on this?

Join the discussion and add your perspective.   Thanks, KC

NetApp vol copy of snapshot for vSphere

Using Data from a NetApp snapshot via vol copy

make sure the destination volume is not being used,

if mapped to VMware delete VMFS and unmap and rescan from vCenter

offline the volume

verify the snapshot needed with snap list command

vol copy -s start snapshot_wanted source_vol_with_snap dest_vol_alredy_offlined

online the destination volume and you'll see message from NetApp CLI warning that lun has been offlined as it is copy and therefore has duplicate information as the source:

Fri Jan 13 14:51:52 GMT [SnapVaultA: lun.newLocation.offline:warning]: LUN /vol/sv_lun_svcopy/sv1/lun96 has been

taken offline to prevent map conflicts after a copy or move operation.

go to FilerView and change the path and lun number of the lun for the dest. volume.

Now you can online the volume and map it out with the correct, non-duplicate, lun number.

rescan storage on vCenter

add Storage and choose "Assign a new signature" when adding dest. lun into vCenter.

find the VMFS on your dest. lun which will have it's old name but after prefix "snap-xxxxx" and rename it appropriately.

add the VMDK on the VMFS on the lun to your VM and you're laughing!

Resetting Lost ESXi password

There's one blog post I've found about booting your ESXi host to a bootable Linux CD and hacking the root password, but VMware says the only supported solution is to do a recovery install.

The nice folks at VCE have suggested something else:

You need the full Enterprise license in order to use host profiles for this fix. We did this on ESXi 4.1 update 1 with Virtual Distributed switches and SAN.

1. Login to VC as Administrator and create a new host profile using another ESXi host.
2. Edit the Host profile and change the "Administrator password" to a fixed password.
3. Next "Attach" and "Apply" the modified profile to the host that you don’t know the password for.
4. You should now be able to login via the console or SSH however the change is only temporary.
5. From ESXi command line execute the backup script to make changes persistent - ~ # /sbin/auto-backup.sh
7. Finally put ESX into Maintenance mode, Reboot and verify new login credentials.

Thanks again to VCE!