False Alert in SCOM | Cluster Resource Group Offline

I’ve noticed a strange alert in a Hyper-V cluster monitoring scenario earlier today. Though I did a cluster validation immediately and noticed a couple of network warnings due to an incorrect IP address configuration there was nothing wrong in the cluster itself.

Cluster Resource Offline (1)Digging into Health explorer I could see below. All three nodes in the cluster were reporting critical health states.Then I realized that there is Network adapter in each host which we haven’t assigned a Gateway or DNS and it is displaying an IP address failed error in Health explorer. But still that wasn’t the issue.

Cluster Resource Offline (3)Solution

  1. Open a PowerShell session as an administrator and type Get-ClusterGroup and press Enter. This will have an output similar to below.Cluster Resource Offline (2)
  2. Have you noticed Available Storage is offline in above screenshot? That is the culprit. What happens here is when some clusters  are built up, they won’t have Available Storage in the cluster console and therefore the Available Storage cluster group is offline by default. All you have to do is make it online by typing Start-ClusterGroup -Name “Available Storage” and et viola alert is gone.
  3. Cluster Group is displayed as failed. This is because of the IP address issue of unused adapter as I mentioned earlier. I had to bear that for the time being as we are using those NICs for a separate task later.

Data Access Service SPN not registered in SCOM 2012 R2

If you are using a less privileged standard domain account as your SCOM default action or installation account, below is one of the very first common errors it will throw in your fresh SCOM installation. This issue is persistent in SCOM 2012, 2012 SP1 & SCOM 2012 R2 as well.

SPN Issue

A standard user account does not have the necessary rights to update its own SPN in an domain environment by default and a domain admin should create the SPN. What happens is if you have domain admin rights for your SCOM action account (which is not recommended) it can bypass this situation.

Let’s first check the SPN for System Center Data Access account (SDK). Execute below command in a command prompt.

C:\>setspn -L <DOMAIN>\sdkdomainuseraccount

Output will be similar to something like this. In this case if you have only one management server you will get only one SPN.

Registered ServicePrincipalNames for CN=sdkdomainuseraccount,OU=Service Accounts,OU=Accounts,OU=US,DC=domain,DC=com:
        MSOMSdkSvc/<MS Name>
        MSOMSdkSvc/<MS Name>.<Domain>

Fix

  • Open ADSI Edit > Browse to the Management Server’s Computer account (the FQDN of the computer account)
  • Right click the account and select Properties.
  • In the Attribute Editor add the SPN you got in the previous section to the servicePrincipalName attribute. Note that for a single Management Server there are two values like in above example. If you have more MS that mean you will have to enter the relavent SPNs for those computer accounts as well.

And that should do the trick.

Docker Client for Windows is here

Last year Microsoft has partnered with Docker Inc to provide the next generation applications called Containers. As a result of the journey towards heterogeneous apps,  Microsoft has released the GA version of Docker CLI for Windows last week. As of today, using this tool you can manage Linux containers hosted in Azure or your own VMs straight from your Windows desktop. Microsoft plans to introduce their own container technology as below.

Windows Server containers

The idea behind this container is similar to Linux Container technology. Containers are isolated, but they share OS kernel and, where appropriate bins/libraries. Simply put we are talking about OS Virtualization where applications doesn’t need to be OS specific.

Hyper-V Containers

Using Microsoft Hyper-V technology these containers are fully isolated from the OS itself by running on the hypervisor layer. This ensures that one container has no impact on it’s host or any other containers in the same system. Even though these containers are running inside a hypervisor it doesn’t have any restriction over container deployment. You can simply deploy containers that you targeted for Windows Server in Hyper-V containers and vice versa without any modification.

Nano Server

Microsoft’s Nano server is the Windows version of Red Hat’s Atomic host, an OS designed to run containers in cloud. This version of Windows has no GUI stack, 32 bit support (WOW64), MSI and a number of default Server Core components has also been taken off. Also local logon and Remote Desktop has been removed and managing a nano server can be done only via  WMI and PowerShell. As per Microsoft nano server has 93% lower VHD size, 92% fewer critical bulletins and most importantly 80% fewer reboots.

Installing Docker CLI in Windows

There are two methods currently supported for installing Docker CLI for Windows.

Boot2Docker

Boot2Docker will install a tiny Linux VM running on Virtual Box (Yes you will have to disable Hyper-V engine for this). It is a lightweight linux distro called Tiny Core Linux specifically designed to run Docker containers. You can download the Windows version from here.

Chocolatey

This is Machine Package manager like built for Windows.Think it as YUM or apt-get for Windows. Installation is rather simple. Let’s see how we install Docker CLI using this method. You can visit their website for more information on all supported packages other than Docker.

  • Open a Command Prompt as admin and execute below command.

C:/>@powershell -NoProfile -ExecutionPolicy unrestricted -Command “iex ((new-object net.webclient).DownloadString(‘https://chocolatey.org/install.ps1’))” && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

  • Once it finishes open a PS seesion as an administrator and set the execution policy to at least Bypass. Then type the below command to proceed.

PS:/>iex ((new-object net.webclient).DownloadString(‘https://chocolatey.org/install.ps1’))

  • Now it’s time to install the Docker CLI. Using either PowerShell or Command prompt execute below command to install Docker CLI.

C:/>choco install docker

  • To upgrade the Docker Client type choco upgrade docker

 

New DPM 2012 R2 Management Packs available now

Microsoft has released a brand new collection of MPs for DPM 2012 R2 with more feature enhancements. For those who were frustrated with the DPM MP that was shipped with DPM 2012 R2 RTM media this provides a new monitoring MP, Deduplication Reporter MP, Library MP and a Reporting MP.

Please note that there are few prerequisites that needs to be met prior deploying this MP.

  • DPM 2012 R2 UR5 is a mandatory requirement.
  • DPM RTM version MPs should be installed first. This is available in DPM 2012 R2 RTM media. If you are getting any error while importing the new MPs, first try deleting the two RTM Library & Discovery MPs and then try to import the new MPs.
  • DPM Central console should be installed in SCOM.

Following .mp files are available with this download.

  • SystemCenter.DataProtectionManager.2012.Discovery.MP (required) (version 4.2.1276)
  • SystemCenter.DataProtectionManager.2012.Library.MP (required) (version 4.2.1276)
  • SystemCenter.DataProtectionManager.2012.Reporting.MP (required) (version 4.2.1279)
  • SystemCenter.DataProtectionManager.DedupReporter.MP (optional) (version 7.1.10123.0)

You can download this management pack from here.

DPM 2012 R2 UR5 Console Crashing Issue

A known issue has been identified in the latest Update Rollup for Data Protection Manager 2012 R2. Let’s see what is this issue and how to rectify same in this article.

Issue

When you are working in local DPM server or an attached secondary DPM server you may notice that the console crashes basically when you are searching through something or navigating from one section. Int his case even if you resatrt the console it becomes unresponsive and also cmdlets also enter the same state

The Hotfix

Microsoft has released a hotfix package KB3040326 which address the above issue. Note that before you apply this hotfix you’ll have to backup the DPMDB database and after installing the hotfix you need to reboot the DPM server/s.

In my personal experience I would recommend you to wait a while before you apply UR5 in production systems until UR6 is released. But since UR5 has lot of improvements in new workload backups you may want to reconsider that option. For a list of new features in UR5 you can refer DPM Team’s announcement.

Azure VM Reboot Logs

How many of us really pay attention to e-mails sent by Microsoft Azure regarding planned maintenance? Does anyone actually monitor what happens to our VMs during a planned maintenance window schedule by Azure Team? If the answer is NO let’s see how we can get some insight on IaaS VM availability during a scheduled maintenance.

As there are periodic updates that need to be performed in Azure data centers globally, Microsoft will notify you of any scheduled maintenance windows. Usually we are not facing any service interruption during such unless Microsoft has specifically mentioned that there will be a service unavailability. Also sometimes these updates may restart the Azure VMs if required and up to now you as a customer didn’t have any visibility other than the regular email communique.

If you play attention to any cloud service (hosting VMs) > Dashboard section you may notice that there is a new feature called View Reboot Logs is available as per below. Note that this is the only place you get such an option as of now.

Azure VM Reboot Logs 1You can select a data range and analyze what are the planned reboots that has happened. Remember this is only applicable for planned maintenance not unexpected downtime.

Azure VM Reboot Logs 2

For shell lovers below is a sample PowerShell cmdlets that you need to run to get a more liner output.

Get-AzureDeploymentEvent -ServiceName <cloudservicename> -StartTime <start time>-EndTime <end time>

The input for start and end times should be similar to the format of get-date cmdlets output.

As per Microsoft they will be adding more events to this feature where you will have complete visibility over your VM availability. For me this is yet another win for my customers to see really understand the value of Microsoft Cloud platform.

MOM Agent Update Rollup KB 3032946 released

Few days ago Operations Manager Engineering team has announced the availability of a new update rollup for MOM agent. This update rollup fix below mentioned issues that are present in the current version.

  • Proxy settings are now in Agent setup wizard. Earlier users had to manually enable the agent proxy in each agent after installation but in this update you can enable it while pushing agents.
  • Account Validation steps are added. Once a user enters the Azure Operational Insights credentials to connect, credentials are validated before proceeding.
  • Connection Status in Control Panel. Now you can see the connection status of an agent in the Control Panel settings of the MOM Agent.

You can manually download the KB 3032946 from Microsoft Update catalog by visiting here or you can obtain it via Microsoft Update (Optional Updates) if your update service is configured to receive updates for other Microsoft software.

 

Introducing Linux Integration Services 4.0 Preview for Microsoft Azure

In Hyper-V platform integration services or rather device drivers for emulated hardware plays a vital role. The purpose of these services are to enhance the functionality of VMs to get the maximum performance in par with an actual physical server. Microsoft has announced the availability of LIS 4.0 for Azure VMs recently as a early preview.

This preview version of LIS supports CentOS 6.0-6.6, 7.0-7.1 64 bit editions running on Azure VMs and has introduced below additional functionality to Azure Linux IaaS VMs.

  • CentOS version 6.6 through 7.1 is now supported.
  • Dynamic Memory – Hot Add for above CentOS releases which allows you to dynamically increase the amount of memory that is available to a running VM.

You can download and install the LIS Package from this official Microsoft Link. For a list of features offered by Integration Services for Linux & FreeBSD refer here.

Upgrading Linux Integration Services on Azure Linux VMs

Following procedure needs to be performed as a super user or a user in suborders list.

  • Verify the Linux version first by running below command.

# cat /etc/centos-release

  • SCP (secure copy) the lis4.tar.gz file to the target VM. You can use putty for this.
  • Extracted the tar file by executing

# tar xvzf lis4.tar.gz

  • Traverse to the appropriate release version inside the lis4 directory where XX is the version obtained earlier.

# cd lis4/CentOs<XX>

  • Execute the upgrade script and reboot the VM.

# ./upgrade.sh

# reboot

 

 

Azure VM Extensions for Linux VMs

Whenever you are creating a Linux VM (or a Windows VM) in Azure from the gallery you may have noticed that in the last screen you get the option to install Azure Agent depending on the platform. This Azure Virtual Machine Agent can be is used to install, configure, manage and run Azure VM Extensions which extends the functionality of the VMs that they are installed on.

VM Agent

VM Extensions

VM Extensions can,

  • Modify security and identity features, such as resetting account values and using anti malware
  • Start, stop, or configure monitoring and diagnostics
  • Reset or install connectivity features, such as RDP and SSH
  • Diagnose, monitor, and manage your VMs

What we are focusing today is VM Extenstions for Linux VMs.Below are some of the cool VM Extensions that you can use in Linux VMs.

CustomScript: Run any script on a Linux VM

  • Download files and run scripts from an Azure storage account
  • Not limited to a specific scripting language

VMAccess

  • Reset the password of the original provisioned user or create new user
  • (Re)set SSH key for user
  • Ensure SSH firewall port (22) is open
  • Restore the SSH server configuration to a working default

OSPatching

  • Extension enables scheduled/automatic updates for a Linux VM
  • Coordinate patch schedule across multiple VMs
  • Portal integration

VMAccess extension in particular is a great tool if you ever lost access to your Linux VM. In Linux you always reach the VM via SSH and this extension is capable of altering the SSH configuration in case of a disaster.

Below are some great resources you can use if you are interested in leveraging VM extensions for Azure Linux VMs.

Resources

  1. Regaining control of your Linux VM with VMAccess Extension
  2. List of Azure VM Extensions
  3. Automated OS patching for Linux VMs