Category Archives: Backup

File Recovery Error in Azure Backup

While trying to perform an in-place file restore in an Azure VM using Azure Backup, I have encountered an execution error. Azure Backup leverages a PowerShell script to mount the volumes of a Protected VM. In my case the following error was encountered when I executed the recovery script.

Microsoft Azure VM Backup - File Recovery
______________________________________________
Invoke-WebRequest : <HTML><HEAD><TITLE>Error Message</TITLE>
<META http-equiv=Content-Type content="text/html; charset=utf-8">
<BODY>
<TABLE><TR><TD id=L_dt_1><B>Network Access Message: The page cannot be displayed<B></TR></TABLE>
<TABLE><TR><TD height=15></TD></TR></TABLE>
<TABLE>
<TR><TD id=L_dt_2>Technical Information (for Support personnel)
<UL>
<LI id=L_dt_3>Error Code: 407 Proxy Authentication Required. Forefront TMG requires authorization to fulfill the
request. Access to the Web Proxy filter is denied. (12209)
<LI id=L_dt_4>IP Address: 10.31.8.16
<LI id=L_dt_5>Date: 8/8/2017 11:53:42 PM [GMT]
<LI id=L_dt_6>Server: XXXX.ab.abc.net
<LI id=L_dt_7>Source: proxy
</UL></TD></TR></TABLE></BODY></HTML>
At C:\Users\whewes_adm\Desktop\ILRPowershellScript.ps1:101 char:12
+ $output=Invoke-WebRequest -Uri "https://download.microsoft.com/download/E/1/4 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
 eption
 + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
Invoke-WebRequest : <HTML><HEAD><TITLE>Error Message</TITLE>
<META http-equiv=Content-Type content="text/html; charset=utf-8">
<BODY>
<TABLE><TR><TD id=L_dt_1><B>Network Access Message: The page cannot be displayed<B></TR></TABLE>
<TABLE><TR><TD height=15></TD></TR></TABLE>
<TABLE>
<TR><TD id=L_dt_2>Technical Information (for Support personnel)
<UL>
<LI id=L_dt_3>Error Code: 407 Proxy Authentication Required. Forefront TMG requires authorization to fulfill the
request. Access to the Web Proxy filter is denied. (12209)
<LI id=L_dt_4>IP Address: 10.31.8.16
<LI id=L_dt_5>Date: 8/8/2017 11:53:42 PM [GMT]
<LI id=L_dt_6>Server: XXXX.ab.abc.net
<LI id=L_dt_7>Source: proxy
</UL></TD></TR></TABLE></BODY></HTML>
At C:\Users\whewes_adm\Desktop\ILRPowershellScript.ps1:102 char:12
+ $output=Invoke-WebRequest -Uri "https://download.microsoft.com/download/E/1/4 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
 eption
 + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
Unable to access the recovery point. Please make sure that you have enabled access to Azure public IP addresses on the
outbound port 3260 and 'https://download.microsoft.com/'

One thing I noticed was it was complaining about outbound access to Azure public IP addresses on port 3260. The VMs were connected to on-premises environment via a dedicated ExpressRoute circuit so there were no issues with white listing Azure public IP addresses according to my knowledge. Also there were no NSGs controlling the traffic in the subnet where this VM was deployed.

I had a look on another server that is running in a VMware cluster on-premises and noticed that there is a HTTP proxy present in the environment. Once I have added the proxy settings in the VM , I could execute the recovery script without any hassle. 

The article “Prepare your environment to back up Azure virtual machines” published in the Microsoft documentation, explains the required network configuration for Azure Backup in case your environment has policies governing outbound Internet connectivity. Therefore I recommend you to have a look on that first before planning your Azure Backup deployment to protect Azure VMs.

Instant File Recovery with Azure Backup

Microsoft Azure now allows file and folder level recovery with Azure Backup. This feature has been available as a preview for both Windows & Linux VMs and is now generally available. Previously Azure Backup allowed VM level recovery and for those who wanted to leverage file level recovery had to leverage solutions like DPM or Azure Backup to achieve their protection goals.

Restore-as-a-Service in Azure Backup allow syou to recovery files or folders instantaneously without deploying any other additional components. Azure Backup leverages an iSCSI-based approach to open/mount application files directly from its’ recovery points. This eliminates the need to restore the entire VM to recover files. 

RaaS in Action

In below example I’m going to explain how you can recover files or folders from a Windows IaaS VM in Azure.

  • Select the VM that you want to recover files from, in the recovery services vault under Backup items. Click the File Recovery option.

  • Select the recovery point as in (1) and download the executable that allows you to browse and recovery files as in (2).  Run this as an administrator and you will have to provide the password as in (3) to execute this file.

  • This script can be executed on any machine that has the same (or compatible) operating system as the backed-up VM. Unless the the protected Azure VM uses Windows Storage Spaces (for Windows Azure VMs) or LVM/RAID Arrays(for Linux VMs), you can run the executable/script on the same VM. If they do, run it on any other machine with a compatible operating system.
Server OS Compatible client OS
Windows Server 2012 R2 Windows 8.1
Windows Server 2012 Windows 8
Windows Server 2008 R2 Windows 7
Ubuntu 12.04 and above
CentOS 6.5 and above
RHEL 6.7 and above
Debian 7 and above
Oracle Linux 6.4 and above

If you are restoring from a Linux VM you need bash version 4 or above and python version 2.6.6 and above to execute the script. 

  • When you run the script the volumes are mounted in the client OS that you are using and will have different drive letters that the ones from the original VM. Make sure you identify the new drives attached.  You can view your new drives in Windows Explorer and copy them to an alternate location.

  • Finally after restoring your files/folders to unmount the drives, Click the File Recovery blade in the Azure portal and select Unmount Disks as in (4).

 

New Security Features in Azure Backup

Recently Microsoft has introduced new security capabilities to Azure Backup which allows you to secure your backups against any data compromise and attacks. These features are now built into the recovery services vault and you can enable and start using them within a matter of 5 minutes.

Prevention

For critical operations such as  delete backup data, change passphrase, Azure Backup now allows you to use an additional authentication layer where you need to provide a  Security PIN which is available only for users with valid azure credentials to access the backup vaults.

Alerting

You can now configure email notifications to be sent for specified users for operations that have an impact on the availability of the backup data .

Recovery

You can configure Azure backup to retain deleted backup data for 14 days where you can recover the deleted data using the recovery points. When enabled, this will always maintain more than one recovery point so that there will be enough recovery points from which you can recover the deleted data.

How do I enable security features in Azure Backup?

These security features are now built into the recovery services vault where you can enable all of them with a single click.

1-enable-azure-backup-security

Following are the requirements and considerations that you should be aware of when you enable these new security features.

  • The minimum MAB agent version should be 2.0.9052 or you should upgrade to this agent version immediately after you have enabled these features.
  • If you are using Azure Backup Server the minimum MAB agent version should be 2.0.9052 with Azure Backup Server upgrade 1
  • Currently these settings won’t work with Data Protection Manager and will only be enabled with future Update Roll-ups.
  • Currently these settings won’t work with IaaS VM Backups.
  • Enabling these settings is a one-time action which is irreversible.

Testing new security features

In below video I’m trying to change the passphrase of my Azure Backup agent and save it. Note that here I will have to provide a Security PIN in order to proceed or otherwise the operations fails. 

Next I’m going to setup backup alerts for my recovery services vault. Once I create an alert subscription I’m going to delete my previous backup schedule. Here I will have the chance of restoring the data within 14 days after deletion.

Backup ARM VMs in Azure | Tips & tricks

As you already know Microsoft Azure Fabric is now in version 2 which is sometimes referred to as Azure Resource Manager (ARM) deployment model. Most of the services from old Azure Service Management model are now available in the new model (the new portal) and today we are going to see how we can backup VMs deployed using ARM deployment model using a Azure Recovery Services Vault.

Note that you may notice another two services in your Azure subscription called Backup vaults & Site Recovery vaults which are redundant and has no use. (They are just placeholders which will be removed soon I assume)

Backup ARM VMs (1)

Essentially following scenarios are supported in a new Recovery Services vault. If you are using premium storage accounts for your VMs  keep in mind that it is only supported in a public preview and not generally available as of yet.

  • Azure Resource Manager VMs
  • Classic VMs

The process can be done in few easy steps.

Creating a Recovery Services Vault

A Recovery Services vault holds all the backups and recovery points of the VMs that are being protected along with the backup policy applied to that vault.  One important thing to keep in mind is that Recovery Services Vaults are geo specific, meaning if you need to backup a VM in one region the target vault should reside in the same region as well.

In the Hub menu, click Browse and then search for Recovery Services. I’ve already added it as a favorite by clicking the star right next. Then select Recovery Services vault and click Add.

Backup-ARM-VMs-2.png

Provide a name, select the target Azure subscription, create a new resource group or select an existing one and finally select the region for your Recovery Services vault.

Backup-ARM-VMs-3.png

Next you can select the storage replication option. The default is Geo-redundant storage and if you want a cheaper (but not durable as Geo-redundant) option you can opt out for locally-redundant storage.  Click the All Settings option in your vault dashboard to get started.

Backup-ARM-VMs-4.png

Select a Backup Target

You need to discover your Azure ARM VMs first before they are added to a recovery services vault. This will identify the VMs that can be protected by your recovery services vault.

Backup-ARM-VMs-5.png

Define a Backup Policy

A backup policy defines how frequent the VMs are protected and when the recovery points are created along with the retention range for those recovery points. You can edit the default policy to fit to your needs or create new policy here. You can choose between a daily or weekly schedule to backup your VMs.

Backup-ARM-VMs-6.png

Next select the desired VMs that you wish to backup and finally click Enable Backup.

Backup-ARM-VMs-7.png

Backup-ARM-VMs-8.png

Start the Initial Backup

By default the first scheduled backup is the initial backup. If you want to manually force the first backup it is also possible. In the vault dashboard click Azure Virtual Machines and right click on the desired VM and select Backup Now.

Backup-ARM-VMs-9.png

You can see the backup job progress by clicking All Settings > Jobs > Backup Jobs as below from the vault dashboard.

Backup-ARM-VMs-10.png

When you further expand the backup job you can see the status of each task running underneath.

Backup-ARM-VMs-11.png

Why you should update DPM 2012 R2 to UR9?

Microsoft has released the Update rollup 9 for Data Protection Manager 2012 R2 a little over one month back. This UR contains a number of significant improvements to the current DPM version thereby enabling more capabilities for your enterprise backup strategy.

Here are four reasons that you should consider to apply this UR.

No need for consistency check for file server backups in case of a DPM Filter corruption

When your production file servers encounter an unexpected downtime, DPM file tracking filter gets corrupted and results in an inconsistent replica. In UR9 DPM leverages USN journal to track the changes in files, thereby running a consistent check to repair the damaged filters is no longer needed. The repair operation will be displayed as a synchronization job in DPM which will also sync the replica to latest. Running a consistency check is really painful especially when the replica is huge.

Say Goodbye to Production Server Restart

One of the biggest headaches while upgrading the DPM agent is the requirement to restart the protected servers unlike SCOM or SCCM agents. Finally Microsoft has got rid of the restart requirement. Microsoft has eliminated all the causes for restarting servers while upgrading DPM agents except the filter driver update. Any backup products that perform incremental backups use a filter driver, and whenever there is an update to the filter driver a reboot is needed. If you are already on UR6 or above you can easily upgrade your DPM agents without the restart requirement.  So unless any future UR doesn’t contain a filter driver update no restart required at all.

Cache Space for Online Backup has been reduced

In previous versions of Microsoft Azure Backup Agent local disk cache space requirement was 15% of the data source size for backup to Azure which is a big issue if your data source is  over 10 TB.  This has been reduced to 5% now.

Number of Recovery Points for Online Backups has been increased

For organizations with strict industry compliance requirements the need to have longer retention policies is a mandatory requirement. DPM now supports 9999 recovery points for a online backup in Azure where previously it was 366. This enables more flexible and consistent recovery policies for cloud backup.

You can refer the full KB article and download the binaries for UR9 package from here.

 

DPM 2012 R2 UR7 Re-released

It’s been a while since my last blog post. I’m working on a DPM 2012 R2 test lab these days which I’ve planned to update to the latest UR version. When I checked for the latest UR7 got to know that the bits have been re-released.

As for the DPM team there is an issue in DPM 2012 R2 UR7 released on 28.07.2015 which causes expired recovery points on the disk were not getting cleaned up, resulting an increase in DPM recovery point volume after installing UR7. This re-release has addressed this concern and you can download the upadted bits via DPM 2012 R2 UR7 KB or Microsoft Update Catalog as of today.

OK I have updated to UR7 before 21.08.2015. Now what?

For those who are facing this dilemma should know that the re-released UR7 is not pushed via Microsoft Update and advised to manually install the new package  on the DPM Servers with older UR7 package installed. The installation process will automatically execute pruneshadowcopiesDpm2010.ps1 PowerShell script which contains the fix.

Post-deployment Tips

There is no change in the DPM version (4.2.1338.0) in this re-release and it will remain same after the update. Also you will have to update the Azure Backup Agent to latest version (2.0.8719.0) prior installing DPM UR7 to ensure the integrity of your cloud backups after this release.

For those who like me updating to UR7 the old fashion way (wait for a month or two, lookout for bugs and then update) you’ve got nothing to worry.

New DPM 2012 R2 Management Packs available now

Microsoft has released a brand new collection of MPs for DPM 2012 R2 with more feature enhancements. For those who were frustrated with the DPM MP that was shipped with DPM 2012 R2 RTM media this provides a new monitoring MP, Deduplication Reporter MP, Library MP and a Reporting MP.

Please note that there are few prerequisites that needs to be met prior deploying this MP.

  • DPM 2012 R2 UR5 is a mandatory requirement.
  • DPM RTM version MPs should be installed first. This is available in DPM 2012 R2 RTM media. If you are getting any error while importing the new MPs, first try deleting the two RTM Library & Discovery MPs and then try to import the new MPs.
  • DPM Central console should be installed in SCOM.

Following .mp files are available with this download.

  • SystemCenter.DataProtectionManager.2012.Discovery.MP (required) (version 4.2.1276)
  • SystemCenter.DataProtectionManager.2012.Library.MP (required) (version 4.2.1276)
  • SystemCenter.DataProtectionManager.2012.Reporting.MP (required) (version 4.2.1279)
  • SystemCenter.DataProtectionManager.DedupReporter.MP (optional) (version 7.1.10123.0)

You can download this management pack from here.

Backup Azure IaaS VMs with Azure Backup

We have an exciting update this week with Azure Backup. Now you can directly backup your Azure VMs to Azure Backup vaults easily. This is something that customers were asking for sometime. Let’s take a look at what are the considerations you are going to take into account if you are using this new feature.

  • Backup with no impact to production workloads
  • You do not need to shutdown the VMs
  • Provides application level consistency for Windows operating systems
  • Provides file system level consistency for Linux Operating systems

Backup Procedure

  • Create a backup vault in the same region as your VMs. Currently this feature supports within a single region. But I expect them to make it a geo-enabled feature as keeping the backup in the same data center seems little odd.Azure VM Backup 1
  • Discover the VMs that you need to backup first. For that expand the backup vault > Registered Items > Click DiscoverAzure VM Backup 2

Azure VM Backup 3

  • The next step is to register your VMs in the backup vault. Click the Register button as in the above picture. Keep in mind the VM should be running for the registration to be successfully completed.Azure VM Backup 4
  • Once registration is done click Protect to start protection. Here you need to select the VMs that you need to backup and create a backup policy for the same. You can select a backup frequency as well as a retention range that suits your backup requirement.Azure VM Backup 5

Azure VM Backup 6

  • Remember you can add only one backup policy per VM. Also the maximum retention period is 30 days and you only have backup time slots that are predefined with 30 minute intervals.

Performing a Backup

If you want to perform an adhoc backup out of the backup policy in the Protected Items tab of the backup vault select Backup Now. You can even stop protecting the VM by clicking Stop Protection icon.Azure VM Backup 7

Restore from a backup

  • Go to the Protected Items tab and click Restore. This opens the Restore an Item wizard.Azure VM Backup 10
  • In the Select a recovery point page you can select a restore point from available list of restore points.Azure VM Backup 11
  • In the Select restore instance page you need to specify where you want to restore the VM. This is an alternate location with new VM name, can be a different cloud service and a different Virtual Network. It’s up to you to select those parameters but you might need a new cloud service and a new network if you want to test the back up isolated first.Azure VM Backup 12

Monitor Backup Progress

You can monitor the backup progress in the Jobs page. This is important as you may need to know if a backup operation has failed or server registration has failed.Azure VM Backup 8

If I drill down through my existing adhoc backup I can see the task sequence there.

Azure VM Backup 9As you can see the word PREVIEW in this service (some pages) I wouldn’t be doing this on production but it’s still worth a try.

 

Azure Backup now supports x64 versions of Windows Client OS

If you are running Windows 7 SP1, Windows 8 or Windows 8.1 x64 version I have some good news for you. Microsoft Azure backup is now supported in these versions of Client OS. Microsoft will be dynamically updating the capabilities to provide more integration with Client OS.

Let’s see some need-to-knows about Azure Backup on your device.

  1. Backup is incremental over https
  2. There are two options for backup. Option 1 you can register one device per backup vault where you can create 25 backup vaults per subscription.
  3. Option 2 you can register up to 50 devices in a single vault. Each of these have different pass-phrase used for encryption & decryption.
  4. If your laptop is running on battery scheduled backups are automatically skipped until you plugged in to A/C.

Prerequisites

  • Install KB3015072
  • Download and Install Azure Backup Agent from Azure Portal.

Azure Site Recovery | On-Premises to Cloud Series Introduction

Today we start a new blog post series on Azure Site Recovery. In this series we are going to implement a DR solution for Hyper-V VMs in a VMM cloud. The series is a collection of 4 posts where I’ll guide you through each step in the process. Note that this is just a Proof-of-Concept lab where I’ve used minimal resources to setup.

Scenario

In this setup we will be replicating one Linux VM from a Hyper-V cluster environment to Azure for DR purposes. This VM contains a sample Hello World page in an apache web server.

First Things First

This is the checklist that you want to have for this scenario.

  • Azure Account – An active Azure subscription. You can also use a free trial.
  • Storage Account – This should be Geo-Replicated in the same region as the Recovery Site service.
  • VMM Server – Should be System Center 2012 R2
  • VMM Clouds – At least one VMM cloud  with one or more VMM Host Groups, Hyper-V host servers or clusters in each host group and one or more Generation 1 VMs. Please see here for the compatibility matrix for VMs.

Lets take a look at the tasks that need to be performed in an overview.

  1. Create an Azure Site Recovery Vault
  2. Install Azure Site Recovery Provider & Generate a registration key
  3. Configure Azure Storage Account
  4. Install ASR agent on Hyper-V Hosts
  5. Configure Cloud protection
  6. Configure network mapping – map source VM networks to target Azure Virtual networks
  7. Enable VM protection
  8. Test run – run a test fail-over or create a recovery plan and r un a test fail-over for same.

Lets  discuss how to create the ASR vault & install the ASR Provider in our next post.