Replication Failure in Azure Site Recovery

Azure Site Recovery is a great product for those who want to setup their DR environment with a minimal cost. It is based on Hyper-V replica technology for Hyper-V workloads and supports replication VMware & Physical server workloads to DR as well. Today I’m going to discuss a common issue one can encounter when enabling ASR replication to the cloud.

I’ve been working on an ASR setup during couple months and encountered strange issue when I enabled replication in protected VMs.

The enable protection job fails with below error.

Job ID: f9f84765-b18c-4002-96a4-d420dfb76ea6-2015-05-14 10:00:29Z

Start Time: 5/14/2015 3:30:29 PM

Duration: 10 MINUTES

Protection couldn’t be enabled for the virtual machine. (Error code: 70094)

Provider error: Unable to complete the request. Operation on the <Hyper-V Node>  timed out.

Try the operation again. (Provider error code: 2924)

Possible causes: Protection can’t be enabled with the virtual machine in its current state. Check the Provider errors for more information.

Recommendation: Fix any issues in the Event Viewer logs (Applications and Service Logs – MicrosoftAzureRecoveryServices) on the Hyper-V host server. If this virtual machine is enabled for replication on the Hyper-V host, disable this setting. Then try to enable protection again.

UTC Time: Thu May 14 2015 10:15:59 GMT+0530 (Sri Lanka Standard Time)

Browser: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36

Language: en-us

Portal Version: 5.4.00298.11 (rd_auxportal_stable.150511-1702)

PageRequestId: a04f08ed-8932-43f2-95bc-2faab60ed958

Email Address: xxxxxx@outlook.com (MSA)

Subscriptions: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx

In the particular Hyper-V host following error has been logged in Event logs.

Enable replication failed for virtual machine ‘XXXXXX’ due to a network communication failure. (Virtual Machine ID 807780f6-bb7c-48d5-937d-4857a654dec3, Data Source ID 2256321007502018113, Task ID 8c1a5d7d-0693-4d6b-9243-37cc5e96a7d6)

This ASR setup was a on-premise to Cloud scenario with a single SCVMM server.

After spending a good number of troubleshooting hours I finally figured out what went wrong. The Hyper-V Hosts themselves need Internet connectivity to replicate the VMs to ASR. If you cannot enable direct Internet connectivity on the Hyper-V hosts you should do so via a proxy setup. You can change the proxy settings in ASR Provider in Hyper-V Host.

ASR replication requires traffic to be sent over port 443 (SSL) and in my case only the SCVMM server was configured with Internet access. If you are using a proxy server you may need to consider allowing below for successful replication.

  • *.hypervrecoverymanager.windowsazure.com
  • *.accesscontrol.windows.net
  • *.backup.windowsazure.com
  • *.blob.core.windows.net
  • *.store.core.windows.net
  • Allow the IP addresses in Azure Datacenter IP Ranges and HTTPS (443) protocol. Also your IP address whitelist should contain that of your primary region and  West US IP address ranges.
  • Billy Jeffs

    Who would have thought that you’d need an internet connection to backup something local to the internet 😉

    • Seán Connolly

      Good lord!