Improving application performance from Azure with some network routing changes

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Over the last few months, I’ve been working with a UK Government customer to move them from a legacy managed services contract with a systems integrator to a disaggregated solution built around SaaS services and a virtual datacentre in Azure.  I’d like to write a blog post on that but will have to be careful about confidentiality and it’s probably better that I wait until (hopefully) a risual case study is created.

One of the challenges we came across in recent weeks was application performance to a third-party-hosted solution that is accessed via a site-to-site VPN from the virtual datacentre in Azure.

My understanding is that outside access to Microsoft services hits a local point of presence (using geographically-localised DNS entries) and then is routed across the Microsoft global network to the appropriate datacentre.

The third-party application in Bedford (UK) and the virtual datacentre is in West Europe (Netherlands) so the data flows should have just been in Europe.  Even so, a traceroute from the third-party provider’s routers to our VPN endpoint suggested several long (~140ms) hops once traffic hit the Microsoft network. These long hops were adding significant latency and reducing application performance.

I logged a call under the customer’s Azure support contract and after several days of looking into the issue, then identifying a resolution, Microsoft came back and said words to the effect of “it should be fixed now – can you try again?”.  Sure enough, ping times (not the most accurate performance test it should be said) were significantly reduced and a traceroute showed that the last few hops on the route were now down to a few milliseconds (and some changes in the route). And overnight reports that had been taking significantly longer than previously came down to a fraction of the time – a massive improvement in application performance.

I asked Microsoft what had been done and they told me that the upstream provider was an Asian telco (Singtel) and that Microsoft didn’t have direct peering with them in Europe – only in Los Angeles and San Francisco, as well as in Asia.

The Microsoft global network defaults to sending peer routes learned in one location to the rest of the network.  Since the preference of the Singtel routes on the West Coast of the USA was higher than the preference of the Singtel routes learned in Europe, the Microsoft network preferred to carry the traffic to the West Coast of the US.  Because most of Singtel’s customers are based in Asia, it generally makes sense to carry traffic in that direction.

The resolution was to reconfigure the network to stop sending the Singtel routes learned in North America to Europe and to use one of Singtel’s local transit providers in Europe to reach them.

So, if you’re experiencing poor application performance when integrating with services in Azure, the route taken by the network traffic might just be something to consider. Getting changes made in the Microsoft network may not be so easy – but it’s worth a try if something genuinely is awry.

Short takes: calculating file transfer times; Internet breakout from cloud datacentres; and creating a VPN with a Synology NAS

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Another collection of “not-quite-whole-blog-posts”…

File transfer time calculations

There are many bandwidth/file transfer time calculators out there on the ‘net but I found this one particularly easy to work with when trying to assess the likely time to sync some data recently…

Internet breakout from IaaS

Anyone thinking of using an Azure IaaS environment for Internet breakout (actually not such a bad idea if you have no on-site presence, though be ready to pay for egress data) just be aware that because the IP address is in Holland (or Ireland, or wherever) location-aware websites will present themselves accordingly.

One of my customers was recently caught out when Google defaulted to Dutch after they moved their client Internet traffic over to Azure in the West Europe region… just one to remember to flag up in design discussions.

Creating a VPN with a Synology NAS

I’ve been getting increasingly worried about the data I have on a plethora of USB hard disks of varying capacities and wanted to put it in one place, then sync/archive as appropriate to the cloud. To try and overcome this, I bought a NAS (and there are only really two vendors to consider – QNAP or Synology).  The nice thing is that my Synology DS916+ NAS can also operate many of the network services I currently run on my Raspberry Pi and a few I’ve never got around to setting up – like a VPN endpoint for access to my home network.

So, last night, I finally set up a VPN, following Scott Hanselman’s (@shanselman) article on Setting up a VPN and Remote Desktop back into your home. Scott’s article includes client advice for iPhone and Windows 8.1 (which also worked for me on Windows 10) and the whole process only took a few minutes.

The only point where I needed to differ from Scott’s article was the router configuration (the article is based on a Linksys router and I have a PlusNet Hub One, which I believe is a rebadged BT Home Hub). L2TP is not a pre-defined application to allow access, so I needed to create a new application (I called it L2TP) with UDP ports 500, 1701 and 4500 before I could allow access to my NAS on these ports.

Creating an L2TP application in the PlusNet Hub One router firewall

Port forwarding to L2TP in the PlusNet Hub One router firewall

End user computing – the device doesn’t matter

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Following a recent Windows update that “went bad”, I needed to have my work PC rebuilt.  That left me with a period when I had work to do, but only a smartphone to work on or my personal devices. To me, this was also a perfect opportunity to put cloud services to work.

So, armed only with a web browser on another PC, I was perfectly able to access email and send/receive IMs (it’s all in Office 365), pester people on Yammer, catch up on some technical videos, etc. There was absolutely nothing (technically) preventing me from doing my job on another device. That’s how End User Computing should work – providing a flexible computing workstyle that’s accessible regardless of the device and the location.

The real issues are not around technology, but process: questions were asked about why I wasn’t following policy and using my company-supplied device; and I was able to answer with clear reasons and details of what I was doing to ensure no customer information was being processed on a non-corporate device. There are technical approaches to ensuring that only approved devices can be used too – but what’s really needed is a change of mindset…

Short takes: pairing my headphones, firewalls and Exchange SMTP communications, tethered photos with a Mac

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Some more snippets that don’t quite make a blog post…

Because I always forget how to do this: how to pair a Plantronics BackBeat PRO headset with a mobile device.

And a little tip whilst troubleshooting connectivity to an Exchange Server server for hybrid connectivity with Office 365… if telnet ipaddress 25 gives a banner response from the SMTP server then that’s a good thing – if the firewall is interrupting transmission then I’ll get nothing back, or asterisks ********. Joe Palarchio (@JoePalarchio) writes about this (see issue 7) in his post on Common Exchange Online Hybrid Mail Flow Issues. Note that firewalls doing any form of blocking between Exchange servers are unsupported but that doesn’t stop customers from putting them between their email servers and anything running in the cloud (e.g. Hybrid server in Azure).  If you need to do this, then you should have any ANY/ANY rule (i.e. allow free flow of traffic) between the Exchange Server servers.

Take photos with OS X Image CaptureFinally, back in 2009, I  wrote about tethering a DLSR to a computer and taking pictures using Windows PowerShell (I think I’ve also written about using software to do this). Well, it turns out that the OS X Image Capture utility can also take a photo on a supported camera – either on a timed basis or by pressing a key.  Could be useful to know if setting up a time-lapse, or for studio work…

Copy NTFS permissions from one folder/file to another

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

I’m working with a customer who is migrating from on-premises datacentres to the cloud – using a virtual datacentre in Microsoft Azure. One of the challenges we have is around the size of the volumes on a file server: Azure has a maximum disk size of 1023GB and the existing server has a LUN attached that exceeds this size.

We can use other technologies in Windows to expand volumes over multiple disks (breaking the <1TB limit) but the software we intend to use for the migration (Double Take Move) needs the source and target to match. That means that the large volume needs to be reduced in size, which means moving some of the data to a new volume (at least temporarily).

One of my colleagues moved the data (using a method that retained permissions) but the top level folders that he created were new and only had inherited permissions from their parent. After watching him getting more and more frustrated, manually configuring access control lists and comparing them in the Windows Explorer GUI, I thought there had to be a better way.

A spot of googling turned up some useful information from forums and this is what I did to copy NTFS permissions from the source to the target (thanks to Kalatzis Stefanos for his answer on Server Fault).

First of all, export the permissions from the source folder with the icacls.exe command:

icacls D:\data /save perms.txt [/t /c]

/c is continue on error; /t is to work through subfolders too

Then, apply these permissions to the target volume. They can be applied at volume level, because the export includes the file names and an associated ACL (i.e. it only applies to matching files)

icacls D:\ /restore perms.txt

But what if the source and destination folders/files have different names? That’s answered by Scott Chamberlain in another post, which tells me I can just edit my perms.txt file and change the file/folder name before each ACL.

By following this, I was able to export and re-apply permissions on several folders in a few minutes. Definitely a time saver!

Thoughts on the use of Sway as a presentation tool

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

A couple of weeks ago, I gave a short talk on adopting cloud services at Milton Keynes Geek Night (MKGN). I’ll admit being a little nervous – the talk was supposed to be 5 minutes (and I had more to say than would ever have fitted – I later learned it’s pretty rare for anyone to stick to their allotted time) and I’m not used to speaking to an audience larger than a meeting room-full (a typical MKGN audience in the current venue is around about 100).  Just to make things a little harder for myself, I decided to use Microsoft Sway for my visual aids.

For those who are unfamiliar with Sway, I got excited about it when it first previewed in 2014. Since then it’s shipped and is available as part of Office 365 or as a standalone product. It’s a tool for presenting content from a variety of sources in a visually-appealing style that works cross-platform and cross-form factor.

Even though Sway has an app for Windows 10, some of the content (e.g. embedded tweets) relies on having an Internet connection at the time of presenting.  Wi-Fi at conferences is notoriously bad and 3G/4G at the MKGN venue is not much better (although it did hold up for me on the night). So, with that and the 7Ps in mind I had PowerPoint and PDF fallback plans but I persisted with Sway.

I’m still not sure Sway is a presentation tool though…

You see, as I swiped and clicked my way through, the audience saw everything I saw. I prefer the simplicity of a picture, with my notes on my screen – I talk, the audience listens, the image re-enforces the view. Sway didn’t work for me like that. Indeed, Sway falls into what Matt Ballantine recently described as the latest whizz-bang tool in a post about a request he was given to knock up a few slides of PowerPoint:

“PowerPoint [… is …] rarely used to perform the task it was designed to do […] The latest whizz-bang tool is the answer! Prezi, Sway or whatever it is that the cool kids are using. Actually, though, the answer probably lies as much in new skills that people need to develop to communicate in a Digital era. Questions like:

  • Who is your audience?
  • What is the message that you are trying to deliver?
  • Where will they be?
  • How will they consume your content?
  • How can you extend the conversation?”

We use Sway at work for weekly updates on what’s been happening in the company – internal communications that used to make use of lengthy HTML emails (I almost never used to read to the end) became more immersive and easier to engage with. And that’s where I think Sway fits – as a tool for communications that are read asynchronously. Not as a tool for presenting a message to an audience in real time.

You can see what you think about the use of Sway as a presentation tool when you take a look at the Sway I used for my MKGN talk.

Have I been pwned?

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

You’re probably aware that LinkedIn suffered a major security breach, in which something like 164,611,595 sets of user credentials were stolen. Surprisingly, you won’t find anything about this in LinkedIn’s press releases.

In less enlightened times (and before I started using LastPass), I may have re-used passwords. That’s why breaches like the one at LinkedIn are potentially bad. Re-using that identity means someone can potentially log in as me somewhere else – I could be pwned.

Microsoft Regional Director and MVP, Troy Hunt (@troyhunt) has set up an extremely useful site called HaveIBeenPwned. Entering your email address (yes, that means trusting the site) checks it against a number of known lists and yes, it seems mine was compromised in three hacks (at LinkedIn, Adobe and Gawker). In all of those cases, I’ve since changed my passwords and for popular sites – where they offer the option – I’ve started to use second factor authentication solutions (Azure MFA has been on my Office 365 subscription for a long time, I use Google two-step verification too and, since tonight, I’ve added LinkedIn’s two-step verification and Facebook Login Approvals).

So, I guess the two points of this post are:

  1. For heavens sake stop re-using passwords on multiple sites – you can’t rely on the security of others.
  2. Turn on 2FA where it’s available.

Hopefully one day soon, passwords will be consigned to the dustbin of technology past…

Scripting Azure VM build tasks: static IP addresses, BGInfo and anti-malware extensions

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

Following on from yesterday’s blog post with a pile of PowerShell to build a multiple-NIC VM in Azure, here are some more snippets of PowerShell to carry out a few build-related activities.

Setting a static IP address on a NIC

$RGName = Read-Host "Resource Group"
$VNICName = Read-Host "vNIC Name"
$VNIC=Get-AzureRmNetworkInterface -Name $VNICName -ResourceGroupName $RGName
$VNIC.IpConfigurations[0].PrivateIpAllocationMethod = "Static"
Set-AzureRmNetworkInterface -NetworkInterface $VNIC

Installing BGInfo

$RGName = Read-Host "Resource Group"
$VMName = Read-Host "Virtual Machine Name"
$Location = Read-Host "Region/Location"
Set-AzureRmVMExtension -ExtensionName BGInfo -Publisher Microsoft.Compute -Version 2.1 -ExtensionType BGInfo -Location $Location -ResourceGroupName $RGName -VMName $VMName

Installing Microsoft Antimalware

This one is a little more difficult – the script is a development of Mitesh Chauhan’s work entitled Installing Microsoft Anti Virus Extension to Azure Resource Manager VM using Set-AzureRmVMExtension

It’s worth reading Mitesh’s post for more background on the Microsoft Anti Virus Extension (IaaS Antimalware) and also taking a look at the Security Health in the Azure Portal (currently in preview), which will highlight VMs that have no protection (amongst other things).

Mitesh’s script uses a simple settings string, or for more complex configuration, it reads from a file. I tried to use a more complex setting and it just resulted in PowerShell errors, suggesting this wasn’t proper JSON (it isn’t):

$AntiMalwareSettings = @{
"AntimalwareEnabled" = $true;
"RealtimeProtectionEnabled" = $true;
"ScheduledScanSettings" = @{
"isEnabled" = $true;
"day" = 1;
"time" = 180;
"scanType" = "Full"
};
"Exclusions" = @{
"Extensions" = ".mdf;.ldf;.ndf;.bak;.trn;";
"Paths" = "D:\\Logs;E:\\Databases;C:\\Program Files\\Microsoft SQL Server\\MSSQL\\FTDATA";
"Processes" = "SQLServr.exe;ReportingServicesService.exe;MSMDSrv.exe"
}
}

Set-AzureRmVMExtension : Error reading JObject from JsonReader. Current JsonReader item is not an object: Null. Path”, line 1, position 4.

If I use the JSON form it’s no better:

$AntiMalwareSettings = {
"AntimalwareEnabled": true,
"RealtimeProtectionEnabled": true,
"ScheduledScanSettings": {
"isEnabled": true,
"day": 1,
"time": 180,
"scanType": "Full"
},
"Exclusions": {
"Extensions": ".mdf;.ldf;.ndf;.bak;.trn",
"Paths": "D:\\Logs;E:\\Databases;C:\\Program Files\\Microsoft SQL Server\\MSSQL\\FTDATA",
"Processes": "SQLServr.exe;ReportingServicesService.exe;MSMDSrv.exe"
}
}

Set-AzureRmVMExtension : Unexpected character encountered while parsing value: S. Path ”, line 0, position 0.

So the actual script I used is below:

# Install Microsoft AntiMalware client on an ARM based Azure VM
# Check note at the end to be able to open up the SCEP antimalware console on the server if there are problems.
# Author – Mitesh Chauhan – miteshc.wordpress.com (updated by Mark Wilson - markwilson.co.uk)
# For Powershell 1.0.1 and above
# See https://miteshc.wordpress.com/2016/02/18/msav-extension-on-azurearm-vm/

# Log in with credentials for subscription
# Login-AzureRmAccount

# Select your subscription if required (or default will be used)
# Select-AzureRmSubscription -SubscriptionId "Your Sub ID here"

$RGName = Read-Host "Resource Group"
$VMName = Read-Host "Virtual Machine Name"
$Location = Read-Host "Region/Location"

# Use this (-SettingString) for simple setup
# $AntiMalwareSettings = ‘{ "AntimalwareEnabled": true,"RealtimeProtectionEnabled": true}’;

# Use this (-SettingString) to configure from JSON file
$AntiMalwareSettings = Get-Content ‘.\MSAVConfig.json’ -Raw

$allVersions= (Get-AzureRmVMExtensionImage -Location $location -PublisherName "Microsoft.Azure.Security" -Type "IaaSAntimalware").Version
$typeHandlerVer = $allVersions[($allVersions.count)–1]
$typeHandlerVerMjandMn = $typeHandlerVer.split(".")
$typeHandlerVerMjandMn = $typeHandlerVerMjandMn[0] + "." + $typeHandlerVerMjandMn[1]

Write-Host "Installing Microsoft AntiMalware version" $typeHandlerVerMjandMn "to" $vmName "in" $RGName "("$location ")"
Write-Host "Configuration:"
$AntiMalwareSettings

# Specify for -SettingString parameter here which option you want, simple $settingsstring or $MSAVConfigfile to sue json file.
Set-AzureRmVMExtension -ResourceGroupName $RGName -VMName $vmName -Name "IaaSAntimalware" -Publisher "Microsoft.Azure.Security" -ExtensionType "IaaSAntimalware" -TypeHandlerVersion $typeHandlerVerMjandMn -SettingString $AntiMalwareSettings -Location $location

# To remove the AntiMalware extension
# Remove-AzureRmVMExtension -ResourceGroupName $resourceGroupName -VMName $vmName -Name "IaaSAntimalware"

# If you have error saying Admin has restricted this app, Navigate to “C:\Program Files\Microsoft Security Client”
# Run "C:\Program Files\Microsoft Security Client\ConfigSecurityPolicy.exe cleanuppolicy.xml"
# Or simply drag the cleanuppolicy.xml file above onto the ConfigSecurityPolicy.exe to sort it and you should be in.

The MSAVconfig.json file contains the JSON version of the Anti-Malware settings:

{
"AntimalwareEnabled": true,
"RealtimeProtectionEnabled": true,
"ScheduledScanSettings": {
"isEnabled": true,
"day": 1,
"time": 180,
"scanType": "Full"
},
"Exclusions": {
"Extensions": ".mdf;.ldf;.ndf;.bak;.trn",
"Paths": "D:\\Logs;E:\\Databases;C:\\Program Files\\Microsoft SQL Server\\MSSQL\\FTDATA",
"Processes": "SQLServr.exe;ReportingServicesService.exe;MSMDSrv.exe"
}
}

Building a multiple NIC VM in Azure

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

I recently found myself in the situation where I wanted to build a virtual machine in Microsoft Azure (Resource Manager) with multiple network interface cards (vNICs). This isn’t available from the portal, but it is possible from the command line.

My colleague Leo D’Arcy pointed me to Samir Farhat’s blog post on how to create a multiple NIC Azure virtual machine (ARM). Samir has posted his script on the TechNet Gallery but I made a few tweaks in my version:

#Variables
$VMName = Read-Host "Virtual Machine Name"
$RGName = Read-Host "Resource Group where to deploy the VM"
$Region = Read-Host "Region/Location"
$SAName = Read-Host "Storage Account Name"
$VMSize = Read-Host "Virtual Machine Size"
$AvailabilitySet = Read-Host "Availability Set ID (use Get-AzureRMAvailabilitySet to find this)"
$VNETName = Read-Host "Virtual Network Name"
$Subnet01Name = Read-Host "Subnet 01 Name"
$Subnet02Name = Read-Host "Subnet 02 Name"
$cred=Get-Credential -Message "Name and password for the local Administrator account"
 
# Getting the Network
$VNET = Get-AzureRMvirtualNetwork | where {$_.Name -eq $VNETName}
$SUBNET01 = Get-AzureRmVirtualNetworkSubnetConfig -Name $Subnet01Name -VirtualNetwork $VNET
$SUBNET02 = Get-AzureRmVirtualNetworkSubnetConfig -Name $Subnet02Name -VirtualNetwork $VNET
 
# Create the NICs
$NIC01Name = $VMName+'-NIC-01'
$NIC02Name = $VMName+'-NIC-02'
Write-Host "Creating" $NIC01Name
$VNIC01 = New-AzureRmNetworkInterface -Name $NIC01Name -ResourceGroupName $RGName -Location $Region -SubnetId $SUBNET01.Id
Write-Host "Creating" $NIC02Name
$VNIC02 = New-AzureRmNetworkInterface -Name $NIC02Name -ResourceGroupName $RGName -Location $Region -SubnetId $SUBNET02.Id
 
# Create the VM config
Write-Host "Creating the VM Configuration"
$VM = New-AzureRmVMConfig -VMName $VMName -VMSize $VMSize -AvailabilitySetId $AvailabilitySet
$pubName="MicrosoftWindowsServer"
$offerName="WindowsServer"
$skuName="2012-R2-Datacenter"
Write-Host " - Setting the operating system"
$VM = Set-AzureRmVMOperatingSystem -VM $vm -Windows -ComputerName $vmName -Credential $cred -ProvisionVMAgent -EnableAutoUpdate
Write-Host " - Setting the source image"
$VM = Set-AzureRmVMSourceImage -VM $vm -PublisherName $pubName -Offer $offerName -Skus $skuName -Version "latest"
#Adding the VNICs to the config, you should always choose a Primary NIC
Write-Host " - Adding vNIC 1"
$VM = Add-AzureRmVMNetworkInterface -VM $VM -Id $VNIC01.Id -Primary
Write-Host " - Adding vNIC 2"
$VM = Add-AzureRmVMNetworkInterface -VM $VM -Id $VNIC02.Id
 
# Specify the OS disk name and create the VM
$DiskName=$VMName+'-OSDisk'
Write-Host " - Getting the storage account details"
$SA = Get-AzureRmStorageAccount | where { $_.StorageAccountName -eq $SAName}
$OSDiskUri = $SA.PrimaryEndpoints.Blob.ToString() + "vhds/" + $vmName+"-OSDisk.vhd"
Write-Host " - Setting up the OS disk"
$VM = Set-AzureRmVMOSDisk -VM $VM -Name $DiskName -VhdUri $osDiskUri -CreateOption fromImage
Write-Host "Creating the virtual machine"
New-AzureRmVM -ResourceGroupName $RGName -Location $Region -VM $VM

Upgraded Azure support for Enterprise Agreement customers

This content is 9 years old. I don't routinely update old blog posts as they are only intended to represent a view at a particular point in time. Please be warned that the information here may be out of date.

I recently found myself in a situation where I tried to log a support request on my customer’s Microsoft Azure subscription, only to find that they didn’t have any eligible support agreements in place.

You'll need to buy a support plan before you can submit a technical support request

That seemed strange, as from 1 May 2016, Microsoft is offering a 12-month support upgrade to all customers that have or intend to buy Microsoft Azure services on an Enterprise Agreement (EA), except those customers with a Premier support contract.

Digging a little deeper, I found that:

“Microsoft will begin upgrade for existing Azure customers on Enterprise Agreement on May 1, 2016, and plans to complete the upgrades by September 30, 2016. New customers will be upgraded within 30 days of account activation. Customers will be notified by email upon being upgraded. For more information, please talk with your account manager or contact EA Azure Support through the Enterprise Portal”

But, the Enterprise Agreement Support Offer page that contains this information is subtitled: “to activate, contact your Microsoft account team”, so I contacted my customer’s account team.  Initially, they said that the customer needed to contact their Microsoft Licensing Solution Provider (LSP), who were equally confused, but I pushed a little harder and the account team investigated further, before arranging the necessary support.

So, if you’re an EA customer and you can’t wait until September to get an upgrade to your Azure support agreements, it may just be worth a chat with your Microsoft account team.