2015 Exchange Oscars

Wednesday, November 11, 2015
I'm honored to have be awarded the award for "Best at Knowledge Sharing" earlier this month at the 2015 MVP Global Summit in Redmond, WA.

Netmail and Binary Tree generously sponsored the second Exchange Oscars party at the Tavern Hall in Bellevue, WA. This was a really fun event where the Exchange product group and Exchange MVPs got together for food, drinks and awards.

The Exchange MVPs voted to give awards to the following Exchange product group members:

  • Most Helpful Product Group Member - Brian Day
  • Best Advice Given - Timothy Heeney
  • Best Tool - The Exchange Remote Connectivity Analyzer (ExRCA) - Brad Hughes and Shawn McGrath
  • Best EHLO Blog Post - Ross Smith IV

The Exchange product group also voted to give awards to the following Exchange MVPs:

The hilarious Greg Taylor did a fantastic job as MC of the ceremonies. Lots of fun was had by all!

Read more ...

Don't Delete or Rename the Default MRM Policy

Wednesday, October 28, 2015
I was troubleshooting an issue with an Exchange 2013 customer where their "Default Archive and Retention Policy" was not being automatically applied to archive-enabled mailboxes. Admins could manually apply the policy, but they wanted it applied automatically when the mailbox is enabled for an in-place archive or Exchange Online Archiving.

It turns out that the only retention policy that will be automatically applied is one named "Default MRM Policy". That name is hard-coded into the product, so if you delete it all you need to do is recreate it with that same name to make it the default policy. There is no property for that policy that makes it the default. It's just the name.

Here's some of the back story. Exchange 2007 introduced Managed Folders, which was an early form of retention management, but it was rather clumsy to work with. Exchange 2010 SP1 and later uses message records management (MRM) policies and policy tags. MRM policies are collections of policy tags that tell Exchange server how manage mail data. Some tags apply by default, such as the "Default 2 year move to archive" tag. Others are personal tags that users can chose to apply to one or more mail items, or entire folders, such as "1 Year Delete". Exchange setup creates the "Default MRM Policy" which includes a default set of policy tags.

The Default MRM Policy does not actually apply to any mailboxes by default, unless the the mailbox is given an archive mailbox. At that time, Exchange looks for an MRM policy named, "Default MRM Policy" and automatically applies it to the mailbox. If the Default MRM Policy does not exist, no retention policy is applied.

If you want to apply a different MRM policy you need to manually update the retention policy in the EAC (Mailbox User > Mailbox Features < Retention Policy - see above) or from EMS:
Set-Mailbox <username> -RetentionPolicy "Sales Dept MRM Policy"
For hybrid customers and those using Exchange Online Archiving, you manage MRM policies for on-prem mailboxes on-prem. The policy settings for these mailboxes will sync to Exchange Online with the DirSync process and will be applied to the online archive mailbox.

If a user mailbox and archive mailbox are both in the cloud, you will apply the cloud version of the Default MRM Policy in Office 365. Here, you should configure the Office 365 version of the Default MRM Policy to match the same settings as on-prem.

Read more ...

Enabling Kerberos Authentication in a Mixed Exchange 2013 / 2016 Environment

Tuesday, October 27, 2015
Recently, the Exchange Team published an article, "Exchange 2016 Coexistence with Kerberos Authentication" explaining how to enable Kerberos authentication in a mixed environment. Using Kerberos authentication for Exchange is a best practice and is part of the preferred architecture.

The Kerberos article above gives all the details of how to create the alternate service account (ASA) account and necessary SPNs in Active Directory, as well as how to configure Exchange to use Kerberos.

Once the ASA account is configured on the first Exchange server (2013 or 2016) using the RollAlternateServiceAccountPassword.ps1 script, you run the same script on your other 2013 and 2016 servers to copy the ASA account and password to those servers. We are told to use the 2013 version of the script to deploy across all Exchange servers, even 2016.

However, if you run the Exchange 2013 version of the script from Exchange 2013 targeting an Exchange 2016 server, you will get an error as shown below:

It appears that the Exchange 2013 script invokes a remote PowerShell call to the Exchange 2016 server because we can see a warning that the Get-ClientAccessServer cmdlet is being deprecated. Exchange 2013 would not know this. The script then fails due to a serialization error.

The fix is to copy the 2013 version of the RollAlternateServiceAccountPassword.ps1 script to your Exchange 2016 servers and run it from there. You will see several warnings that the Get-ClientAccessServer cmdlet is being deprecated, but the script will complete successfully.

Another gotcha in the article, Configuring Kerberos authentication for load-balanced Client Access servers is in the step to Enable Kerberos authentication for Outlook clients. This step instructs you to enable Kerberos for the MAPI virtual directory using the following command:
Get-MapiVirtualDirectory -Server CAS-1 | Set-MapiVirtualDirectory -IISAuthenticationMethods Ntlm, Negotiate
If you are a hybrid customer or have configured OAuth for any other reason, the command above will remove OAuth as an authentication method for IIS. A better way of doing this would be to use the following two commands:
$server = Get-MapiVirtualDirectory –Server CAS-1
$server | Set-MapiVirtualDirectory -IISAuthenticationMethods ($server.IISAuthenticationMethods += "Negotiate")
This will add Negotiate (Kerberos) to the existing IIS authentication methods if it does not already exist.

Read more ...

Set-AutodiscoverSCP.ps1 script is now on the TechNet Gallery

Thursday, October 8, 2015
How many times have you installed a new Exchange 2010-2016 server only to hear users complain about a security pop-up in Outlook referencing the new server?

This happens because Exchange setup uses the FQDN of the server as the service connection point (SCP) that Outlook clients use for autodiscover requests (for example, https://exch03.contoso.local/autodiscover/autodiscover.xml). This new SCP is configured when the Front-End Client Access role or components are installed during setup. In most load balanced environments the valid SCP should be something like https://autodiscover.contoso.com/autodiscover/autodiscover.xml. Outlook will prompt users with a security warning because the server FQDN is not on the Exchange certificate and it is not trusted.


Older versions of Outlook (~Outlook 2010 RTM and earlier) used to use the oldest SCP value in the AD site, but newer versions use the newest SCP for foreground and background Autodiscover requests, causing these errors.

I wrote a script, Set-AutodiscoverSCP.ps1 (available in the TechNet Gallery), that automatically updates the SCP for the server your specify to the value you provide as soon as the new SCP for that server is detected in AD. It will continually poll Active Directory until it finds the new SCP value and sets it to the one you specify. A progress bar indicates that the script is polling AD.

The script is intended to run on another Exchange server in the org running the same version of Exchange as the new server. This is because Exchange 2010 cannot update SCP values for Exchange 2013 or 2016, and vice versa. You can also have the script target a particular domain controller. This is useful when the new server you are installing is in a different AD site.

The syntax for Set-AutodiscoverSCP.ps1 is:
Set-AutodiscoverSCP.ps1 [-Server] <String> [-NewSCP] <String> [[-DomainController] <String>] [<CommonParameters>]

Two examples of usage:
PS C:\>Set-AutodiscoverSCP.ps1 -Server exch01 -NewSCP https://autodiscover.contoso.com/autodiscover/autodiscover.xml
Example #1 continually queries the local Active Directory domain until it finds an SCP for server EXCH01 and then sets that SCP to https://autodiscover.contoso.com/autodiscover/autodiscover.xml.

PS C:\>Set-AutodiscoverSCP.ps1 -Server exch01.contoso.local -NewSCP https://autodiscover.contoso.com/autodiscover/autodiscover.xml -DomainController dc03.contoso.local
Example #2 is almost the same as the command in the previous example, except it continually queries DC03.CONTOSO.LOCAL for the SCP record and configures it on that domain controller. This is useful when configuring the SCP for a new Exchange server in a different Active Directory site.

I’ve included error handling for the following conditions:
  1. The script notifies you what version of Exchange is running the script and warns you to make sure the new server is running the same version. Note that Exchange 2013/2016 servers can update each other. This warning really only applies to Exchange 2010 and Exchange 2013/2016 coexistence. 
  2. The script checks that the server you want to configure is pingable. If the server cannot be pinged, the script will terminate.
  3. If a domain controller is specified, it validates that the DC specified is actually a domain controller

Set-AutodiscoverSCP.ps1 is a useful addition to your Exchange toolbox. Please let me know if you have any questions or feature requests. I'll update the script on the TechNet Gallery as needed.

Read more ...

Second Edition of Office 365 for Exchange Professionals

Thursday, September 10, 2015
I am very pleased to announce that the second edition of Office 365 for Exchange Professionals will be released next week at the IT/Dev Connections conference in Las Vegas, where all four of us will be speaking.

This self-published eBook is the most comprehensive publication available of all things Office 365. Over 150 pages of new content has been added since the first release at Microsoft Ignite in May 2015. The book covers new features and capabilities of Microsoft's cloud offering from an Exchange professional's perspective. You will learn how to take advantage of all the cloud has to offer in this comprehensive but surprisingly easy to read book.

The MVP co-authors are Tony Redmond, Michael Van Horenbeeck, and Paul Cunningham. Once again I provide my services as technical editor for this 770+ page book. Together, we bring over 60 years' experience in the IT and messaging industry. We have spent many hours exploring and understanding, participating in technical preview programs, and developing best practices in order to update this second edition with brand new content.

You can preorder Office 365 for Exchange Professionals from Paul Cunningham's ExchangeServerPro.com website or for your Kindle on Amazon.

If you are attending the IT/Dev Connections conference please join us for our second edition book launch party hosted by Binary Tree.

Read more ...

Fix for MSExchange Unified Messaging Event ID 1423 "A TLS API failure occurred. Error = 0x80090331"

Thursday, August 20, 2015
I was working with a customer who is implementing Unified Messaging in an Exchange 2013 CU7 environment with Lync 2013. All Exchange servers are running on Windows Server 2012 R2. Exchange UM was configured properly, including setting the Unified Messaging Call Router and Unified Messaging services to "Dual" and configuring valid SSL certs for both UM services. The dial plans and Auto Attendants were created and they started to test.

Calls made to the UM enabled user would go to voicemail, the greeting would play, and the caller could record a voicemail, but the voicemail would not be delivered to the UM user's mailbox. We turned up UM logging using the following cmdlet:
Get-EventLogLevel "msexchange unified*" | Set-EventLogLevel -Level Expert
Another voicemail test showed the following MSExchange Unified Messaging event ID 1423 - A TLS API failure occurred. Error = 0x80090331.

A quick look in the %ExchangeInstallPath%\UnifiedMessaging\voicemail folder on the mailbox server hosting the UM user's mailbox showed all the voicemail WAV files queued for delivery.

I'm familiar with error code 0x80090301 on this event ID, which is caused by too many root certificates in the Trusted Root Certification Authorities store, but this code and error sting is different. 0x8009331 means, "SEC_E_ALGORITHM_MISMATCH - The client and server cannot communicate, because they do not possess a common algorithm." Definitely sounds like a TLS negotiation problem.

After double-checking the UM configuration and the SSL certificate configuration used by the UM services it was found that TLS 1.0 was disabled for clients on the Exchange 2013 servers. The registry key HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.0\Client\DisabledByDefault was set to 1. This key doesn't exist by default and was not being configured via GPO, so it must have been configured in their server build. Setting the value to 0 (or deleting the DisabledByDefault key) fixed the problem.

The problem occurred because the UM server acts as both a client and a server to itself. The UM service acts as a client when it reads the voicemail WAV files from the voicemail folder and uses TLS to do so.

Note that there were several improvements to TLS and SSL in Exchange 2013 CU8 and Exchange 2010 SP3 RU9. These versions support TLS 1.1 and 1.2, which should also work. Please read Scott Landry's excellent article, Exchange TLS & SSL Best Practices, for more information.

Read more ...

Exchange 2013 Migration Batch Completes Successfully, But Mailbox is Not Moved

Friday, August 14, 2015
Here's another weird one. I'm testing a mailbox move from Exchange 2013 to Exchange 2016 (beta) for a mailbox with an in-place archive. I can create a new migration batch for the user and it completes in short order, but the mailbox and archive are not moved. The number of Total, Synced, Failed, and Successful mailboxes for the batch is zero. I get a notification email saying the batch completed successfully, but no mailboxes were moved or synced.

"Successful" Migration Batch Notification
When I look at the details of the migration batch in EMS I see that the ValidationWarningCount is 1 and the ValidationWarnings show, "Error:The user already exists, but the migration batch that includes it couldn't be found. Before you try migrating the user within a batch again, please remove the existing user by running the Remove-MigrationUser cmdlet."

Migration Batch Validation Warnings
Whoever wrote that error message deserves a raise. MUCH better than the "Error code 2" I'm used to getting.

A "MigrationUser" object is created for each user who is in the process of being migrated or the migration has not been completed, so it remains in a syncing state. I ran the Get-MigrationUser cmdlet which revealed that the mailbox was indeed "stuck" in a migration since 7/20/2013(!!) with no other issues.

This mailbox has been "migrating" for over two years!
So, I removed the migration user using the Remove-MigrationUser cmdlet. I needed to use the -Force parameter since the move request no longer exists.

Remove-MigrationUser clears the way to re-run the migration
This fixed the problem and I was able to move the mailbox and archive successfully. This condition would prevent a mailbox from moving to any other database or Office 365. It's strange that Exchange doesn't detect this when the batch request is created.

Read more ...