Wednesday, April 16, 2014

Tips for Connecting to Office 365 using PowerShell in Hybrid Environments

The Office 365 portal and Exchange Admin Console are fairly powerful to allow you to manage your tenant and on-premises environments. But as you are no doubt aware, there are many administrative tasks that require you to use PowerShell.

The sequence you usually find on the web to connect to Office 365 via PowerShell is:
[PS] C:\>$LiveCred = Get-Credential -credential
[PS] C:\>$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri -Credential $LiveCred -Authentication Basic -AllowRedirection
[PS] C:\>Import-PSSession $Session
If you are in a hybrid environment with Exchange and Office 365 you will discover that both environments have a lot of the same cmdlets, such as Get-Mailbox, Set-DistributionGroup, etc. This causes a conflict when the Office 365 PowerShell cmdlets are loaded within the Exchange Management Shell. You either need to connect to Office 365 PowerShell from a regular PowerShell console (separate window) or you need to use the -AllowClobber parameter, which overwrites the existing EMS cmdlets with the Office 365 versions. This is not ideal if you are working with both on-prem and cloud objects at the same time.

Proxy creation has been skipped for the following command ... Use the AllowClobber parameter if you want to shadow existing local commands
I wrote a PowerShell script called Connect-Office365.ps1 that overcomes these conflicts by using the -Prefix parameter with the Import-PSSession cmdlet. The Prefix parameter tells PowerShell to add the specified prefix to all cmdlets it loads from Office 365. For example, if you set the prefix to "cloud" the Get-Mailbox cmdlet for Office 365 becomes Get-cloudMailbox and the Get-Mailbox cmdlet still applies to on-prem. This way you can use both sets of cmdlets in the same EMS console. The script also connects to the MSOLService so you can use the MSOL cmdlets to manage Azure AD.

Connect-Office365.ps1 with "cloud" prefix
Here's the four line Connect-Office365.ps1 script:
$LiveCred = Get-Credential -credential ""
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri -Credential $LiveCred -Authentication Basic -AllowRedirection

Import-PSSession $Session -WarningAction SilentlyContinue -Prefix cloud

Connect-MsolService -Credential $LiveCred
I usually copy the script to the C:\Windows folder on my Exchange servers and my management computer so it can be run from any directory whenever I need it.

Thursday, April 10, 2014

The 'Heartbleed' Security Flaw - Are You Affected?

(CNN) -- A major online security vulnerability dubbed "Heartbleed" could put your personal information at risk, including passwords, credit card information and e-mails.

Heartbleed is a flaw in OpenSSL, an open-source encryption technology that is used by an estimated two-thirds of Web servers. It is behind many HTTPS sites that collect personal or financial information. These sites are typically indicated by a lock icon in the browser to let site visitors know the information they're sending online is hidden from prying eyes.

Cybercriminals could exploit the bug to access visitors' personal data as well as a site's cryptographic keys, which can be used to impersonate that site and collect even more information.
You can use the Heartbleed Test website ( to test your external websites and external-facing web appliances to see if they are vulnerable. I encourage you to make a quick test of your systems ASAP.

If you use Google Chrome, I encourage you to install the Chromebleed plug-in which displays a warning if the site you are browsing is affected by the Heartbleed bug.

Tuesday, April 8, 2014

Notes on the Windows 8.1 Update for Windows Server 2012 R2

Microsoft has released the Windows 8.1 Update to MSDN subscribers and is due to be released today to the general public. This update also applies to Windows Server 2012 R2 servers and includes some important updates, including one that will be required so that Windows 8.1 and Windows Server 2012 R2 computers will continue to receive important security updates.

The Windows 8.1 Update (no number, as if there will never be another) is actually a series of 6 updates which should be installed in a certain order.  I created an installer batch file that installs the updates for Windows Server 2012 R2 in the recommended order with minimal output.

Copy the following text to Notepad and save it as Install.bat in the same folder as all the x64 patch files:
start /wait Windows8.1-KB2919442-x64.msu /quiet /norestart
start /wait Windows8.1-KB2919355-x64.msu /quiet /norestart
start /wait Windows8.1-KB2932046-x64.msu /quiet /norestart
start /wait Windows8.1-KB2937592-x64.msu /quiet /norestart
start /wait Windows8.1-KB2938439-x64.msu /quiet /norestart
start /wait Windows8.1-KB2949621-v2-x64.msu /quiet /norestart
@echo Please restart the computer now.
And here's a version for Windows 8.1 x86:
start /wait Windows8.1-KB2919442-x86.msu /quiet /norestart
start /wait Windows8.1-KB2919355-x86.msu /quiet /norestart
start /wait Windows8.1-KB2932046-x86.msu /quiet /norestart
start /wait Windows8.1-KB2937592-x86.msu /quiet /norestart
start /wait Windows8.1-KB2938439-x86.msu /quiet /norestart
start /wait Windows8.1-KB2949621-v2-x86.msu /quiet /norestart
@echo Please restart the computer now.
Once you install the update and restart the computer you'll immediately notice that the Windows App Store icon is on your taskbar.

Windows App Store is added to the Windows 2012 R2 Taskbar

If you're running Windows Server 2012 R2 and signed in the the Admin account you'll find that the Store won't open and gives the error, "This app can't open. Store can't be opened using the Built-in Administrator account. Sign in with a different account and try again." Doh! Right-click the icon and unpin it from the taskbar to recover that real estate.

This dawg won't hunt
It's interesting to see Microsoft slowly unraveling the "Modern UI" to behave more like the desktop UI. They've added a power button to the Modern UI desktop so users can easily shutdown or restart the computer and added a search icon because users didn't intuitively know that they can just start typing to search.

Wednesday, March 26, 2014

MEC 2014 is Right Around the Corner! Are You Registered?

I'm very much looking forward to the Microsoft Exchange Conference March 31-April 2 in Austin, TX. I hope you can join me there!

I'll be moderating three MEC Unplugged interactive sessions this year and will be on the experts panel for the Exchange Deployment session.  It's important to know that the Experts Unplugged sessions will not be recorded, so if you want to see them or participate in the discussion you'll need to register for MEC.

Session Date/Time Room Speakers
Experts Unplugged: Architecture – Client Access and Connectivity  Tuesday, April 1 9:00AM - 10:15AM MR 17b Greg Taylor, Jeff Guillet, Jeff Mealiffe, Ross Smith IV, Venkat Ayyadevara
Experts Unplugged: Architecture - Transport and Hygiene Tuesday, April 1 10:45AM - 12:00PM MR 17b Brian Reid, Jeff Guillet, Khushru Irani, Ross Smith IV, Scott Landry, Wendy Wilkes
Experts Unplugged: Architecture - Transport and Hygiene (repeat session) Wednesday, April 2 8:30AM - 9:45AM MR 13ab Brian Reid, Jeff Guillet, Khushru Irani, Ross Smith IV, Scott Landry, Wendy Wilkes
Wednesday, April 2 8:30AM - 9:45AM MR 17b Brian Day, Greg Taylor, Jeff Guillet, Jeff Mealiffe, Ross Smith IV, Scott Schnoll

Click the sessions above to add them to your MEC schedule. If you haven't registered for MEC yet, it's not too late.

There will be no paper copy of the schedule this year. The schedule will be available via an HTML5 app that should work on all platforms, but I suggest you print a copy of your schedule or add it to your calendar before you arrive. Technology sometimes has a nasty way of not working when you need it.

Here's a breakdown of the current attendee profile, based on registered attendees so far.

With 87% of the attendees from the US, it looks like Europe would definitely be served well by having its own MEC. That would better align with the global deployment of Exchange Server. I'd say the chances of seeing MEC in Europe would be pretty slim, though.

Monday, March 17, 2014

Announcing the 7th Annual UC Roundtable at TechEd 2014, Houston!

I'm pleased to announce the 7th Annual UC Roundtable at Microsoft TechEd North America 2014 in Houston, TX!

The purpose of the UC Roundtable is to gather Exchange and Lync admins, MCMs, MVPs, Exchange product group members, architects, and experts for a free-flowing discussion about issues, questions, and experiences related to Exchange, Office 365, and Lync Server.  If you work with Exchange, Office 365, or Lync you need to be here!

The UC Roundtable will be held Wednesday, May 14, 2014 from 6:00-8:00PM CDT and will be within walking distance or a short cab ride from the TechEd hotels. A special thanks to my friends at F5 who will be hosting the event for the third year in a row!

Please RSVP to for event details and location. I will email you with the location details once they're set.

Help spread the word and I hope you can join me!

Tuesday, March 11, 2014

Pending Messages in Exchange Online Will Not Be Delivered

There’s a pretty big unpublicized bug in Office 365 transport that you need to be aware of.

If emails cannot be delivered due to a problem with TLS those emails will queue in Exchange Online Protection (EOP) and will be marked as Pending or Deferred. If you change the Outbound Connector in EOP to work around the problem these messages will NEVER be delivered, even after the transport issue is resolved. This happens because the outbound messages in Office 365 are stamped with the old TLS configuration and are not reevaluated when the Outbound Connector configuration is changed.

The following example shows a message that was sent at 2:46PM UTC and the status shows as Pending. The last event for this message shows DEFER at 3:29PM UTC with the detail, "The last attempt to deliver the message encountered an error".

Sample Message Trace from Office 365
For example, say the TLS certificate expires on your hybrid server. Inbound messages from EOP to the hybrid server will queue because the Outbound Connector is using Forced TLS, but the certificate is invalid. If you resolve the problem by reconfiguring the Office 365 Outbound Connector to use Opportunistic TLS the problem is solved for new emails - they get delivered right away, but the Pending messages will never get resubmitted and eventually expire after 48 hours.

This same behavior would occur if you have a custom Outbound Connector that forces TLS with a business partner. If their TLS certificate expires or they reconfigure their server to not use TLS. The messages will not be resubmitted to use the new configuration.

Incredibly, there currently is no way for Microsoft to resubmit these messages like there is for on-prem Exchange. After opening a high-priority case with Microsoft Online, the only "solution" they could give is to contact all the senders and ask them to resend their emails.

To work around this, you may want to ensure that your Outbound Connectors that use TLS are configured to use opportunistic TLS. That way if something changes in the TLS configuration, such as the receiving server's cert expires or TLS is disabled, emails can still be delivered.

Monday, March 10, 2014

Troubleshooting TLS SMTP Connections to Exchange Online Protection

By default Office 365 uses Transport Layer Security (TLS) to send encrypted SMTP emails between Exchange Online and Exchange on-prem. This provides end-to-end encryption of emails between your on-prem Exchange Hybrid Server and Exchange Online Protection (EOP), just like they were the same organization.

TLS utilizes x.509 (SSL) certificates to encrypt the email in transport. Exchange Online and Exchange 2013's implementation of TLS differs from previous implementations of TLS in several important ways:
  • The certificates used for TLS must be issued by a trusted third-party CA (Digicert, Godaddy, Verisign, etc.)
  • The CRL for the certification authority must be available. If Exchange or O365 can't read the CRL it will not trust the certificate.
  • The FQDN that the Receive Connector provides in response to EHLO must match the subject name or a subject alternative name on the certificate. SAN certificates and wildcard certificates are both valid for TLS use. If you use a wildcard cert the FQDN used on the connector can be any name that is valid for the wildcard cert's domain.
  • In previous versions of TLS any certificate could be used to encrypt SMTP traffic, even expired or self-signed certificates. This is not the case with Office 365 and Exchange 2013.
As previously mentioned, TLS is required by default for SMTP communication between Hybrid servers and Exchange Online Protection (EOP). When you run the Hybrid Configuration Wizard it will configure forced TLS for all Send and Receive Connectors. Note that this configuration is updated whenever you run the HCW, so if you reconfigure the connectors for opportunistic TLS or turn it off completely, the HCW will reconfigure them to use forced TLS again.

The following options are available for TLS encryption in Office 365:
  • Force TLS - Email sent across this connector must use TLS. If TLS is unavailable messages will queue until they are delivered or expired.
  • Opportunistic TLS - The connector tries to setup a TLS connection using the STARTTLS verb. If a TLS connection cannot be made the connector falls back to regular ESMTP or SMTP. This is functionally equivalent to turning TLS off.
Normally the use of TLS is configured on Receive or Inbound Connector. You do this to control how email is accepted by your domain and it can be easily configured using the Exchange Admin Console. 
Office 365's Inbound Connector from Hybrid Server
Send Connectors can also be configured to require TLS. This is used to enforce that email is sent by this connector must only use TLS, and is configured in the shell using the RequireTLS property:
Outbound Connector TLS Settings to Office 365
Using this configuration if the corresponding Receive Connector does not offer or require TLS, messages will queue on the sending server until they are finally delivered or expire.

You can easily tell if a receiving SMTP server is configured to use TLS using Telnet. Install the Telnet Client feature, Telnet to the server using TCP port 25, and look for the STARTTLS verb after issuing the EHLO command. For example:
Telnet servername 25
If you see that the STARTTLS verb is missing, the server is not offering TLS. If your Send Connector is configured to require TLS, the messages will queue on the sending server with the error,
451 4.4.0 Primary target IP address responded with: "451 5.7.3 STARTTLS is required to send mail."
There are several reasons that the STARTTLS verb might be missing:
  • The receiving server is not configured to Force TLS or use Opportunistic TLS.
  • The sending server's IP is on an SMTP block list (aka SMTP blacklist or SMTP blocklist). Office 365 will not attempt to send TLS traffic with a server it can't trust.
  • The receiving server is configured to only respond to SMTP (not ESMTP) commands. TLS is part of the ESMTP protocol.
  • Your firewall is doing some form of email inspection and is filtering "unsafe" verbs from the SMTP conversation. Some examples of firewalls that do this are:
Office 365 uses internal and external SMTP blocklists to protect the network. If the sending server is on one of these black lists the EOP servers will not offer TLS. You can test this by sending a non-TLS email to EOP using Telnet. See Brian Reid's article, "Cannot Send Emails to Office 365 or Exchange Online Protection Using TLS". The response from the EOP server will tell you which block list you're on and how to request removal.

If the receiving server requires TLS, but the sending server is not configured to use a TLS certificate messages will queue on the sending server with the error,
451 4.4.0 Primary target IP address responded with: "451 5.7.3 Must issue a STARTTLS command first."
The fix here is to configure the Send Connector to use TLS and a valid certificate using the Enable-ExchangeCertificate cmdlet, such as:
Enable-ExchangeCertificate -Thumbprint 5DC5902752816FD2FC51D5564C363F68D8F7FFC4 -Services SMTP
Make sure to use Get-ExchangeCertificate to get the correct certificate's thumbprint for the command above.

Hopefully this information will help you understand TLS and will assist you with troubleshooting.