Document previews using Cloud Hybrid Search

Summary: This blog post describes how to get document previews for SharePoint Online and SharePoint on-premises results after configuring Cloud Hybrid Search.

After setting up Cloud Hybrid Search, you might find that document previews are working for your SharePoint Online content, but not for your on-premises content.
No_document_preview

Microsoft uses Office Online Server (OOS) in Office 365 to enable document previews for your SharePoint Online files.
Unfortunately, it is not possible to leverage OOS from Office 365 for your on-premises content.
This is why deploying Office Web Apps or Office Online Server is required in order to get document previews for your on-premises content.

Choosing between Office Web Apps and Office Online Server
When choosing the right product for your organization, keep in mind that Office Web Apps is not supported for SharePoint 2016.

  Office Web Apps Office Online Server
SharePoint 2013                     
SharePoint 2016                     

This article describes the configuration of Office Online Server for SharePoint 2016. The configuration for SharePoint 2013 is exactly the same: https://docs.microsoft.com/en-us/officeonlineserver/configure-office-online-server-for-sharepoint-server-2016/configure-office-online-server-for-sharepoint-server-2016.

After configuring Office Online server for your on-premises SharePoint farm, you will notice that document previews start working!

Document_preview

Considerations
– If you have multiple on-premises SharePoint farms, you can connect all these SharePoint farms to a single Office Online Server farm.
– Make sure to make the Office Online Server farm accessible from outside the organization network if required.
– Please use HTTPS for Office Online Server.

Office 365 Mail pushing Microsoft Teams

Office 365 Mail has an option that allows you to chat with your contacts using Skype for Business without leaving the browser. This feature was announced in June 2015: https://blogs.office.com/en-us/2015/06/08/new-user-experiences-in-office-365-on-the-web/

Office 365 Mail Banner

However, Microsoft changed the way this works for some tenants.
Clicking the Skype for Business icon now, results in the following:

Microsoft Teams integration

It pushes me to Start using Teams. Clicking “Start using Teams” opens a new tab that navigates me to https://aka.ms/weblaunchclient. This redirects me to https://teams.microsoft.com. Without opening the “Chats” tab.

This change affects all users, even users that are not in first-release mode. Even when Teams licenses are not activated for the user, it still pushes for Microsoft Teams, which results in an error.

I couldn’t find any announcement by Microsoft that this change was about to happen, so we had no way to test this new “feature” and communicate this with our employees. It would be great to have a way to disable this feature, to allow people to use the in-line chat when Microsoft Teams is not enabled in their tenant.

Update: After refreshing my page, it goes back to the old user interface, opening the in-line Skype for Business chat functionality. It seems this was a temporary bug/feature.

But still, the text was pretty clear: “Skype for Business is now Microsoft Teams!“.
We’ll see what happens in the future.

All you need to know about Hybrid Auditing in SharePoint 2016

Summary: This blog post will show you how to configure Hybrid Auditing in SharePoint 2016. It will also point out some considerations when deploying this new feature.
Microsoft released a new hybrid feature for SharePoint 2016: Hybrid Auditing. This feature will automatically upload your on-premises user activity logs to Office 365, so administrators can generate reports for users across SharePoint on-premises and Office 365.

* Note: This feature is still in preview! The configuration and capabilities might change in the future.

Configuring Hybrid Auditing

Microsoft did a great job documenting how to configure this new feature for SharePoint 2016. The steps are outlined in this article: https://technet.microsoft.com/en-us/library/mt622371(v=office.16).aspx. I am not going over every step, I am just summarizing the steps below:

  1. Meet the prerequisites
  2. Turn On Audit Log Search Recording
  3. Run the Hybrid picker and select Hybrid Auditing
  4. Check Audit Log Report

Thanks to Vlad Catrinescu who reminded me that you need to restart the Microsoft SharePoint Insights service after patching your SharePoint 2016 farm.

You can do this by running the following PowerShell code:

Add-PSSnapin Microsoft.SharePoint.PowerShell 
Stop-SPService -Identity "Microsoft SharePoint Insights" -IncludeCustomServerRole 
Start-SPService -Identity "Microsoft SharePoint Insights" -IncludeCustomServerRole 

Verify your configuration

If you want to verify that the configuration was successful, here are some tips:

Get Microsoft SharePoint Insights configuration
If you want to make sure that you Hybrid Auditing configuration was done correctly, you can use the following PowerShell cmdlet to see the current configuration:

Get-SPInsightsConfig

This will show you the current configuration for your Hybrid Auditing feature. This might also help you to find any issues you’re facing.

Configure usage and health data collection
In Central Administration, under Monitoring -> Configure usage and health data collection make sure “Enable usage data collection” is checked.
For now I am not sure which checkboxes are required for the hybrid functionality, but these are the ones I have active at the moment.
Hybrid-Auditing-UsageHealth

Troubleshooting

After configuring Hybrid Auditing, I found that I wasn’t getting any on-premises results from the Office 365 Security & Compliance center.
Here you can find some issues that I ran in to and the solution for these issues.

Failed to start a service (Microsoft SharePoint Insights service) that is needed for Hybrid Auditing (Preview) scenario

After running the Hybrid picker I would get this error. It seems that there is a time-out when the picker tries to stop and start the Microsoft SharePoint Insights service. To get around this error, make sure the Microsoft SharePoint Insights service is started from the Services on Server or Services in Farm menu in Central Administration.

I have changed my log location in my on-premises farm
Whenever you change your log location, this change is not automatically picked up by the Microsoft SharePoint Insights service.
Instead, you should run the Hybrid Picker again, then restart the Microsoft SharePoint Insights service from Central Administration.
Your logs will appear in Office 365 after performing these steps.

Results from on-premises aren’t shown in the Office 365 Security & Compliance Center

During my testing, I found that the user mapping isn’t done correctly. As this is a preview feature, I am hoping this will be resolved when the feature will be GA.

I have a user named Kim Akers. In Active Directory, this user is known as:

  • Userprincipalname: kimakers@sharepointrelated.com
  • User logon name (pre-Windows 2000): sprelated\kimakers

Kim works in both SharePoint 2016 (on-premises) and SharePoint Online.
Opening the Security & Compliance Center in the Office 365 Admin Portal, the “Users” field automatically resolves “Kim” as “Kim Akers”. See the screenshot below as a reference:
Hybrid-auditing-kim

The results coming back for this search only show the SharePoint Online activities for Kim.
This is because the results for the on-premises activities for Kim are actually displayed under the user sprelated\kimakers.
hybrid-Auditing-kim-results

Until now, I haven’t found a way to find results for on-premises users directly. The only way to find on-premises activities is to leave the “Users” field empty. This means you will get all results, without any user filter. This makes it hard to find the activities for on-premises users.

If you have any trouble configuring Hybrid Auditing, contact me on Twitter or LinkedIn and I will help wherever I can.

Hybrid taxonomy considerations

Summary: This post will describe all things to keep in mind when configuring Hybrid Taxonomy.

Microsoft released Hybrid Taxonomy in preview for SharePoint 2013 and SharePoint 2016 on-premises. This makes it possible to replicate terms between your on-premises SharePoint farm and SharePoint Online. For more information on hybrid taxonomy, see https://support.office.com/en-us/article/Configure-hybrid-SharePoint-taxonomy-Preview-dd287a75-09e0-403e-974e-4cc84a152815

1. Term limit

In SharePoint on-premises (2010, 2013 and 2016) the maximum number of items in a term store is 1.000.000. For more information, see: https://technet.microsoft.com/en-us/library/cc262787.aspx#termstore

In SharePoint Online, the maximum numbers in your term store is only 200.000.

This means that you can only have a maximum of 200.000 terms in your on-premises Managed Metadata Service Application if you are planning to use hybrid taxonomy. Take this into consideration when you configure hybrid taxonomy.

2. Preview

Hybrid taxonomy is currently still in preview. Keep in mind that things might change along the way.

If I find any more considerations I will update this post.

Hybrid features in SharePoint 2013 and 2016

Summary: This post provides an overview of all hybrid SharePoint features that were released by Microsoft for SharePoint 2013 and SharePoint 2016.

During Ignite 2016 in Atlanta, Microsoft released some really cool hybrid features, that I would like to share some information about. The really cool thing about this is that they are not only available for SharePoint 2016, but Microsoft actually made most of them available in SharePoint 2013. The following table will show the availability per feature, so you know which one is available to your environment.

For more information on any specific hybrid feature, click the feature in the table below.

(1) Breaks ALL existing server-to-server trusts. Provider-hosted add-ins are the most commonly found that use server-to-server trust. Make sure to read this blog post for a solution.
(2) There have been major improvements in the CU’s after the initial August 2015 CU for Cloud Hybrid Search. I advise downloading the last CU that has no regressions.

In the last months I have been actively configuring and testing hybrid capabilities in SharePoint 2013. If you have any questions during configuring hybrid features in SharePoint, make sure to contact me on Twitter for the fastest response! I’ll be glad to help with any question.

Fixing apps after configuring SharePoint Hybrid

Update: Microsoft included the fix in the hybrid picker experience. This means you no longer have to perform the steps outlined in this blog post.
You can find the updated article by Microsoft here: https://blogs.technet.microsoft.com/beyondsharepoint/2016/09/15/considerations-when-deploying-sharepoint-office365-hybrid-workloads-in-a-farm-utilizing-provider-hosted-add-ins-or-workflow-manager/

For hybrid search (outbound/inbound query federation or Cloud Hybrid Search Service Application) a manual approach is needed to remediate this scenario.
A KB article was released, which can be found here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error


Summary: This article provides a solution to broken provider-hosted add-ins after configuring SharePoint hybrid features. For a full list of hybrid features, see the following article: https://www.sharepointrelated.com/2016/10/04/hybrid-features-sharepoint-2013-and-2016

The following hybrid features will break your server-to-server trusts that were already set up before configuring hybrid for SharePoint 2013 or SharePoint 2016:

This post will describe why this happens and how we can fix this.

In order to establish a server-to-server trust between your on-premises SharePoint environment and Office 365, Microsoft relies on the SPAuthenticationRealm. More information can be found here: https://technet.microsoft.com/en-us/library/jj219756.aspx.

This article has a “Caution” section, warning that any access tokens created for a specific realm, won’t work after changing the SPAuthenticationRealm.
SharePoint hybrid

To fix this, I wrote a script that gives you 2 options:

Undo Fix
Reverse the changes made by configuring Hybrid. It will change the SPAuthenticationRealm back to the old value. All SharePoint hybrid features stop working. All your provider-hosted add-ins will work again. This option will try to change your SPTrustedSecurityTokenIssuers so that it uses the new SPAuthenticationRealm set by configuring hybrid.

CautionThere are some notes that I described later in this post, make sure to read them.

Running the script will result in something like this:
SharePoint hybrid
Running the Fix-Hybrid.ps1 script

You can download the script here:
Fix-Hybrid

Notes
If you choose to fix your SPTrustedSecurityTokenIssuers, you will need to do some additional work to have everything work again.

  • Regrant app permissions

App permissions rely on the SPAuthenticationRealm.
This means that any App permissions that you set, will be gone after updating your SPTrustedSecurityTokenIssuers.
You will have to register the apps again and assign the permissions to the app.
The following script can do this for you (the current script is app-instance based, this means you have to run it for every app instance.
Also, make sure to change the variables in the script before running it.
Set-SPAppPermissions

  • Workflow Manager

Workflow Manager also relies on the SPAuthenticationRealm. Thanks to Ruben de Boer for proposing the solution.
After running the Fix-Onboarding.ps1 script, make sure to remove the existing Workflow Service Application Proxy.
Then run the Register-SPWorkflowService cmdlet again. Make sure to use the same scope that you used before. I recommend using the -Force parameter.

I hope this helps anyone! Do not hesitate to contact me if you have any trouble using the script of have any questions.

Microsoft IT Pro Cloud Essentials program benefits

Summary: This blog post shows the benefits that come with the Cloud Essentials program that was released by Microsoft.

Microsoft released a new program that gives IT Pros the opportunity to enhance their skill set by giving free access to Microsoft resources.

IT Pro Cloud Essentials program

Below is an overview of resources that will be available to you after you sign up at https://itprocloudessentials.com:

  • Enterprise Mobility Suite (3 months access instead of 1 month)
  • Extended Office 365 trial (60 days instead of 30 days)
  • 100 Azure credits every month, for 3 months. Microsoft is looking into extending this with a lower credit rate after 3 months.
  • Recommended test scenarios in Azure:
    * Iaas
    * Security
    * Enterprise mobility
  • 3 months subscription for selected Pluralsight courses on Microsoft Azure
  • Free Microsoft exam voucher
  • Priority support on TechNet forums
  • 1 technical support call with a Microsoft professional for troubleshooting a specific problem or error message.

This is a great service by Microsoft and I would advise everyone to sign up and activate your subscription!

People picker settings for all Web Applications one-liner

Summary: This PowerShell one-liner will show the people picker settings for each Web Application, including Central Administration.

If you want to know the settings for the people picker in your SharePoint farm, you can run the following line of PowerShell.
It will retrieve all Web Applications and show the people picker settings for each one.

Get-SPWebApplication -IncludeCentralAdministration | %{Write-Host $_.url; $_.peoplepickersettings | select * | fl }

Running the one-liner will result in something like this:

People picker settings

Cloud hybrid search considerations

Summary: This blog post describes some limitations that you need to consider before implementing cloud hybrid search in your organization.

Do not hesitate to contact me on twitter or LinkedIn to see if there are any changes that are not reflected in this blog post.
For a full overview of how Cloud Hybrid Search works, read my blog post: Everything you need to know about Cloud Hybrid Search

1. Provider-hosted apps
All provider-hosted add-ins will break when running the onboarding script *
One part of the onboarding script will change the SPAuthenticationRealm for your entire SharePoint farm.
As all your SPTrustedSecurityTokenIssuers rely on this SPAuthenticationRealm, they stop working after running the onboarding script.

Microsoft released a new hybrid picker that includes a fix for this issue. If you are configuring Cloud Hybrid Search only using the onboarding script, you have to do some additional work to get your apps to work again. Microsoft released a KB article with the fix here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error

2. Search customizations
The Cloud Search Service Application shares a similar architecture as the native SharePoint Search Service Application.
However, customization is limited because the search experience is derived from Office 365.

Below is a table that shows the current limitations in search customizations when using Cloud Hybrid Search.
Hybrid Cloud Search customizations

3. Searching on Non-default Zone URL’s
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Crawling Happens on-premises and the Default Zone url which you are crawling is fed to the Index on the Cloud. Since the Query is being served from Components in the Cloud which have no knowledge of the alternate URL’s available for On-prem Web-application, no URL Translation happens.
In Fact there is No concept of AAM’s in cloud Space.

The Only way to work around these limitations is to ensure that:

  1. The Public Default Zone URL should be a FQDN URL or The END URL which you want to Show up to the users in search results.
  2. The Default Zone URL which is being crawled should have Windows Claims Auth so that ACL information sent along with crawled Content is recognized by Search Components correctly.
  3. The User Needs to Perform Search from a URL while logged in windows Identity , so that it can be resolved effectively to validate Claims & Security Trimming to work effectively.

Note : Please ensure that Public URL which you want end users to see , should be accessible to Crawler on premise to be able to crawl the site successfully.

4. Windows Claims Only
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Cloud SSA or To be Specific the Search Components in SharePoint online at present do not understand any other ACL type other than Windows Claims , This ACL information ,about the Content which is fed along the index is used for Security Trimming the results . SAML Identity Providers are not supported with Cloud SSA . So basically the On-prem Site you are crawling using the Cloud SSA should be using Windows Auth only .

if you search on an extended URL of Web-applications which is ADFS / LDAP or Some other type of Auth other than Windows Claims , No search results will be returned as the logged in user’s Identity cannot be resolved to ACL mapping we have on an Index Item.

5. Index item limits and pricing
Vlad Catrinescu blogged about this on his blog.
For every 1TB of pooled storage in SharePoint Online, we are allowed to put one million index items from our On-Premises SharePoint Farm.

You can check your current searchable items from the Search Service Application in Central Administration
Cloud hybrid search considerations

In this example, we would need 13TB of pooled storage in SharePoint Online. This might mean that you have to reconfigure your content sources.

6. Security and compliancy
As your index is stored in Office 365, what does this mean compliancy wise?
This is the answer from Microsoft:

The content that is passed from on-premises to the azure cloud search connector (SCS) consists of crawled properties, keywords, ACLs, tenant info and some other metadata about the item. This is encrypted on-premises using a key supplied by the SCS and transmitted to the endpoint in Azure. Once there it is stored in an encrypted blob store and queued for processing. We retain the encrypted package in the blob store for use should we need to issue a content recrawl. The encrypted object is not the document though, it is just a parsed and filtered version that makes sense to the search engine.

In my opinion it would be wise to consult with your legal department before setting up Cloud Hybrid search to make sure it is okay to store content in the cloud.
You can always modify the content sources to exclude highly classified documents.

7. Licensing and Office 365 accounts
Every user that wants to use the new Cloud Hybrid Search functionality will need an active Office 365 license and an account synchronized from your on-premises Active Directory.
Cloud-Hybrid-Search-Identity
As items are indexed in Office 365, the access control entities are looked up in the cloud directory service.
Hybrid-Search-FederatedAccount
User SIDs are mapped to PUIDs; Group SIDs are mapped to Object IDs; and and are mapped to .

8. Documentation
As the cloud hybrid search is still in preview, documentation is limited at best.
For example, if you are using a proxy for Internet access on your server, make sure to specify this in your machine.config.
I have created a more detailed blog post around this issue here: https://www.sharepointrelated.com/2015/12/11/cloud-hybrid-search-proxy-settings/

The documentation for Cloud Hybrid Search has been greatly improved. You can find all information you need here: https://technet.microsoft.com/en-us/library/dn720906.aspx