All you need to know about Hybrid Auditing in SharePoint 2016

Summary: This blog post will show you how to configure Hybrid Auditing in SharePoint 2016. It will also point out some considerations when deploying this new feature.
Microsoft released a new hybrid feature for SharePoint 2016: Hybrid Auditing. This feature will automatically upload your on-premises user activity logs to Office 365, so administrators can generate reports for users across SharePoint on-premises and Office 365.

* Note: This feature is still in preview! The configuration and capabilities might change in the future.

Configuring Hybrid Auditing

Microsoft did a great job documenting how to configure this new feature for SharePoint 2016. The steps are outlined in this article: https://technet.microsoft.com/en-us/library/mt622371(v=office.16).aspx. I am not going over every step, I am just summarizing the steps below:

  1. Meet the prerequisites
  2. Turn On Audit Log Search Recording
  3. Run the Hybrid picker and select Hybrid Auditing
  4. Check Audit Log Report

Thanks to Vlad Catrinescu who reminded me that you need to restart the Microsoft SharePoint Insights service after patching your SharePoint 2016 farm.

You can do this by running the following PowerShell code:

Add-PSSnapin Microsoft.SharePoint.PowerShell 
Stop-SPService -Identity "Microsoft SharePoint Insights" -IncludeCustomServerRole 
Start-SPService -Identity "Microsoft SharePoint Insights" -IncludeCustomServerRole 

Verify your configuration

If you want to verify that the configuration was successful, here are some tips:

Get Microsoft SharePoint Insights configuration
If you want to make sure that you Hybrid Auditing configuration was done correctly, you can use the following PowerShell cmdlet to see the current configuration:

Get-SPInsightsConfig

This will show you the current configuration for your Hybrid Auditing feature. This might also help you to find any issues you’re facing.

Configure usage and health data collection
In Central Administration, under Monitoring -> Configure usage and health data collection make sure “Enable usage data collection” is checked.
For now I am not sure which checkboxes are required for the hybrid functionality, but these are the ones I have active at the moment.
Hybrid-Auditing-UsageHealth

Troubleshooting

After configuring Hybrid Auditing, I found that I wasn’t getting any on-premises results from the Office 365 Security & Compliance center.
Here you can find some issues that I ran in to and the solution for these issues.

Failed to start a service (Microsoft SharePoint Insights service) that is needed for Hybrid Auditing (Preview) scenario

After running the Hybrid picker I would get this error. It seems that there is a time-out when the picker tries to stop and start the Microsoft SharePoint Insights service. To get around this error, make sure the Microsoft SharePoint Insights service is started from the Services on Server or Services in Farm menu in Central Administration.

I have changed my log location in my on-premises farm
Whenever you change your log location, this change is not automatically picked up by the Microsoft SharePoint Insights service.
Instead, you should run the Hybrid Picker again, then restart the Microsoft SharePoint Insights service from Central Administration.
Your logs will appear in Office 365 after performing these steps.

Results from on-premises aren’t shown in the Office 365 Security & Compliance Center

During my testing, I found that the user mapping isn’t done correctly. As this is a preview feature, I am hoping this will be resolved when the feature will be GA.

I have a user named Kim Akers. In Active Directory, this user is known as:

  • Userprincipalname: kimakers@sharepointrelated.com
  • User logon name (pre-Windows 2000): sprelated\kimakers

Kim works in both SharePoint 2016 (on-premises) and SharePoint Online.
Opening the Security & Compliance Center in the Office 365 Admin Portal, the “Users” field automatically resolves “Kim” as “Kim Akers”. See the screenshot below as a reference:
Hybrid-auditing-kim

The results coming back for this search only show the SharePoint Online activities for Kim.
This is because the results for the on-premises activities for Kim are actually displayed under the user sprelated\kimakers.
hybrid-Auditing-kim-results

Until now, I haven’t found a way to find results for on-premises users directly. The only way to find on-premises activities is to leave the “Users” field empty. This means you will get all results, without any user filter. This makes it hard to find the activities for on-premises users.

If you have any trouble configuring Hybrid Auditing, contact me on Twitter or LinkedIn and I will help wherever I can.

Hybrid features in SharePoint 2013 and 2016

Summary: This post provides an overview of all hybrid SharePoint features that were released by Microsoft for SharePoint 2013 and SharePoint 2016.

During Ignite 2016 in Atlanta, Microsoft released some really cool hybrid features, that I would like to share some information about. The really cool thing about this is that they are not only available for SharePoint 2016, but Microsoft actually made most of them available in SharePoint 2013. The following table will show the availability per feature, so you know which one is available to your environment.

For more information on any specific hybrid feature, click the feature in the table below.

(1) Breaks ALL existing server-to-server trusts. Provider-hosted add-ins are the most commonly found that use server-to-server trust. Make sure to read this blog post for a solution.
(2) There have been major improvements in the CU’s after the initial August 2015 CU for Cloud Hybrid Search. I advise downloading the last CU that has no regressions.

In the last months I have been actively configuring and testing hybrid capabilities in SharePoint 2013. If you have any questions during configuring hybrid features in SharePoint, make sure to contact me on Twitter for the fastest response! I’ll be glad to help with any question.

Fixing apps after configuring SharePoint Hybrid

Update: Microsoft included the fix in the hybrid picker experience. This means you no longer have to perform the steps outlined in this blog post.
You can find the updated article by Microsoft here: https://blogs.technet.microsoft.com/beyondsharepoint/2016/09/15/considerations-when-deploying-sharepoint-office365-hybrid-workloads-in-a-farm-utilizing-provider-hosted-add-ins-or-workflow-manager/

For hybrid search (outbound/inbound query federation or Cloud Hybrid Search Service Application) a manual approach is needed to remediate this scenario.
A KB article was released, which can be found here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error


Summary: This article provides a solution to broken provider-hosted add-ins after configuring SharePoint hybrid features. For a full list of hybrid features, see the following article: https://www.sharepointrelated.com/2016/10/04/hybrid-features-sharepoint-2013-and-2016

The following hybrid features will break your server-to-server trusts that were already set up before configuring hybrid for SharePoint 2013 or SharePoint 2016:

This post will describe why this happens and how we can fix this.

In order to establish a server-to-server trust between your on-premises SharePoint environment and Office 365, Microsoft relies on the SPAuthenticationRealm. More information can be found here: https://technet.microsoft.com/en-us/library/jj219756.aspx.

This article has a “Caution” section, warning that any access tokens created for a specific realm, won’t work after changing the SPAuthenticationRealm.
SharePoint hybrid

To fix this, I wrote a script that gives you 2 options:

Undo Fix
Reverse the changes made by configuring Hybrid. It will change the SPAuthenticationRealm back to the old value. All SharePoint hybrid features stop working. All your provider-hosted add-ins will work again. This option will try to change your SPTrustedSecurityTokenIssuers so that it uses the new SPAuthenticationRealm set by configuring hybrid.

CautionThere are some notes that I described later in this post, make sure to read them.

Running the script will result in something like this:
SharePoint hybrid
Running the Fix-Hybrid.ps1 script

You can download the script here:
Fix-Hybrid

Notes
If you choose to fix your SPTrustedSecurityTokenIssuers, you will need to do some additional work to have everything work again.

  • Regrant app permissions

App permissions rely on the SPAuthenticationRealm.
This means that any App permissions that you set, will be gone after updating your SPTrustedSecurityTokenIssuers.
You will have to register the apps again and assign the permissions to the app.
The following script can do this for you (the current script is app-instance based, this means you have to run it for every app instance.
Also, make sure to change the variables in the script before running it.
Set-SPAppPermissions

  • Workflow Manager

Workflow Manager also relies on the SPAuthenticationRealm. Thanks to Ruben de Boer for proposing the solution.
After running the Fix-Onboarding.ps1 script, make sure to remove the existing Workflow Service Application Proxy.
Then run the Register-SPWorkflowService cmdlet again. Make sure to use the same scope that you used before. I recommend using the -Force parameter.

I hope this helps anyone! Do not hesitate to contact me if you have any trouble using the script of have any questions.