Hybrid features in SharePoint 2013 and 2016

Summary: This post provides an overview of all hybrid SharePoint features that were released by Microsoft for SharePoint 2013 and SharePoint 2016.

During Ignite 2016 in Atlanta, Microsoft released some really cool hybrid features, that I would like to share some information about. The really cool thing about this is that they are not only available for SharePoint 2016, but Microsoft actually made most of them available in SharePoint 2013. The following table will show the availability per feature, so you know which one is available to your environment.

For more information on any specific hybrid feature, click the feature in the table below.

(1) Breaks ALL existing server-to-server trusts. Provider-hosted add-ins are the most commonly found that use server-to-server trust. Make sure to read this blog post for a solution.
(2) There have been major improvements in the CU’s after the initial August 2015 CU for Cloud Hybrid Search. I advise downloading the last CU that has no regressions.

In the last months I have been actively configuring and testing hybrid capabilities in SharePoint 2013. If you have any questions during configuring hybrid features in SharePoint, make sure to contact me on Twitter for the fastest response! I’ll be glad to help with any question.

Fixing apps after configuring SharePoint Hybrid

Update: Microsoft included the fix in the hybrid picker experience. This means you no longer have to perform the steps outlined in this blog post.
You can find the updated article by Microsoft here: https://blogs.technet.microsoft.com/beyondsharepoint/2016/09/15/considerations-when-deploying-sharepoint-office365-hybrid-workloads-in-a-farm-utilizing-provider-hosted-add-ins-or-workflow-manager/

For hybrid search (outbound/inbound query federation or Cloud Hybrid Search Service Application) a manual approach is needed to remediate this scenario.
A KB article was released, which can be found here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error

Summary: This article provides a solution to broken provider-hosted add-ins after configuring SharePoint hybrid features. For a full list of hybrid features, see the following article: https://www.sharepointrelated.com/2016/10/04/hybrid-features-sharepoint-2013-and-2016

The following hybrid features will break your server-to-server trusts that were already set up before configuring hybrid for SharePoint 2013 or SharePoint 2016:

This post will describe why this happens and how we can fix this.

In order to establish a server-to-server trust between your on-premises SharePoint environment and Office 365, Microsoft relies on the SPAuthenticationRealm. More information can be found here: https://technet.microsoft.com/en-us/library/jj219756.aspx.

This article has a “Caution” section, warning that any access tokens created for a specific realm, won’t work after changing the SPAuthenticationRealm.
SharePoint hybrid

To fix this, I wrote a script that gives you 2 options:

Undo Fix
Reverse the changes made by configuring Hybrid. It will change the SPAuthenticationRealm back to the old value. All SharePoint hybrid features stop working. All your provider-hosted add-ins will work again. This option will try to change your SPTrustedSecurityTokenIssuers so that it uses the new SPAuthenticationRealm set by configuring hybrid.

CautionThere are some notes that I described later in this post, make sure to read them.

Running the script will result in something like this:
SharePoint hybrid
Running the Fix-Hybrid.ps1 script

You can download the script here:

If you choose to fix your SPTrustedSecurityTokenIssuers, you will need to do some additional work to have everything work again.

  • Regrant app permissions

App permissions rely on the SPAuthenticationRealm.
This means that any App permissions that you set, will be gone after updating your SPTrustedSecurityTokenIssuers.
You will have to register the apps again and assign the permissions to the app.
The following script can do this for you (the current script is app-instance based, this means you have to run it for every app instance.
Also, make sure to change the variables in the script before running it.

  • Workflow Manager

Workflow Manager also relies on the SPAuthenticationRealm. Thanks to Ruben de Boer for proposing the solution.
After running the Fix-Onboarding.ps1 script, make sure to remove the existing Workflow Service Application Proxy.
Then run the Register-SPWorkflowService cmdlet again. Make sure to use the same scope that you used before. I recommend using the -Force parameter.

I hope this helps anyone! Do not hesitate to contact me if you have any trouble using the script of have any questions.

People picker settings for all Web Applications one-liner

Summary: This PowerShell one-liner will show the people picker settings for each Web Application, including Central Administration.

If you want to know the settings for the people picker in your SharePoint farm, you can run the following line of PowerShell.
It will retrieve all Web Applications and show the people picker settings for each one.

Get-SPWebApplication -IncludeCentralAdministration | %{Write-Host $_.url; $_.peoplepickersettings | select * | fl }

Running the one-liner will result in something like this:

People picker settings

Cloud hybrid search considerations

Summary: This blog post describes some limitations that you need to consider before implementing cloud hybrid search in your organization.

Do not hesitate to contact me on twitter or LinkedIn to see if there are any changes that are not reflected in this blog post.
For a full overview of how Cloud Hybrid Search works, read my blog post: Everything you need to know about Cloud Hybrid Search

1. Provider-hosted apps
All provider-hosted add-ins will break when running the onboarding script *
One part of the onboarding script will change the SPAuthenticationRealm for your entire SharePoint farm.
As all your SPTrustedSecurityTokenIssuers rely on this SPAuthenticationRealm, they stop working after running the onboarding script.

Microsoft released a new hybrid picker that includes a fix for this issue. If you are configuring Cloud Hybrid Search only using the onboarding script, you have to do some additional work to get your apps to work again. Microsoft released a KB article with the fix here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error

2. Search customizations
The Cloud Search Service Application shares a similar architecture as the native SharePoint Search Service Application.
However, customization is limited because the search experience is derived from Office 365.

Below is a table that shows the current limitations in search customizations when using Cloud Hybrid Search.
Hybrid Cloud Search customizations

3. Searching on Non-default Zone URL’s
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Crawling Happens on-premises and the Default Zone url which you are crawling is fed to the Index on the Cloud. Since the Query is being served from Components in the Cloud which have no knowledge of the alternate URL’s available for On-prem Web-application, no URL Translation happens.
In Fact there is No concept of AAM’s in cloud Space.

The Only way to work around these limitations is to ensure that:

  1. The Public Default Zone URL should be a FQDN URL or The END URL which you want to Show up to the users in search results.
  2. The Default Zone URL which is being crawled should have Windows Claims Auth so that ACL information sent along with crawled Content is recognized by Search Components correctly.
  3. The User Needs to Perform Search from a URL while logged in windows Identity , so that it can be resolved effectively to validate Claims & Security Trimming to work effectively.

Note : Please ensure that Public URL which you want end users to see , should be accessible to Crawler on premise to be able to crawl the site successfully.

4. Windows Claims Only
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Cloud SSA or To be Specific the Search Components in SharePoint online at present do not understand any other ACL type other than Windows Claims , This ACL information ,about the Content which is fed along the index is used for Security Trimming the results . SAML Identity Providers are not supported with Cloud SSA . So basically the On-prem Site you are crawling using the Cloud SSA should be using Windows Auth only .

if you search on an extended URL of Web-applications which is ADFS / LDAP or Some other type of Auth other than Windows Claims , No search results will be returned as the logged in user’s Identity cannot be resolved to ACL mapping we have on an Index Item.

5. Index item limits and pricing
Vlad Catrinescu blogged about this on his blog.
For every 1TB of pooled storage in SharePoint Online, we are allowed to put one million index items from our On-Premises SharePoint Farm.

You can check your current searchable items from the Search Service Application in Central Administration
Cloud hybrid search considerations

In this example, we would need 13TB of pooled storage in SharePoint Online. This might mean that you have to reconfigure your content sources.

6. Security and compliancy
As your index is stored in Office 365, what does this mean compliancy wise?
This is the answer from Microsoft:

The content that is passed from on-premises to the azure cloud search connector (SCS) consists of crawled properties, keywords, ACLs, tenant info and some other metadata about the item. This is encrypted on-premises using a key supplied by the SCS and transmitted to the endpoint in Azure. Once there it is stored in an encrypted blob store and queued for processing. We retain the encrypted package in the blob store for use should we need to issue a content recrawl. The encrypted object is not the document though, it is just a parsed and filtered version that makes sense to the search engine.

In my opinion it would be wise to consult with your legal department before setting up Cloud Hybrid search to make sure it is okay to store content in the cloud.
You can always modify the content sources to exclude highly classified documents.

7. Licensing and Office 365 accounts
Every user that wants to use the new Cloud Hybrid Search functionality will need an active Office 365 license and an account synchronized from your on-premises Active Directory.
As items are indexed in Office 365, the access control entities are looked up in the cloud directory service.
User SIDs are mapped to PUIDs; Group SIDs are mapped to Object IDs; and and are mapped to .

8. Documentation
As the cloud hybrid search is still in preview, documentation is limited at best.
For example, if you are using a proxy for Internet access on your server, make sure to specify this in your machine.config.
I have created a more detailed blog post around this issue here: https://www.sharepointrelated.com/2015/12/11/cloud-hybrid-search-proxy-settings/

The documentation for Cloud Hybrid Search has been greatly improved. You can find all information you need here: https://technet.microsoft.com/en-us/library/dn720906.aspx

Everything you need to know about Cloud Hybrid Search

Summary: This article discusses the new cloud hybrid search service application. Use this article to configure cloud hybrid search in your organization and learn what you need to know.

If you need to decide whether to use or not use cloud hybrid search, please read my Cloud hybrid search considerations blog post before deploying cloud hybrid search: https://www.sharepointrelated.com/2016/03/02/cloud-hybrid-search-considerations/

What is cloud hybrid search
Before cloud hybrid search, Microsoft provided hybrid search scenarios between your on-premises SharePoint environment and SharePoint Online. These solutions were based on query federation. For instance, when you searched for a document in SharePoint Online in your on-premises environment, the query would be sent to the on-premises environment, and the results are returned back to the user in SharePoint Online. Microsoft released a script to automate this: http://blogs.msdn.com/b/spses/archive/2015/11/17/office-365-sharepoint-hybrid-configuration-wizard.aspx.

In September 2015, Microsoft released the new cloud hybrid search service application.

Instead of using query federation to surface results in your environment, it relies on indexing your on-premises content in Office 365. This takes away a lot of complexity setting it up, and makes it possible to mix results from SharePoint on-premises and Office 365 in a single result block. You can set up this new feature using SharePoint 2013 or SharePoint 2016.

Figure 1 shows a representation of the “old” hybrid search architecture and figure 2 shows the “new” hybrid search architecture.
Hybrid federated search architecture (old)
Figure 1: Hybrid federated search architecture

In this old scenario, the user enters a query in the on-premises search center. SharePoint sends the query to the on-premises query component and the SharePoint Online query component. Results cannot be interleaved out-of-the-box in this scenario, as there are separated indexes for SharePoint on-premises and SharePoint Online. However, there are several third-party solutions available that make it possible.
New cloud hybrid search architecture

Figure 2: New cloud hybrid search architecture

In this scenario, the cloud search service crawls the content sources on-premises and sends the parsed content to the SharePoint Online content processing component. After processing the content and doing ACL mapping – for security trimming purposes – the data is saved in the SharePoint Online index. Because the index is saved online, it is now possible to interleave results in your search results and use the data in Delve.

If you want your on-premises SharePoint to show SharePoint Online results, you have to configure a remote result source in your SharePoint on-premises server. This makes sure your on-premises farms uses the SharePoint Online index. This is described later in this post.

Why cloud hybrid search
Not all companies are ready to make the move to the cloud for all their workloads. In order to help customers make the move for specific workloads, Microsoft now provides an easy way to gradually move to the cloud while maintaining a great search experience for end users.

By using the new Cloud hybrid search solution, users are able to search content from the following sources from within SharePoint Online:

  • SharePoint 2007/2010/2013/2016
  • File shares
  • BCS

The index for all these sources is indexed in Office 365, which gives Microsoft the ability to interleave results across sources based on relevancy, use the Office 365 ranking model and even include all of this in Delve!

Organizations can also scale down search infrastructure as content processing and analytics are handled by Office 365.

Prerequisites for cloud hybrid search
In order to use the new Hybrid search functionality, make sure you have installed the following prerequisites for your environment.

SharePoint on-premises

  • If you use SharePoint 2013, make sure you installed the August 2015 CU or later. I recommend the latest CU without known regressions, as there have been improvements to the hybrid search.
  • Public preview of SharePoint 2016 IT Preview.

Office 365
All users that want to benefit from these new hybrid capabilities need to have an active SharePoint Online license in Office 365.

Account synchronization
Accounts need to be synchronized to Office 365 in order to have a single identity for users.

The tools below are supported to perform directory synchronization:

If you do not have any of the above synchronization tools deployed in your environment, I recommend using AADConnect. AADConnect can configure the full Single-sign on experience for you just by specifying server names on which it should deploy ADFS.

Software needed during configuration of hybrid search
On the SharePoint server where you are performing the configuration of hybrid search, you will need to install the following prerequisite software in this specific order.

Onboarding script
The onboarding script will create the trust between your on-premises SharePoint environment and Office 365. You can download the script along with documentation from the Microsoft Download Center.

The scripts can be found here: https://www.microsoft.com/en-us/download/details.aspx?id=51490.

Make sure you are using the latest version prior to execution.

Creating the cloud search service application
After you have installed all the prerequisites, it’s now time to create the cloud search service application, which is pretty straightforward. You could use any script that you prefer; just add the parameter, “CloudIndex $true” to the New-SPEnterpriseSearchServiceApplication cmdlet.

On the server that is running SharePoint Server 2013 or SharePoint Server 2016 Preview, copy the sample script below and save it as CreateCloudSSA.ps1 and run it. This will create a single-server Search Service Application topology. If you want a highly available search service infrastructure, you have to manually adjust the script to your needs.

This script was taken from: http://blogs.msdn.com/b/spses/archive/2015/09/15/cloud-hybrid-search-service-application.aspx

## Gather mandatory parameters ##    
## Note: SearchServiceAccount needs to already exist in Windows Active Directory as per Technet Guidelines https://technet.microsoft.com/library/gg502597.aspx ##    

[Parameter(Mandatory=$true)][string] $SearchServerName,      
[Parameter(Mandatory=$true)][string] $SearchServiceAccount,     
[Parameter(Mandatory=$true)][string] $SearchServiceAppName,     
[Parameter(Mandatory=$true)][string] $DatabaseServerName     

Add-PSSnapin Microsoft.SharePoint.Powershell -ea 0     

## Validate if the supplied account exists in Active Directory and whether supplied as domainusername    
    if ($SearchServiceAccount.Contains("")) # if True then domainusername was used     
    $Account = $SearchServiceAccount.Split("")     
    $Account = $Account[1]     
    else # no domain was specified at account entry     
    $Account = $SearchServiceAccount     
    $domainRoot = [ADSI]''     
    $dirSearcher = New-Object System.DirectoryServices.DirectorySearcher($domainRoot)     
    $dirSearcher.filter = "(&(objectClass=user)(sAMAccountName=$Account))"     
    $results = $dirSearcher.findall()     
if ($results.Count -gt 0) # Test for user not found     
    Write-Output "Active Directory account $Account exists. Proceeding with configuration"     
## Validate whether the supplied SearchServiceAccount is a managed account. If not make it one.    
if(Get-SPManagedAccount | ?{$_.username -eq $SearchServiceAccount})      
        Write-Output "Managed account $SearchServiceAccount already exists!"     
        Write-Output "Managed account does not exists - creating it"     
$ManagedCred = Get-Credential -Message "Please provide the password for $SearchServiceAccount" -UserName $SearchServiceAccount     
        New-SPManagedAccount -Credential $ManagedCred     
         Write-Output "Unable to create managed account for $SearchServiceAccount. Please validate user and domain details"     
Write-Output "Creating Application Pool"      
$appPool = New-SPServiceApplicationPool -name $appPoolName -account $SearchServiceAccount     
Write-Output "Starting Search Service Instance"     
Start-SPEnterpriseSearchServiceInstance $SearchServerName     
Write-Output "Creating Cloud Search Service Application"

$searchApp = New-SPEnterpriseSearchServiceApplication -Name $SearchServiceAppName -ApplicationPool $appPool -DatabaseServer $DatabaseServerName -CloudIndex $true     
Write-Output "Configuring Admin Component"     
$searchInstance = Get-SPEnterpriseSearchServiceInstance $SearchServerName     
$searchApp | get-SPEnterpriseSearchAdministrationComponent | set-SPEnterpriseSearchAdministrationComponent -SearchServiceInstance $searchInstance     
$admin = ($searchApp | get-SPEnterpriseSearchAdministrationComponent)     
Write-Output "Waiting for the admin component to be initialized"     
do {Write-Output .;Start-Sleep 10;} while ((-not $admin.Initialized) -and ($timeoutTime -ge (Get-Date)))     
if (-not $admin.Initialized) { throw 'Admin Component could not be initialized'}     

Write-Output "Inspecting Cloud Search Service Application"

$searchApp = Get-SPEnterpriseSearchServiceApplication $SearchServiceAppName     

Write-Output "Setting IsHybrid Property to 1"     

#Output some key properties of the Search Service Application    
Write-Host "Search Service Properties"      
Write-Host "Hybrid Cloud SSA Name    : " $searchapp.Name     
Write-Host "Hybrid Cloud SSA Status  : " $searchapp.Status     
Write-Host "Cloud Index Enabled      : " $searchApp.CloudIndex     
Write-Output "Configuring Search Topology"     

$searchApp = Get-SPEnterpriseSearchServiceApplication $SearchServiceAppName     
$topology = $searchApp.ActiveTopology.Clone()     

$oldComponents = @($topology.GetComponents())
if (@($oldComponents | ? { $_.GetType().Name -eq "AdminComponent" }).Length -eq 0)
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.AdminComponent $SearchServerName))     
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.CrawlComponent $SearchServerName))     
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.ContentProcessingComponent $SearchServerName))     
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.AnalyticsProcessingComponent $SearchServerName))     
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.QueryProcessingComponent $SearchServerName))     
$topology.AddComponent((New-Object Microsoft.Office.Server.Search.Administration.Topology.IndexComponent $SearchServerName,0))     

$oldComponents  | ? { $_.GetType().Name -ne "AdminComponent" } | foreach { $topology.RemoveComponent($_) }
Write-Output "Activating topology"     

do {Write-Output .;Start-Sleep 10;} while (($searchApp.GetTopology($topology.TopologyId).State -ne "Active") -and ($timeoutTime -ge (Get-Date)))     

if ($searchApp.GetTopology($topology.TopologyId).State -ne "Active")  { throw 'Could not activate the search topology'}     
Write-Output "Creating Proxy"     
$searchAppProxy = new-spenterprisesearchserviceapplicationproxy -name ($SearchServiceAppName+"_proxy") -SearchApplication $searchApp     
Write-Output " Cloud hybrid search service application provisioning completed successfully."     
    else # The Account Must Exist so we can proceed with the script     
    Write-Output "Account supplied for Search Service does not exist in Active Directory."     
    Write-Output "Script is quitting. Please create the account and run again."     
} # End Else

The output should look similar to figure 3.Create-SSA.ps1 output, creating a cloud search service application
Figure 3: Create-SSA.ps1 output, creating a cloud search service application

Proxy configuration for hybrid cloud search
If your organization uses a proxy to allow Internet access, you have to configure this proxy for hybrid cloud search as well. For a more in-depth article, please look at https://www.sharepointrelated.com/2015/12/11/cloud-hybrid-search-proxy-settings/, but for now we can just add the proxy settings to the machine config: “C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config”

Here’s an example of what this would look like:

    <proxy usesystemdefault="false" proxyaddress="" bypassonlocal="true" />

Place this anywhere between your and tag. To make it easier to find when you need it, you could place it right before the tag.

Onboarding the cloud search service application
After successfully installing the prerequisites and configuring the cloud search service application, it is time to start the onboarding process. The onboarding process will create a trust between your SharePoint on-premises and Office 365 environment. This will allow SharePoint to move the index to Office 365 for further processing.

Run the onboarding script:

.Onboard-CloudHybridSearch.ps1 -PortalUrl "https://yourtenant.sharepoint.com" -CloudSSAId "<Cloud Search Service Application name>"

Enter your Global Administrator credentials when prompted.

Figure 4 shows what your output should resemble.
Cloud hybrid search onboarding script

Figure 4: Running the cloud hybrid search onboarding script on the server that runs SharePoint Server 2013

The script name and the parameters have changed a bit since I ran the script. Make sure you check to see what the correct parameters are when you run the script.

Configure content source in cloud search service application
You can configure the content source in your new cloud search service application as you would in any other on-premises SharePoint environment. As Figure 5 shows, you configure the content source of the cloud search service application in Search Administration.
Configure the content source for the cloud search service application
Figure 5: In SharePoint Search Administration you can edit (configure) the content source for the cloud search service application.

Enter the start addresses that you would like to crawl and start a full crawl for the content source. After the crawl is done, check the crawl log for the specific content source to see if all went well.
Check the cloud search service application crawl logs for errors or warningsFigure 6: Check the crawl logs for any errors or warning

If you see 1 Top Lever Error with the following error message:

AzureServiceProxy caught Exception: *** Microsoft.Office.Server.Search.AzureSearchService.AzureException: AzurePlugin was not able to get Tenant Info from configuration server    
 at Microsoft.Office.Server.Search.AzureSearchService.AzureServiceProxy.GetAzureTenantInfo(String portalURL, String realm, String&amp;amp; returnPropertyValue, String propertyName)    
 at Microsoft.Office.Server.Search.AzureSearchService.AzureServiceProxy.SubmitDocuments(String azureServiceLocation, String authRealm, String SPOServiceTenantID, String SearchContentService_ContentFarmId, String portalURL, String testId, String encryptionCert, Boolean allowUnencryptedSubmit, sSubmitDocument[] documents, sDocumentResult[]&amp;amp; results, sAzureRequestInfo&amp;amp; RequestInfo) ***

Make sure to check your proxy configuration (https://www.sharepointrelated.com/2015/12/11/cloud-hybrid-search-proxy-settings/).

Configure your on-premises farm to use the SharePoint Online index
In order to use the SharePoint Online index in your on-premises farm, you have to configure a remote result source. This can be done by following this article: https://technet.microsoft.com/en-us/library/mt668455.aspx.

Verifying results: perform a query in SharePoint Online and SharePoint on-premises
In Office 365, search for a document and it will return results for both SharePoint Online and SharePoint on-premises if cloud hybrid search is configured correctly.

Figure 7 shows example results from a search that includes the following sources:

  • SharePoint Online
  • SharePoint on-premises
  • File shares

Searching content in Office 365 returning results from both on-premises and SharePoint Online
Figure 7: Searching content in Office 365 returning results from both on-premises and SharePoint Online

If you want to return results only from your on-premises site, you can use the “isexternalcontent:1” property.

As figure 8 shows, this returns only on-premises results.
Search results only from on-premises
Figure 8: Using the isexternalcontent:1 property shows search results only from on-premises.

The new cloud hybrid search solution is a great way to provide a great end-user search experience, wherever the information is stored. In just a few hours, your users will be able to benefit from this new feature. If you have any problems while configuring the new cloud hybrid search capabilities, you can reach Microsoft directly by using the Cloud Search Service Application Preview forum: https://social.technet.microsoft.com/Forums/office/en-US/home?forum=CloudSSA.

Cloud Hybrid Search proxy settings

The new Cloud Hybrid Search has been released in preview in September 2015. Seeing the demos I was really excited to try this. One of our customers was already in need of this technology, so we proposed to configure Cloud Hybrid Search on their test environment.

The onboarding process seemed pretty straightforward and we used http://blogs.msdn.com/b/spses/archive/2015/09/15/cloud-hybrid-search-service-application.aspx as a reference. We read the documentation, grabbed the on-boarding script, installed the prerequisites and got started.

After running the onboarding script the first time, some errors were thrown, but after running the script for the 2nd time, everything seemed to work out just perfectly. All steps were executed, no red parts in the output of the script.Cloud Hybrid Search proxy settingsThe next step would be to start a Full crawl. We created a specific content source in the on-premises environment. This content source contains only 1 site collection with a few documents inside, just to check the functionality. Unfortunately, the crawl failed. After inspecting the log, we saw the following:

AzureServiceProxy caught Exception: *** Microsoft.Office.Server.Search.AzureSearchService.AzureException: AzurePlugin was not able to get Tenant Info from configuration server
at Microsoft.Office.Server.Search.AzureSearchService.AzureServiceProxy.GetAzureTenantInfo(String portalURL, String realm, String&amp;amp;amp; returnPropertyValue, String propertyName)
at Microsoft.Office.Server.Search.AzureSearchService.AzureServiceProxy.SubmitDocuments(String azureServiceLocation, String authRealm, String SPOServiceTenantID, String SearchContentService_ContentFarmId, String portalURL, String testId, String encryptionCert, Boolean allowUnencryptedSubmit, sSubmitDocument[] documents, sDocumentResult[]& results, sAzureRequestInfo&amp;amp;amp; RequestInfo) ***

For some reason, SharePoint wasn’t able to submit index data for documents to Azure for processing. We created a thread on the TechNet forum for Cloud Search Service Application Preview.

As we were aware that a proxy was being used, we started checking the proxy configuration. There are a lot of places where you can configure a proxy if you are looking for them:

  • Web.config
  • Netsh winhttp set proxy
  • Browser settings for service account
  • Search Service Application proxy

After trying all these, we still couldn’t get it to work. After using Wireshark, we found out that the upload was still not using our proxy server. Instead, it tried uploading directly to Azure.

The solution
After discussing this with my colleague @vanHooijdonk, he reminded me that there is another place where you can configure proxy settings. The machine config: “C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config”

    <proxy usesystemdefault="false" proxyaddress="" bypassonlocal="true" />

After applying the proxy there, everything started to work! Place the proxy settings anywhere between the <configuration> and </configuration> tag. Personally, I placed it right before the end, so I can find it easier next time.

Please read my full blog post on how to configure Cloud Hybrid Search here: https://www.sharepointrelated.com/2015/12/21/everything-you-need-to-know-about-cloud-hybrid-search/

Thanks to @johankroese and @vanHooijdonk for helping with troubleshooting this issue!

Download content from a site collection

I’ve been working on a script that will allow you to download all files that are stored in SharePoint in a given site collection.

If the path does not exist, the script will prompt you to create it for you. Before the script runs, it also checks if the site collection exists.

Run the script like this:

.\Get-SPContent.ps1 -SiteCollection "<SiteCollectionURL>" -Destination "<Path>"

Download content

The console shows which libraries were exported to your file system.

—– * Advanced * —–

If you have specific requirements as to which (type of) libraries you want to export, you can change the following line to fit your requirements:

$lists = $web.lists | ?{$_.itemcount -ge &quot;1&quot; -And $_.Hidden -eq $false -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Excludes all hidden libraries and empty libraries

Below is the code you can save as Get-SPContent.ps1

[ValidateScript({asnp *sh* -EA SilentlyContinue;if (Get-SPSite $_){$true}else{Throw &quot;Site collection $_ does not exist&quot;}})]
if (Test-Path $_)
$d = $_
$title = &quot;Create Folder?&quot;;
$message = &quot;$_ doesn't exist, do you want the script to create it?&quot;;
$yes = New-Object System.Management.Automation.Host.ChoiceDescription &quot;&amp;Yes&quot;, &quot;Creates directory $_&quot;;
$no = New-Object System.Management.Automation.Host.ChoiceDescription &quot;&amp;No&quot;, &quot;Exits script&quot;;
$options = [System.Management.Automation.Host.ChoiceDescription[]]($yes,$no);
$result = $host.ui.PromptForChoice($title,$message,$options,1);
0 {New-Item $d -Type Directory;$true}
1 {Throw &quot;Please create the folder before running the script again. `nExiting script&quot;}

Asnp *sh* -EA SilentlyContinue

Start-SPAssignment -Global | Out-Null

function Get-SPWebs($SiteCollection){
$SiteCollection = Get-SPSite $SiteCollection
$webs = @()
$SiteCollection.allwebs | %{$webs += $_.url}
return $webs

function Get-SPFolders($webs)
foreach($web in $webs)
$web = Get-SPWeb $web
Write-Host &quot;`n$($web.url)&quot;

$lists = $web.lists | ?{$_.itemcount -ge &quot;1&quot; -And $_.Hidden -eq $false -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Excludes all hidden libraries and empty libraries
#$lists = $web.lists | ?{$_.title -eq &quot;Documents&quot; -and $_.itemcount -ge &quot;1&quot; -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Change any identifier here
foreach($list in $lists)
Write-Host &quot;- $($list.RootFolder.url)&quot;

#Download files in root folder
$rootfolder = $web.GetFolder($list.RootFolder.Url)

#Download files in subfolders
foreach($folder in $list.folders)
$folder = $web.GetFolder($folder.url)



function Download-SPContent($folder)
foreach($file in $folder.Files)
$binary = $file.OpenBinary()
$stream = New-Object System.IO.FileStream($destination + &quot;/&quot; + $file.Name), Create
$writer = New-Object System.IO.BinaryWriter($stream)

$webs = Get-SPWebs -SiteCollection $Sitecollection
Get-SPFolders -Webs $webs

Stop-SPAssignment -Global

Restore deleted site collections SharePoint 2013

In SharePoint 2013 it is possible to restore a deleted site collection. For more information, read this article: http://technet.microsoft.com/en-us/library/hh272537.aspx

You can use the Restore-SPDeletedSite cmdlet to restore a site collection.

However, if you removed the site collection using the Remove-SPSite cmdlet using PowerShell, the site collection will not be stored in a SPDeletedSite object.

This means you cannot restore a site collection that has been removed using PowerShell.