Hybrid taxonomy considerations

Summary: This post will describe all things to keep in mind when configuring Hybrid Taxonomy.

Microsoft released Hybrid Taxonomy in preview for SharePoint 2013 and SharePoint 2016 on-premises. This makes it possible to replicate terms between your on-premises SharePoint farm and SharePoint Online. For more information on hybrid taxonomy, see https://support.office.com/en-us/article/Configure-hybrid-SharePoint-taxonomy-Preview-dd287a75-09e0-403e-974e-4cc84a152815

1. Term limit

In SharePoint on-premises (2010, 2013 and 2016) the maximum number of items in a term store is 1.000.000. For more information, see: https://technet.microsoft.com/en-us/library/cc262787.aspx#termstore

In SharePoint Online, the maximum numbers in your term store is only 200.000.

This means that you can only have a maximum of 200.000 terms in your on-premises Managed Metadata Service Application if you are planning to use hybrid taxonomy. Take this into consideration when you configure hybrid taxonomy.

2. Preview

Hybrid taxonomy is currently still in preview. Keep in mind that things might change along the way.

If I find any more considerations I will update this post.

Hybrid features in SharePoint 2013 and 2016

Summary: This post provides an overview of all hybrid SharePoint features that were released by Microsoft for SharePoint 2013 and SharePoint 2016.

During Ignite 2016 in Atlanta, Microsoft released some really cool hybrid features, that I would like to share some information about. The really cool thing about this is that they are not only available for SharePoint 2016, but Microsoft actually made most of them available in SharePoint 2013. The following table will show the availability per feature, so you know which one is available to your environment.

For more information on any specific hybrid feature, click the feature in the table below.

(1) Breaks ALL existing server-to-server trusts. Provider-hosted add-ins are the most commonly found that use server-to-server trust. Make sure to read this blog post for a solution.
(2) There have been major improvements in the CU’s after the initial August 2015 CU for Cloud Hybrid Search. I advise downloading the last CU that has no regressions.

In the last months I have been actively configuring and testing hybrid capabilities in SharePoint 2013. If you have any questions during configuring hybrid features in SharePoint, make sure to contact me on Twitter for the fastest response! I’ll be glad to help with any question.

Cloud hybrid search considerations

Summary: This blog post describes some limitations that you need to consider before implementing cloud hybrid search in your organization.

Do not hesitate to contact me on twitter or LinkedIn to see if there are any changes that are not reflected in this blog post.
For a full overview of how Cloud Hybrid Search works, read my blog post: Everything you need to know about Cloud Hybrid Search

1. Provider-hosted apps
All provider-hosted add-ins will break when running the onboarding script *
One part of the onboarding script will change the SPAuthenticationRealm for your entire SharePoint farm.
As all your SPTrustedSecurityTokenIssuers rely on this SPAuthenticationRealm, they stop working after running the onboarding script.

Microsoft released a new hybrid picker that includes a fix for this issue. If you are configuring Cloud Hybrid Search only using the onboarding script, you have to do some additional work to get your apps to work again. Microsoft released a KB article with the fix here: https://support.microsoft.com/en-us/help/4010011/provider-hosted-add-ins-stop-working-and-http-401-error

2. Search customizations
The Cloud Search Service Application shares a similar architecture as the native SharePoint Search Service Application.
However, customization is limited because the search experience is derived from Office 365.

Below is a table that shows the current limitations in search customizations when using Cloud Hybrid Search.
Hybrid Cloud Search customizations

3. Searching on Non-default Zone URL’s
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Crawling Happens on-premises and the Default Zone url which you are crawling is fed to the Index on the Cloud. Since the Query is being served from Components in the Cloud which have no knowledge of the alternate URL’s available for On-prem Web-application, no URL Translation happens.
In Fact there is No concept of AAM’s in cloud Space.

The Only way to work around these limitations is to ensure that:

  1. The Public Default Zone URL should be a FQDN URL or The END URL which you want to Show up to the users in search results.
  2. The Default Zone URL which is being crawled should have Windows Claims Auth so that ACL information sent along with crawled Content is recognized by Search Components correctly.
  3. The User Needs to Perform Search from a URL while logged in windows Identity , so that it can be resolved effectively to validate Claims & Security Trimming to work effectively.

Note : Please ensure that Public URL which you want end users to see , should be accessible to Crawler on premise to be able to crawl the site successfully.

4. Windows Claims Only
From: https://blogs.msdn.microsoft.com/spses/2016/07/19/sharepoint-2016-hybrid-search-stuff-you-should-know-about-cloud-ssa/

Cloud SSA or To be Specific the Search Components in SharePoint online at present do not understand any other ACL type other than Windows Claims , This ACL information ,about the Content which is fed along the index is used for Security Trimming the results . SAML Identity Providers are not supported with Cloud SSA . So basically the On-prem Site you are crawling using the Cloud SSA should be using Windows Auth only .

if you search on an extended URL of Web-applications which is ADFS / LDAP or Some other type of Auth other than Windows Claims , No search results will be returned as the logged in user’s Identity cannot be resolved to ACL mapping we have on an Index Item.

5. Index item limits and pricing
Vlad Catrinescu blogged about this on his blog.
For every 1TB of pooled storage in SharePoint Online, we are allowed to put one million index items from our On-Premises SharePoint Farm.

You can check your current searchable items from the Search Service Application in Central Administration
Cloud hybrid search considerations

In this example, we would need 13TB of pooled storage in SharePoint Online. This might mean that you have to reconfigure your content sources.

6. Security and compliancy
As your index is stored in Office 365, what does this mean compliancy wise?
This is the answer from Microsoft:

The content that is passed from on-premises to the azure cloud search connector (SCS) consists of crawled properties, keywords, ACLs, tenant info and some other metadata about the item. This is encrypted on-premises using a key supplied by the SCS and transmitted to the endpoint in Azure. Once there it is stored in an encrypted blob store and queued for processing. We retain the encrypted package in the blob store for use should we need to issue a content recrawl. The encrypted object is not the document though, it is just a parsed and filtered version that makes sense to the search engine.

In my opinion it would be wise to consult with your legal department before setting up Cloud Hybrid search to make sure it is okay to store content in the cloud.
You can always modify the content sources to exclude highly classified documents.

7. Licensing and Office 365 accounts
Every user that wants to use the new Cloud Hybrid Search functionality will need an active Office 365 license and an account synchronized from your on-premises Active Directory.
Cloud-Hybrid-Search-Identity
As items are indexed in Office 365, the access control entities are looked up in the cloud directory service.
Hybrid-Search-FederatedAccount
User SIDs are mapped to PUIDs; Group SIDs are mapped to Object IDs; and and are mapped to .

8. Documentation
As the cloud hybrid search is still in preview, documentation is limited at best.
For example, if you are using a proxy for Internet access on your server, make sure to specify this in your machine.config.
I have created a more detailed blog post around this issue here: https://www.sharepointrelated.com/2015/12/11/cloud-hybrid-search-proxy-settings/

The documentation for Cloud Hybrid Search has been greatly improved. You can find all information you need here: https://technet.microsoft.com/en-us/library/dn720906.aspx

Change site collection URL in SharePoint 2013

Summary: This post describes the different ways to change the site collection URL in SharePoint 2013.

So, your manager/the business asks you to create a few site collections for some departments in your organization. You quickly spin up some site collections for them to use. After a few weeks, the business decides (of course) SharePoint is great, but the URL’s we chose weren’t all that great. Can you please change them?

There are alot of blog posts out there that describe the different possibilities in this scenario:
– Using backup and restore.
– A great article by Todd Klindt that tells you how to use the Copy-SPSite cmdlet to achieve the same goal but easier!

There is another (easier) way in some scenario’s in which you can change the site collection URL by just using 2 lines of PowerShell with only a second of waiting time!

The solution
Let’s get down to it.. I created this very nice Marketing site collection, using the url: http://sharepoint/sites/marketing. Now I would like to change this to http://sharepoint/sites/sales. Just use the following 2 lines of PowerShell code and you are done!

 $site = Get-SPSite http://sharepoint/sites/marketing
$site.Rename("http://sharepoint/sites/sales")

That is all.. try it out and see that your new URL is working, and the old URL is not working anymore!
*NOTE* Thanks to Jaymeson in the comments for pointing out an IISRESET is needed as well!

A little catch
There is a little catch to this. You can only use this to rename site collection URL’s that
– Use “Wildcard inclusion” Managed Paths.
– Are Host named site collections (In which case you could also use Set-SPSiteURL)

You can’t use it to change http://sharepoint/sites/marketing to http://sharepoint/marketing (Even if the Explicit inclusion managed path exists).

Hope this helps anyone out there!

Download content from a site collection

I’ve been working on a script that will allow you to download all files that are stored in SharePoint in a given site collection.

If the path does not exist, the script will prompt you to create it for you. Before the script runs, it also checks if the site collection exists.

Run the script like this:

.\Get-SPContent.ps1 -SiteCollection "<SiteCollectionURL>" -Destination "<Path>"

Download content

The console shows which libraries were exported to your file system.

—– * Advanced * —–

If you have specific requirements as to which (type of) libraries you want to export, you can change the following line to fit your requirements:

$lists = $web.lists | ?{$_.itemcount -ge &quot;1&quot; -And $_.Hidden -eq $false -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Excludes all hidden libraries and empty libraries

Below is the code you can save as Get-SPContent.ps1

param
(
[Parameter(Mandatory=$true)]
[ValidateScript({asnp *sh* -EA SilentlyContinue;if (Get-SPSite $_){$true}else{Throw &quot;Site collection $_ does not exist&quot;}})]
[string]$SiteCollection,
[Parameter(Mandatory=$true)]
[ValidateScript(
{
if (Test-Path $_)
{$true}
else{
$d = $_
$title = &quot;Create Folder?&quot;;
$message = &quot;$_ doesn't exist, do you want the script to create it?&quot;;
$yes = New-Object System.Management.Automation.Host.ChoiceDescription &quot;&amp;Yes&quot;, &quot;Creates directory $_&quot;;
$no = New-Object System.Management.Automation.Host.ChoiceDescription &quot;&amp;No&quot;, &quot;Exits script&quot;;
$options = [System.Management.Automation.Host.ChoiceDescription[]]($yes,$no);
$result = $host.ui.PromptForChoice($title,$message,$options,1);
switch($result)
{
0 {New-Item $d -Type Directory;$true}
1 {Throw &quot;Please create the folder before running the script again. `nExiting script&quot;}
}
}
})]
[string]$Destination
)

Asnp *sh* -EA SilentlyContinue

Start-SPAssignment -Global | Out-Null

function Get-SPWebs($SiteCollection){
$SiteCollection = Get-SPSite $SiteCollection
$webs = @()
$SiteCollection.allwebs | %{$webs += $_.url}
return $webs
}

function Get-SPFolders($webs)
{
foreach($web in $webs)
{
$web = Get-SPWeb $web
Write-Host &quot;`n$($web.url)&quot;

$lists = $web.lists | ?{$_.itemcount -ge &quot;1&quot; -And $_.Hidden -eq $false -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Excludes all hidden libraries and empty libraries
#$lists = $web.lists | ?{$_.title -eq &quot;Documents&quot; -and $_.itemcount -ge &quot;1&quot; -And $_.BaseType -eq &quot;DocumentLibrary&quot;} #Change any identifier here
foreach($list in $lists)
{
Write-Host &quot;- $($list.RootFolder.url)&quot;

#Download files in root folder
$rootfolder = $web.GetFolder($list.RootFolder.Url)
Download-SPContent($rootfolder)

#Download files in subfolders
foreach($folder in $list.folders)
{
$folder = $web.GetFolder($folder.url)
Download-SPContent($folder)

}

}
$web.dispose()
}
}

function Download-SPContent($folder)
{
foreach($file in $folder.Files)
{
$binary = $file.OpenBinary()
$stream = New-Object System.IO.FileStream($destination + &quot;/&quot; + $file.Name), Create
$writer = New-Object System.IO.BinaryWriter($stream)
$writer.write($binary)
$stream.Close()
$writer.Close()
}
}

$webs = Get-SPWebs -SiteCollection $Sitecollection
Get-SPFolders -Webs $webs

Stop-SPAssignment -Global

Add-PSSnapIn Microsoft.SharePoint.PowerShell shortcut

Are you tired of typing Add-PSSnapIn Microsoft.SharePoint.PowerShell every time you open your PowerShell console?
You can do this by Adding/Editing profile.ps1 (I don’t do this because I have too many machines where I should change this)

If you find you do not want to change the profile.ps1 on every server you are working on, you can type this:

asnp *sh*

This adds all SnapIns that contain *sh*. In most cases, this will only add the Microsoft.SharePoint.PowerShell SnapIn.
Credits go to Koen Zomers!

Happy PowerShelling!

 

Restore deleted site collections SharePoint 2013

In SharePoint 2013 it is possible to restore a deleted site collection. For more information, read this article: http://technet.microsoft.com/en-us/library/hh272537.aspx

You can use the Restore-SPDeletedSite cmdlet to restore a site collection.

However, if you removed the site collection using the Remove-SPSite cmdlet using PowerShell, the site collection will not be stored in a SPDeletedSite object.

This means you cannot restore a site collection that has been removed using PowerShell.

 

Add PDF mimetype for all Web Applications oneliner

By default, PDF files cannot be opened directly from SharePoint 2010/SharePoint 2013.

To add the PDF mimetype to all Web Applications (Instead of doing it seperately for each Web Application), you can use the following oneliner:

Get-SPWebApplication | %{$_.AllowedInlineDownloadedMimeTypes.Add("application/pdf");$_.Update()}

Get all subsites of a subsite using PowerShell

Getting a list of all subsites of a particular site (not a site collection) was a little more work than I expected, so here is how I did it.

Let’s say we have the following situation site structure:

SiteStructure

What if we want an overview of all sites under “Https://portal.sharepointrelated.com/Projects”?

My first thought was to use the “Webs” property of the SPWeb object. Unfortunately, this only shows the direct subsites for this site. This means that for “Https://portal.sharepointrelated.com/projects”, it only shows the Level 3 sites.

Solution

To work around this, I used the “AllWebs” property of the SPSite object and filtered the URL’s starting with “Https://portal.sharepointrelated.com/projects”.

Here is the code used: (Download .zip file)


param ( [Parameter(Mandatory=$true)][ValidateNotNullOrEmpty()] [String]$StartWeb, [Boolean]$IncludeStartWeb = $true )

Add-PSSnapin Microsoft.SharePoint.PowerShell -ErrorAction SilentlyContinue

$subsites = ((Get-SPWeb $StartWeb).Site).allwebs | ?{$_.url -like "$StartWeb*"}

foreach($subsite in $subsites) { Write-Host $subsite.url }

As you can see in the source code, I added 2 parameters to the script:

StartWeb: String. This is the starting URL. All subsites under this site will be showed in the result.

IncludeStartWeb: Boolean. When set to $false, the output will not include the URL provided in the StartWeb parameter.