CHUVASH.eu

CHunky Universe of Vigouros Astonishing SHarepoint :)

Export SharePoint List Data To Xml through PowerShell

Today my colleague Johannes Milling wrote an awesome post:

Export SharePoint List Data To XML Directly from the GUI

He uses the old and forgotten RPC-based web service owssvr.dll (that has existed since SharePoint 2003). This url pattern is used to get the xml-formatted list data for a specific ListView:

http://<site_url>/_vti_bin/owssvr.dll?Cmd=Display&List=<list guid>&View=<view guid>&Query=*&XMLDATA=TRUE

To automate this I have written a PowerShell function. All the details are in the comments:

function Export-OIPSPListDataAsXml {
<#
.Synopsis
   Exports list items as xml
.DESCRIPTION
   This function uses a built-in hidden service owssvr.dll in SharePoint
.PARAMETER Web
   SPWeb or a url for the SPWeb object where the list resides
.PARAMETER ListName
   List Name (List Title)
.PARAMETER ViewName
   Name of the List View to export the data
.EXAMPLE
   Export-OIPSPListView -Web http://myawesomesite -ListName "Teddy Bears" -ViewName "All Items"
.EXAMPLE
   Export-OIPSPListView -Web $webInstance -ListName "Top Links" -ViewName "All Items"
.Notes
    Name: Export-OIPSPListDataAsXml
    Author: Anatoly Mironov
    Last Edit: 2014-09-11
    Keywords: SPList, SPWeb, SPListView, Xml
.Link

http://chuvash.eu

#>
    [CmdLetBinding()]
    param(
	[Parameter(Mandatory=$true)]
    [Microsoft.SharePoint.PowerShell.SPWebPipeBind]$Web,
    [Parameter(Mandatory=$true)]
    [string]$ListName,
    [Parameter(Mandatory=$true)]
    [string]$ViewName)

    $spWeb = $Web.Read()
    $list = $spWeb.Lists[$ListName]
    if (!$list) { throw "List $ListName on web $($spWeb.Url) could not be found" }
    $view = $list.Views[$ViewName]
    if (!$view) { throw "View $ViewName on web $ListName could not be found" }
    $url = "{0}/_vti_bin/owssvr.dll?Cmd=Display&List={1}&View={2}&Query=*&XMLDATA=TRUE" `
            -f $spWeb.Url, $list.ID, $view.ID
    $wc = New-Object Net.WebClient
    $wc.UseDefaultCredentials = $true
    $wc.DownloadString($url)
}

Improving the web performance of an intranet

overloaded-sharepoint-in-browser

All the “small” app parts, web parts, delegate controls, user controls, and other “packages” that “must” be delivered to the users on every page load of the Start Page of your Intranet.

Recently we made an investment to improve the performance of our intranet. We made many changes in different layers: SQL, Network, Browser upgrade and code. Here I want to tell about what code changes we did to improve the web browser performance. Please leave feedback if you find it useful. You can also share your suggestions. We measured the performance before our code changes and after them. We had amazing results. Unfortunately I can not share any numbers, but we improved the Time to First Byte, time to load event firing in the browser, memory consumption in the clients and, perhaps, the most important, we improved the perceived performance of the Intranet, the way how users experience the speed and UI responsiveness. To do this I got many ideas of my project colleagues and branch colleagues. Here is the list of changes we’ve implemented:

- Execute code on demand

Consider this scenario: on a page users can click on a button to download a vcard. Aggregating user information is a costly operation that requires getting data from the User Profile Service, getting the profile image from SharePoint. Don’t ever do this operation on page load. Move the code to the “onclick” action. In other words, work when it is needed. It is like cooking a lot of food, when you are not hungry. It is nothing new, unfortunately there were a couple of the “eager code” places.

- Cache results, investigate what parts can be cached and how fresh data needs to be

All data on your page doesn’t need to be fetched on every page load. In our project We listed all the “parts” of the start page and other often visited pages and went to the business and asked them to evaluate how fresh data should be. Some parts should be as fresh as possible (no cache), whereas it would be tolerated that the information could be dirty (cache up to one week or more).

- Reference javascript and css files from one location

Do you have jQuery in your SharePoint Intranet. How many copies do you have? What we did in our Intranet was that we partially implemented the CDN concept. Even though we don’t distribute our resource files geographically, we have 1-to-1 relation between a file and its absolute URL. In the whole intranet, we have only one jQuery url and only one our intranet.core.js url. We did by creating a dedicated CDN site collection. This alone makes a big difference. To evolve the idea we could provision resources outside SharePoint to remove the authorization overhead. We could also distribute it geographically by having files closer to the end users.

- Unify your framework and your dependencies

In our projects we had a couple of SharePoint-hosted apps developed by different teams. We had different approaches and different framework that solved almost the same problem: AngularJS and KnockoutJS. Eventhough apps are independent pieces of software, they were used within the same page (as app parts). It was too much http traffic. We agreed on Developer Guidelines and chose one framework.

- Do not hide controls on the page with CSS, remove them

On the start page in our intranet we didn’t show the left navigation, but it was still rendered in code behind. Instead hiding it with CSS, we just removed it by an empty ContenPlaceHolder in our Start Page Layout:


<asp:Content ContentPlaceholderId="PlaceHolderLeftNavBar" runat="server"/>

- Optimize jQuery Selectors

We reviewed all the jQuery code and improved the selectors. Optimizing the selectors will improve the overall performance in the browser, especially in older browsers. The worst example is using text selectors, like this one:

jQuery('#NoteBoardContainer*:contains("There are no notes posted yet. You can use notes to comment on a page"):last');

It will sink your IE8 browser.

- Minify javascript and CSS files

Minifying resource files like javascript and css is not hard. My recommendation is to use Web Essentials plugin in Visual Studio. Alternatively you can use the SharePoint Assets Minifier.

- Use the weakest selectors in CSS and in LESS

In our project we are using LESS. With LESS it is easier to write  readable CSS code. But be aware of the output. Do not make the selectors too strong: Use the weakest CSS Selectors. The weakest selectors will make it easier to maintain the CSS and it will minimize the amount of KB the server needs to send to your users’ browsers.

- Ensure javascript and CSS files are cached

JavaScript and CSS files should be cached. You should also avoid 304 responses where the Server answers “Not Modified”, because this has an impact on the performance. Configure the Blob Cache and put your resources into the Style Library.

- Remove all app parts from SharePoint-hosted Apps from the start page

There can be exceptions, but we encountered that client web parts (app parts) from SharePoint-hosted apps had a huge impact on the performance. The combination of a couple of app parts on often visited pages (like the start page on the intranet) led to long page load times. These are the reasons why you should not have SharePoint Hosted App Parts on your start page:

  1. App Parts are iframes. They are loaded simultaneously if you add client web parts (app parts) in a usual way. They hold up the whole page. Users cannot interact with the intranet page until all the content in all app parts has been loaded. This can be partially improved if you introduce a delay in the app part loading, by developing an own engine. See my blog post where I mention such a concept: AppLoader Concept for SharePoint apps.
  2. The content from a SharePoint AppWeb is not fully cached. If you examine the http traffic from the apps you’ll see a lot of 304 responses, meaning the browser requests when the server answers that there is no newer version. This has an impact on the performance. See more in Alik Levin’s blog: ASP.NET Performance: Get Rid of HTTP 401 and HTTP 304. In a SharePoint-hosted app you don’t have any control what so ever to adjust the the cache settings. This is not the case in the Provider-hosted apps.
  3. SharePoint-hosted apps can only use javascript. The code is executed on the client. Older browsers like IE9 or IE8 render the pages slower. The Start Page that is slower than the rest of the Intranet is not something that will engage your users.
  4. App Parts are iframes that do not know about their dimensions. App parts often need to update the height and the width  of the parent iframe. This causes irritating flickering. Perhaps OK on some pages, but I’d say totally unacceptable on the start page of your brand new intranet..

What did we do instead of App Parts on the Start Page? We converted them into Script Editor Web Parts, the app parts were only one-time parts, they only were used on the start page.

Do not get me wrong. What we did was not abandoning apps as a model, we just removed wrong apps, apps that cannot be reused, the SharePoint-hosted apps that had big performance issues. I am looking forward creating right apps, that are written with performance, reusability, scalability and good design in mind.

 

Do not mess with the hive

In our Intranet we got this nasty double encoding error that was introduced in SharePoint April 2014 CU. In July 2014 CU the problem is still there. I often hear the suggestions to update files directly in the hive (the root folder for SharePoint: C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\) In that blog post it is described what the problem is and what files are affected and what code must be altered in order to get it to work (which I like). But it is also suggested to update the Search.ClientControls.js directly on the server (which I don’t like). So my advise is:

Do not mess with the SharePoint hive

Even though you have access, treat it as a black box, because

  • you don’t want to update the full file and the minified version of it on all servers and keep track of possible changes
  • you don’t want to loose Microsoft support because you have messed up the application folder
  • you want to think Cloud and Cloud-Ready. In Office 365 you’ll never get access to the hive.

What to do then?

Allright, enough said about the theory. What should we do to solve this issue? Here is my suggestion

  1. Find out how critical it is, perhaps you can wait until the next CU is released
  2. Contact Microsoft and let them fix it
  3. Find an alternative solution.

A solution without messing with the hive

To solve this particular problem, we can inject our fix into the Search.ClientControls.js on the fly using the SP.SOD framework. The problematic function is the $urlHtmlEncode. Before the April 2014 CU it looked as follows:

function $urlHtmlEncode(str) {
    return $htmlEncode(Srch.U.ensureAllowedProtocol(str)); 
}

So now we can reset it back through our code. Just add these lines of code to a javascript file that is loaded on all pages (within a custom master page, or a custom user action):

/// @author Anatoly Mironov
SP.SOD.executeOrDelayUntilScriptLoaded(function () {
    $urlHtmlEncode = function (str) {
        return $htmlEncode(Srch.U.ensureAllowedProtocol(str));
    }
}, "Search.ClientControls.js");

This code will register code for resetting the problematic function and execute when Search.ClientControls.js is loaded. On pages where Search.ClientControls.js is not loaded (on the majority of the pages except the Search Center), this function will not execute.

Showing Birthdays and Job Anniversaries in the Newsfeed

anniversary-001

In our project we have a requirement to show birthdays and job anniversaries in the Newsfeed. The solution is simple (when you know it) but there has not been any information about it online, until today. I want to share a few lines of PowerShell code to enable Anniversary notifications in the Newsfeed. This desired piece of code was written by my colleague Emma Degerman. Enjoy:

$job = Get-SPTimerJob | ? { $_.TypeName -like "*.UserProfileChangeJob" }
$job.GenerateAnniversaries = $true
$job.Schedule = [Microsoft.SharePoint.SPSchedule]::FromString("hourly between 55 and 55")
$job.Update()

The code retrieves the Timer Job that changes User Profiles, sets “GenerateAnniversaries” to true, then it updates the schedule to run it before the Activity Feed Timer Job and updates it. By the way, it is only applicable for SharePoint On Premises.

This is it, a quick tip for a great Intranet.

Load git into PowerShell

Just a little productivity tip. If you use git on Windows, you probably already have the Github for Windows application. This application adds the Git Shell:

006-gitshell

The Git Shell will open a PowerShell window and execute shell.ps1 from the Github directory:

007-gitshell

What it won’t do is to load your personal PowerShell profile. I want to use my PowerShell profile that creates some links and adjust the look-and-feel and the promt of the shell. By the way I have published my profile.ps1 as a gist:

I also want to have git in PowerShell available directly. The answer is in the shell.ps1 in the Github folder:

008-gitshell

So add this line to your profile.ps1 as I did:

. (Resolve-Path "$env:LOCALAPPDATA\GitHub\shell.ps1")

That’s it. If you haven’t seen the “DOT” in PowerShell scripts, it is called dot sourcing, it will execute the script and keep all the variables and references within the script.

An alternative

If you do not have Github for Windows, there is another way to load git functionality into PowerShell:

AppLoader Concept for SharePoint apps

In this post I want to share an unusual, nevertheless interesting conceptual idea of loading content from SharePoint 2013 apps on many pages. The original awesome concept was proposed and developed by my colleague Martin Villysson at Bool.

The problem we are trying to solve

SharePoint apps are great to extend functionality in SharePoint and integrate other systems (full page apps available through Site Contents), they also provide tools to enrich the default SharePoint experience by App Parts (Client Web Parts) and Custom Actions (additional menus).

One of the biggest shortcomings of that model is the need to add app parts on all pages where it is needed. Let’s say, we want to have some app parts present on every single page in our whole SharePoint tenancy, to provide a consistent look and feel (e.g. navigation, notifications). Traditionally, on premises, we have added user controls in our customized master page. In SharePoint Online that is impossible. The complicated workaround is to add those client web parts (app parts) on every page, be it manually or by automating it (powershell or app). It will require updating all pages. Nevertheless, it will not work on Out-of-the-box application pages (pages from layouts folder). It becomes even more unacceptable when you realize that your app must be added as an app instance on every single site (SPWeb) in your tenancy. 

Towards a solution

Allright, we don’t want to have thousands of app instances of the same app. What we can do is to use Tenant scoped apps (Tenant Apps). Then we’ll need only one app instance. But wait, app parts from a tenant app are only available in the parent site (HostWeb), meaning – App Catalog. That’s not good. So what Martin found in the SharePoint internal javascript code is using of _layouts/15/TenantAppInfo.ashx, a http handler that provides information about all Tenant Apps and their custom actions.That’s how the idea of the AppLoader was born.

Vesa Juvonen

After we had created a working Proof-of-Concept of the AppLoader concept, I met Vesa Juvonen at the SharePoint Conference in Las Vegas and introduced this idea to him (although I didn’t call it AppLoader). He liked it although he pointed out that this TenantAppInfo.ashx is an internal utility only in SharePoint and it is not supported by Microsoft. That’s correct. There is even almost no information about it on the Internet. But I got a feeling of Microsoft that they are willing to hear feedback and improve the product. Vesa encouraged me also to blog about it. So now I am telling about this idea. I hope to hear feedback about it. Unfortunately I cannot share the source code of the working Proof-of-Concept solution.

AppLoader Concept in colors

The AppLoader Concept is quite simple. Look at this picture:

apploader-concept

 

The solution contains a custom Master Page (blue) that references a javascript file called apploader.js (red). This file initializes the whole process. Tenant Apps (green) are the apps that an administrator has installed in the App Catalog and deployed to the whole tenancy. TenantAppInfo.ashx (black) is a handy but officially unsupported OOB service utility (http handler) that returns a json-formatted list of all Tenant Apps (green). AppLoader (red) receives the app list (black) and renders it on the Page (blue) inside new iframes (red). The page a user has navigated to can be any page (wiki page, publishing page, application page, really any page).

To summarize the colors in the diagram: red is our javascript code, green are all the tenant apps and their content, black is the utility and its output, blue is a sharepoint page and its underlying component (master page).

The steps in the AppLoader process:

  1. Make an ajax request to TenantAppInfo.ashx using XHR (XmlHttpRequest)
  2. Receive the app list
  3. For every app information, render app part, or inject css and javascript references. 

 

Reading what to render on the page

You probably have already have tried to navigate to _layouts/15/TenanAppInfo.ashx while reading this post, I know you are curious. Then you’ve noticed that there is no information about app parts. So you may ask: how do we know what app parts to render and where to put them in the page, and how do we know what resources (css and javascript files) to inject on the page. Well there is no information about it in the apps list. But if you have an app with custom actions you’ll see that they are listed in this json-formatted list we receive from the TenantAppInfo.ashx. So the solution is the brilliant idea of my colleage Martin to define custom actions in the app. CustomActions contain a ActionUrl. The ActionUrl points to the url to render (app part page) or to inject (javascript or css file). The apploader.js reads the ActionUrl in the Custom Actions for every app information and takes action upon it (rendering an app part iframe, or injecting a javascript or css file). That’s it. 

Usage and Limitations

This bright idea takes advantages from a huge SharePoint API (that contains a lot of good but not supported parts) to make using of apps in Client Application Model solutions more pragmatic and still provide a consistent design and behavior. By consistent design I mean same parts like additional navigation, notifications etc in the whole Intranet. The AppLoader renders and injects whatever you have rolled out to your whole tenancy (Tenant Scoped Apps) and that on every page (!). It also improves the perceived performance of the page load, because it renders app parts (iframes) after the main page has been loaded preventing freezing of the page. 

There are of course some limitations in the AppLoader Concept. Today we cannot rely on the TenanAppInfo.ashx API (because it is not supported and future updates can break solutions). We have to define our own custom actions in the apps. That means we can only use our own apps, it will hardly work with the apps installed from the Office Store. On the other hand, your customer will not want to have generic apps from the Office Store to be a part of every page on their intranet.

Configuring VirtualBox for SharePoint-Hosted Apps

Recently I have switched from VMWare to VirtualBox for my SharePoint Development. So far it really works good. I have prepared this guide for configuring VirtualBox for SharePoint-hosted apps. That means we need a new adapter with a static ip address. All the steps done inside the virtual machine are applicable for VMWare and Hyper-V, too. This guide does not cover the full configuration of the app environment, it covers only the network and dns settings needed for SharePoint-hosted apps on Premises in a Development machine.

In VirtualBox open Preferences:

002-vbox

In Preferences, click on Network, then on Host-only Networks, click on “plus”

003-vbox

After a new host-only Network adapter has been created, click on the screwdriver image to configure the network:

004-vbox

Remember this ip address, you can alter it, of course. Or use this: 192.168.64.1, then you can have the same settings in your environment as I have:

005-vbox

Go on to the settings for your Virtual Machine. Enable Adapter 2, as shown on the picture below.

006-vbox

The rest are the settings in your virtual machine. Open Network Connections in the Control Panel. For the newly added Network Adapter (Ethernet 2), open Properties:

007-vbox

Then open Properties for “Internet Protocol Version 4 (TCP/IPv4)”

008-vbox

Remember the ip address for you VirtualBox host-only Network? I have 192.168.64.1. Increment the last number (192.168.64.2) and use it as the ip address for your adapter. The default gateway is 192.168.64.1 as the the VirtualBox host-only Network. The DNS Server should be the same as the virtual machine, the same adapter: 192.168.64.2:

009-vbox

Open DNS Manager, right-click on Forward Lookup Zones and start the New Zone Wizard by clicking on “New Zone…”:

010-vbox

Follow the steps in the wizard:

011-vbox

Choose “Primary Zone”:

012-vbox

Keep the default setting: “To all DNS Servers running on domain controllers in this domain:

to-all-servers-vbox

Write your app domain. I use takanaapps.local:

014-vbox

Choose “Do not allow dynamic updates”:

015-vbox

Then finish the New Zone Wizard:

016-png

In the new forward lookup zone (takanaapps.local), add a new wildcard cname entry (alias):

017-vbox

Just add an asterisk and point it to your main domain (takana.local in my case):

018-vbox

After that you should be able to ping any subdomains of your app domain (xyz.takanaapps.local, abc.takanaapps.local):

019-vbox

When you start adding apps to your sites, you should add app sites to your local intranet zone (to be automatically signed in in apps webs). This setting in IE will affect Chrome as well. Go to the Options in the Internet Explorer:

020-vbox

In the Security tab -> Local Intranet, click on Sites:

021-vbox

Click on Advanced button:

022-vbox

Add your new app domain with an asterisk in front of it to the “Websites” of the Local Intranet:

023-vbox

Sources

A presentation about PowerShell and SharePoint

Yesterday I had a little presentation about PowerShell basics in the SharePoint context. Here you can see the presentation I’ve published on slideshare. The text is in Swedish.

A PowerShell one liner

PowerShell is powerful. You can write concise, well formulated, functional-style code. Recently I got the following quiz:

You’ve got $100. You have to buy exactly 100 animal, at least 1 dog, 1 cat and 1 mouse. 1 dog costs $15, 1 cat costs $1, 1 mouse costs $0.25.

There can be  many ways to solve it. But look at this one line solution. It is quite impressive what you can do with PowerShell

1..98 | % {
    $dog = $_
    1..98 | % {
      $cat = $_
      @{
        "Dog" = $dog
        "Cat" = $cat
        "Mouse" = 100 - $dog -$cat
      }
    }
  } | ? {
      $_.Mouse -gt 0
  } | ? { $_.Dog * 15 + $_.Cat * 1 + $_.Mouse * 0.25 -eq 100 }

This solution uses ranges, dynamic objects (PSObject), nested for loops, implicit returns and advanced filtering. All that is is out-of-the-box PowerShell.

AngularJS Performance Tuning for Long Lists

Anatoly Mironov:

This is a must-read for all SharePoint Developers who use Angular.

Originally posted on Small Improvements:

AnglarJS is great! But when dealing with large lists containing complex data structure, things can get very slow! We ran into that problem when migrating our core admin screens to AngularJS. The screens were supposed to work smoothly when displaying some 500 rows. But the first approach took up to 7 seconds to rende. Terrible!

We discovered two main performance issues for our implementation. One is related to the ng-repeat directive, the other was related to the filtering.

The following article summarizes our experiences with different approaches to solve or mitigate the performance problem. It will give you ideas and hints, what you can try out yourself and what is maybe not be worth a try.

Why is ng-repeat in AngularJS slow with large lists?

The ng-repeat directive of AngularJS is getting slow above 2500 two-way data bindings. You can read more about this in a post by Misko Hevery…

View original 1,404 more words

Discovering SharePoint

And having fun doing it

Bram de Jager's SharePoint blog

My view and thoughts on SharePoint.

My programming life

and everything in between

SharePoint Development Lab by @avishnyakov

It is a good place to share some SharePoint stories and development practices.

SharePoint Dragons

Nikander & Margriet on SharePoint

The Zuul Cat Idea Brewery

Where ideas on software development and entrepreneurship brew.

Paul J. Swider

Inspire! Teach! Awe!

Mai Omar Desouki - Avid SharePointer

Egyptian & Vodafoner - Senior SharePoint Consultant

Alexander Ahrens

MCPD | SharePoint | Web Development | JavaScript | .NET

Cameron Dwyer | SharePoint, Outlook, OnePlaceMail

OnePlaceMail, SharePoint, Outlook & Life's Other Little Wonders

.:paul.tavares:.

Me and My doings!

Share SharePoint Points

By Mohit Vashishtha

Jimmy Janlén "Den Scrummande Konsulten"

Erfarenheter, synpunkter och raljerande om Scrum från Jimmy Janlén

SPJoel

SharePoint for everyone

SharePointRyan.com

Ryan Dennis is a SharePoint Solutions Architect with a passion for SharePoint and PowerShell

SharePoint 2020

The Vision for a Future of Clarity

Aharoni in Unicode, ya mama

Treacle tarts for great justice

Follow

Get every new post delivered to your Inbox.

Join 227 other followers