CHUVASH.eu

CHunky Universe of Vigourous Astonishing SHarepoint :)

Tag Archives: Sharepoint

Smarter way of loading SharePoint CSOM dll in PowerShell

Have you also got a legacy powershell script that loads SharePoint dlls and runs CSOM code directly? It’s quite easy to convert to PnP PowerShell. But if you run out of time and just need to execute the script, then I have a quick tip for you.

First of all, a CSOM script can be recognized by Add-Type commands (or Import-Module) plus the SharePoint dll paths.

Loading the dll the old way.

The odds are high that you don’t have those directories and files, unless you run it on a SharePoint Server (who would do that at all?) or you have installed the SharePoint SDK.

SharePoint SDK can be downloaded and installed (as suggested here), but why would you want to do that? An easier way is just to locate the files that are distributed with the PnP.PowerShell module, let me show how to do that.

All the dlls are available from the PnP.PowerShell module directory:

So the only thing you need to do is to re-point the path from the original (the “GAC”) folder to the PnP.PowerShell folder. You don’t need to guess the folder. It’s easy.

Thanks to the PowerTip: Find the Path to a PowerShell Module (Scripting Guy) I could find a way to read the information dynamically, so it doesn’t matter where your folder actually is. The fact what version number the module has, what OS you run on, and whether or not you installed it for your user account only or for all users on your computer – allt that has impact on the folder location. So we need to read the right path and then use it in the Add-Type command.

$pnpModule = Get-Module ListAvailable PnP.Powershell
$base = $pnpModule.ModuleBase
Connect-PnPOnline $url Interactive
Add-Type Path "$base\Framework\Microsoft.SharePoint.Client.dll"
Add-Type Path "$base\Framework\Microsoft.SharePoint.Client.Runtime.dll"
view raw add-type-csom.ps1 hosted with ❤ by GitHub

Other notes

The PnP.PowerShell is built on top of.NET Core and it works cross plattform, that’s better.

Loading dlls on a Mac.

If your legacy script does not work with the newer PnP.PowerShell, you might need to install the older PnP PowerShell and adjust the module name in the script above accordingly.

The SharePoint SDK is built on top of .NET Framework (as of my understanding) and it can only be installed on a Windows machine.

The SharePoint SDK requires local administrator rights to be installed. The PnP.PowerShell can be installed for a user without beeing an administrator by adding -Scope CurrentUser (to the Install-Module), which makes the work much smoother.

If you have two or more versions of the PnP.PowerShell module installed, you have to adjust the script a little by loading only the latest version of the module:

$pnpModule = Get-Module PnP.PowerShell ListAvailable | Sort-Object Version Descending | Select-Object First 1

That was a quick tip on how you can use the types from the original CSOM libraries when you don’t have time to convert a script to a PnP code or if there is some functionality that is not covered in PnP yet (not quite sure if there is something you cannot do with PnP that you can do with CSOM).

The good sides of that approach:

  • it can be a step towards rewriting a legacy script to a newer PnP.PowerShell
  • the dlls are up-to-date thanks to an easy way to update the PowerShell Module (Update-Module)
  • it is cross platform, meaning you can execute your legacy script on a linux or on a Mac as well, good for automation!

Monitoring Microsoft 365 using Raspberry Pi and M365 CLI

I would like to show you my recent hobby project with a raspberry pi, a unicorn phat and the powerful cli-microsoft365: A simple monitoring solution of Microsoft 365 Services.

Status of some important services in Microsoft 365

In essence, I put the unicorn phat onto the raspberry pi zero w and wrote this python script:

https://github.com/mirontoli/tolle-rasp/blob/master/python_scripts/unicorn03_m365.py

The python script checks the service status every five minutes and shows it with colors on the unicorn phat.

Color coding

Since the unicorn phat is just a grid of 8×4 rgb leds, I needed to color code the different service statuses (more on the statuses later in this post). I came up with these color combinations. It doesn’t matter what combinations they are as long as they mean something to you (or as long as you can decode them).

  1. 🟩 🟩 🟩 🟩 ServiceOperational
  2. 🟩 🟩 🟩 🟨 ServiceRestored
  3. 🟪 🟪 🟪 🟪 Investigating
  4. 🟩 🟩 🟩 🟪 FalsePositive
  5. ⬜️ ⬜️ ⬜️ ⬜️ InformationUnavailable
  6. 🟥 🟥 🟥 🟥 ServiceInterruption
  7. 🟥 🟥 🟥 🟨 ExtendedRecovery
  8. 🟥 🟥 🟨 🟩 ServiceDegradation
  9. 🟩 🟩 🟩 🟦 PIRPublished
  10. 🟥 🟨 🟨 🟩 RestoringService

ServiceStatus

There is a list of all possible statuses you can get for Microsoft 365 Services, and it is here:

cli-microsoft365

Install the cli-microsoft365 npm package globally.

sudo npm i -g @pnp/cli-microsoft365

You have to log in, admin consent (if you run this for the first time) and then you can get the status of the Microsoft 365 Services by running:

m365 tenant status list

Services

There are many services in Microsoft 365. I choose the 8 most important ones (from my point of view), because there are only 8 rows on the unicorn phat, you can choose your services and order them as you prefer of course. Beware the spelling and the casing:

  1. SharePoint
  2. microsoftteams
  3. Exchange
  4. OneDriveForBusiness
  5. yammer
  6. Forms
  7. PowerBIcom
  8. Intune

Assembling the hardware

I had my raspberry pi zero w, with raspberry pi os already installed. I attached the unicorn phat using solderless pogo pins. I found a little white cardboard box, cut out a rectangular hole for the unicorn phat and glued the raspberry pi with unicorn inside the box. On the front side I put a sticker with the actual service names for every led row. I connected it to the power, ran the script.

Only three pins are needed.
I tested it first without a box.
I glued the hardware on the inside of the cardboard with a glue gun.
Exploring the @pnp/cli-microsoft365.

Other tips and tricks

The pogo pins were to loose and the leds did not work. I had to shorten the plastic holders a little to tighten the the pins.

Login to cli-microsoft365 as sudo

When I explored the m365 commands, it worked perfectly. My login was cached. Then I needed to run my scripts as sudo, since it requires communication with GPIO pins and the unicorn phat. It didn’t work. The login cache is in different place if you run as sudo. Obvious, when I look at it afterwards, but it took some time to realize that. So, if you are going to do the same, just make sure you log in to m365 as sudo as well, before running the script:

sudo m365 login

Pinout

This web resource is gold, it shows the pinout and connections to many hats etc:

You only need three pins:

  • 5V Power (Pin 2)
  • Ground (Pin 6)
  • GPIO 18, Data (Pin 12)

subprocess in python

m365 is a command that you can run in terminal, from a python script I use subprocess to call it and get the results.

Running the script even when you log out

start the script with nohup:

nohup python3 unicorn03_m365.py &

1 TB = 1024 GB in SPO?

There is confusion around how the storage is calculated in SharePoint Online. I believe, in SharePoint Online 1 TB is 1024 GB (based on powers of two), although the SI Prefix is for numbers based on powers of 10 (1TB = 1000GB, Wikipedia). In this post I would like to summarize the results of my investigations and I hope Microsoft or the community can confirm or disconfirm this.

First, let me explain why we care about it. The storage in SharePont is limited and we need to keep an eye on it. Especially in our case, where we need to track storage utilization across different parts of the organization/our tenant. The storage in SharePoint is calculated like so:

1 TB + 10GB * E-licensed users

The tricky part, though, is how to convert it into TB correctly.

Why I believe Microsoft treats 1 TB as 1024 GB

First of all, I can see it clearly in my dev tenant with exactly 25 licenses.

That would give 1TB + 10GB*25 = 1,25 TB if it would be based on powers of 10. But it isn’t because the storage I get is 1,24 TB, or 1,244 to be precise.

That means, for every E-license you get 10 GB or 10/1024 TB.

That also means you need more licenses to get the desired storeage. E.g. 10 TB more storage requires 1024 licenses and not 1000, 10 TB = 10240 GB, 10240 GB / 10 = 1024 E-licenses.

Also in OneDrive, the initial space I get, is 1024 GB (or 1TB). If 1TB = 1024GB in OneDrive, why should SPO be different?

Further, the MSDocs page reveals that the 25 TB are 25600 GB (which is exactly the product of 25 and 1024):

One contradictory page, though is the news about storage increase:

The calculations there are based on the decimal system:

Calculation of MB and GB

Just to verify how the storage is calculated in KB, MB and GB, I looked at the Storage of a SharePoint site. Luckily, I can get the storage used in Bytes, MB and GB (from different sources) and compare them to each other.

When I calculate back and forth I can defnitely see, it is multipled/divided by 1024, hence powers of 2:

The values in blue are the reported values. The other values are calculated.

The values in GB are exactly the same, the Bytes, KB and MB differ a bit due to rounding

Demote News in SharePoint Online

In case you published a news in SharePoint Online and you now want to “downgrade” it to a regular page, I have a solution for you.

The reasons why you would like to demote a News might vary:

  • You publish an important message, perhaps a note about an operational disturbance. It might not relevant anymore, but you still want to keep the page in case someone wants to access the information.
  • You accidentally create a page as a news.
An example of a news of a temporary character

The News “Promotion” is controlled through the field called “Promoted State”. You cannot edit, but you can include it in a view to check the current state of it.

  • “0” means a regular site page
  • “2” means News
Show the “Promoted State” and ID to check the current status and identify pages and news

So in order to convert a news to a regular page, you need to update the list item’s field “Promoted State” from 2 to 0.

Unfortunately, the Promoted State field is read only. Hence we need to make it editable first, edit it, and in the end, reset the field.

Lucky us, that is easily done with PnP 🙏🙏🙏🙏🙏

$site = "https://{tenant}.sharepoint.com/sites/{site}"
Connect-PnPOnline Url $site SPOManagementShell ClearTokenCache
$field = "Promoted State"
$list = "Site Pages"
$id = 85 #the item id of the news
Set-PnPField Identity $field List $list Values @{ReadOnlyField=$false}
#make sure to enclose the number into quotation marks
Set-PnPListItem List $list Identity $id Values {"Promoted State"="0"}
#reset the field
Set-PnPField Identity $field List $list Values @{ReadOnlyField=$true}
view raw demote-news.ps1 hosted with ❤ by GitHub

That’s it. Easy, isn’t it?. I wonder why the field is readonly, though. What downsides might this procedure have? What do you think?

Kombinera två SharePoint-listor i PowerBI

Det här är en enkel guide på svenska om hur du kan ladda in data från två eller fler listor i SharePoint och lägga ihop dem till en.

Scenariot är följande. Du har två eller fler sajter i SharePoint Online som har var sin lista (med samma kolumner). Du vill ladda in data från båda och se en aggregerad/summerad version. Alternativet är att ha en delad lista, men ibland (av behörighetskäl eller av behovet för smärre anpassningar av enskilda listor), ligger det i separata listor/sajter.

För enkelhetens skull, har jag följande demouppsättning:

  • Två enkla listor i samma sajt:
    • ShapesA
    • ShapesB
  • Listorna innehåller två exakt likadana kolumner
    • Shape
    • Color
Första listan
Andra listan

Målet är att addera raderna ifrån två listorna till en större lista.

Power BI

Att läsa in SharePoint-listor in i Power BI är ganska enkelt. Starta Power BI Desktop på din Windows-dator.

Man börjar med Get Data…
Filtrera på SharePoint och välj SharePoint Online List
Klistra in länken till din sajt
Välj listorna
Viktigt! Tryck på “Transform Data” först
Radera alla kolumner du inte behöver. Gör det på båda listorna.

Nästa steg är det vikigaste i den här guiden: Kombinera de här två listorna: Append Queries

Append Queries
Välj “Append Queries as New”
Välj lista 1 och lista 2 och klicka på OK
När den är färdig, klicka på Close & Apply

Den kombinerade datan ligger i “Append1”

Append1

Resten är “bara” visualisering. “Bara”, eftersom det hårdaste jobbet (Tranformation, kombinering) är redan bakom oss. Testa olika alternativ.

Visualisering

Ett exempel är Treemap:

Treemap
För att få till Treemap i det här exemplet behövde jag ha Title (=Shape) både i Group och Values

Summering och reflektion

Den här guiden visar hur man kombinerar (lägger på) två listor och jobbar med dem som om de vore en lista. Hör av dig om det har varit till hjälp eller om du har frågor eller funderingar.

Själva behovet är verkligt. I SharePoint har det alltid varit en utmaning att samla ihop datan från olika sajter och listor. Vi har använt oss av söken (med Sökcenter och DisplayTemplates) och andra tekniker. Visst vore det enklare att ha det samlat i en central databas eller en lista. Det är dock en viss frihet att låta olika organisatoriska enheter “äga” sina delar (användare kan justera sina vyer, ordna formatering, lägga till extrakolumner och annat trevligt). Aggregeringsbehovet kan nu, med Power BI, lösas på ett relativt smidigt och användarvänligt sätt.

Add a security group as hubbers using PowerShell

Today I needed to add a security group to “People who can associate sites with this hub” through PowerShell. Here is quick how-to. I usually say “hubbers” instead of the long “People who….”. By the way, if you want to know what prerequisites there are for being a hubber, read my other blog post

An example of how to add a security group is missing in the MS Docs:

This is easy, all you have to do is to find out the security group id, prepend it with “c:0t.c|tenant|” and run the command.

$url = "https://tenant.sharepoint.com/sites/hubsite1"
$groupId = "0b2d5885-158a-4aa1-bea3-cd28f71fcc37"
$securityGroup = "c:0t.c|tenant|$groupId"
Grant-SPOHubSiteRights $url -Principals $securityGroup -Rights Join

That’s it. I’ll try to provide this example to the MSDocs github.

Two workarounds for overcoming the listview threshold

These are two workarounds to see documents / list items in a view that exceeds the listview threshold of 5000 items.

Overcoming the listview threshold is as fun as succeeding in the limbo games.

This is changing all the time. When you read this, it might have changed. Today, 2021-01-05, me and my colleague found following two workarounds for listing over 5000 items in a list view in SharePoint Online:

  1. Sorting by name in a view
  2. Adding a shortcut to OneDrive

Both methods require the Modern UI in SharePoint Online.

In our case we have a migrated document library with many items. In the source, the threshold was much higher, in SharePoint Online some folders didn’t show anything. It showed only “Something went wrong”:

The classic view had a better error message, but no solution for that:

This view cannot be displayed because it exceeds the list view threshold (5000 items) enforced by the administrator.
To view items, try selecting another view or creating a new view. If you do not have sufficient permissions to create views for this list, ask your administrator to modify the view so that it conforms to the list view threshold.

Sorting by Name

We will split those big folders into smaller ones. But while trying things out, we found that having Sorting by name, suddenly showed the documents in the big folders.

As a user you have to scroll a lot to find your document (because of the infinite scroll), but still, now you are able to see your documents!

Also, you can start selecting files and re-organizing them by using “MoveTo”.

Adding shortcut to OneDrive

Another workaround (or I’d rather say trick) is to open the folder from within your personal OneDrive by adding a shortcut:

With that you’ll get the folder linked in your OneDrive.

Even with a big number of files, OneDrive will list the folder. Why is that? Maybe, Microsoft treats personal OneDrives differently, more gently, in a more forgiving way.

On your computer you’ll see the linked OneDrive folder, too.

Listing all renamed sites in SharePoint Online

When you rename a site, a new site is REDIRECTSITE#0, you can get all the sites of that type by running

Get-SPOSite -Template REDIRECTSITE#0

Please consider some caveats with renaming a site url

Automatically detect new sites in SharePoint Online

Original image by William Warby. https://www.flickr.com/photos/wwarby/16414155179/in/photostream/

Sites in SharePoint are created all the time, not only for SharePoint, but also as storage for Yammer, Teams, Planner and other services in Microsoft 365. There are ways to keep track of them, but the ability to automatically detect a new site creation is quite appealing. Automatic detection means a trigger of a Power Automate (Flow) or a Logic App.

There are a few blog posts that exactly describe how you can detect when a new site is created in SharePoint Online:

The provided blog posts are great how-tos, I am not giving you a new how-to for that, I’d like to reason about that solution.

The solution for automatic detection of new sites

Power Automate and Logic Apps can listen to new items in SharePoint. There is a list in the admin site (tenant-admin.sharepoint.com) that has SharePoint Sites as list items, its name is DO_NOT_DELETE_SPLIST_TENANTADMIN_ALL_SITES_AGGREGATED_SITECOLLECTIONS.

That’s it, in essence, it’s just setting up a new flow with “When an item is created in SharePoint” as a trigger, and you have thousands business scenarios you could implement, but let’s dig a little bit deeper.

One List to rule them all

Honestly, I was not aware of that list before I started looking at that. What is that list, why is it called DO_NOT_DELETE_SPLIST_TENANTADMIN_ALL_SITES_AGGREGATED_SITECOLLECTIONS.

The name is hillarious. Why name something to “DO_NOT_DELETE…” and all capslock🤣. But I suppose, there were support cases.

Beware, that list is not documented, that means you’re on your own when Microsoft changes the name or moves the list to somewhere else. So don’t build business critical solutions with that.

From what I can see, that list keeps information about all sites (site collections) in SharePoint Online, even those that are deleted and permanently deleted (?). This might be a source for deeper troubleshooting in some scenarios. It is like an old card index in a library you might have seen long time ago. It is hidden nowadays, but it is still there.

Image by LisaJasminAdams from Pixabay

First, that list is in the SharePoint Admin Site Collection, you need to be at least a SharePoint Administrator to access it. Okay, I’d like to know what’s more in its Site Contents (_layouts/15/viewlsts.aspx):

Well, the UI of that page has not been focused on, but nevermind, the lists are there. But you cannot navigate to that list in the browser directly:

It doesn’t matter since we can use it as a trigger but also the SharePoint REST API to get the items, e.g.:

https://tenant-admin.sharepoint.com/_api/Web/Lists/GetByTitle('DO_NOT_DELETE_SPLIST_TENANTADMIN_AGGREGATED_SITECOLLECTIONS')/Items

You can see more examples of listing the sites in the linked posts. Unfortunately I bumped into an issue when trying to filter the results. If that list contains more than 5000 items (and it will, soon or later), you’ll have to deal with the ListView Threshold.

If you filter on Modified, you won’t able to anything because of the ListView Threshold, but filtering on Created will work.

But this is a side note, this post is automatically detecting new sites, not listing them

Alternative solutions

Using this kind of a hidden list mentioned above is a bit of a hack. I’d say it’s okay as long as it works, and it serves an complementary function, e.g. notifying IT about new sites, and the work is backed up by documented and reliable alternatives:

SharePoint Online Admin

Visiting “Active Sites” in SharePoint Online Admin gives you all the sites, you can sort by Created and see all the new sites. You cannot set up an alert or a flow directly from that, but maybe there will be some built-in functionality for that.

Office 365 Usage Reports

You can get all the sites in an Usage Report, their created, size, last activity etc. It’s not real time, but if you’re fine with 1-2 days delay, you can get this report, extract the new ones and do whatever you wanted to do in your original scenario/need.

SharePoint PowerShell Module

It’s worth mentioning, too, although it’s “heavy”. In a tenant with many sites, the scripts for getting all the sites and connected groups may take hours. I am refering to those scripts that start with Connect-SPOService.

Permissions, Licenses, and Security

The SharePoint connection that listens to the DO_NOT_DELETE_SPLIST_TENANTADMIN_ALL_SITES_AGGREGATED_SITECOLLECTIONS list in the Admin Site Collection needs to be set up with a SharePoint Administrator role account. Beware of who has access to that solution (Power Automate or Logic App), this SPO Admin connection in wrong hands can be disastrous. Especially in Azure, pay attention to who has access to the resource, but also to the resource group and the azure subscription.

The account who sets up a Power Automate needs obviously an appropriate license and also Power Automate activated. In my scenario, I don’t need any premium connectors, but depending on your solution, you might need to license your account appopriately.

Scenarios

In “my” scenario, I want to be notified of all new sites in my business unit within a shared tenant, so that we can contact the site owners, provide guidance and also provision important parts (initial folder structure, some spfx solutions etc).

What is your sceanario?

The code

When I am done developing my proof-of-concept, I’ll try to share more details on the actual implementation. It might be an idea to submit the template to the Microsoft Power Community, but I am not sure it will be accepted, given the fact that it uses undocumented and hidden parts of SharePoint Online that soon or later will be subject to change.

Daniel Chronlund Cloud Tech Blog

News, tips and thoughts for Microsoft cloud fans

Вула Чăвашла

VulaCV - Чăвашла вулаттаракан сайт

Discovering SharePoint

And going crazy doing it

Bram de Jager - Architect, Speaker, Author

Microsoft 365, SharePoint and Azure

SharePoint Dragons

Nikander & Margriet on SharePoint

Mai Omar Desouki

PFE @ Microsoft

Cameron Dwyer

Office 365, SharePoint, Azure, OnePlace Solutions & Life's Other Little Wonders

paul.tavares

Me and My doings!

Share SharePoint Points !

By Mohit Vashishtha

Jimmy Janlén "Den Scrummande Konsulten"

Erfarenheter, synpunkter och raljerande om Scrum från Jimmy Janlén

Aryan Nava

DevOps, Cloud and Blockchain Consultant

SPJoel

SharePoint for everyone

SharePointRyan

Ryan Dennis is a SharePoint Solution Architect with a passion for SharePoint and PowerShell

SharePoint 2020

The Vision for a Future of Clarity

Aharoni in Unicode

Treacle tarts for great justice

... And All That JS

JavaScript, Web Apps and SharePoint

blksthl

Mostly what I know and share about...