CHUVASH.eu

CHunky Universe of Vigourous Astonishing SHarepoint :)

Kombinera två SharePoint-listor i PowerBI

Det här är en enkel guide på svenska om hur du kan ladda in data från två eller fler listor i SharePoint och lägga ihop dem till en.

Scenariot är följande. Du har två eller fler sajter i SharePoint Online som har var sin lista (med samma kolumner). Du vill ladda in data från båda och se en aggregerad/summerad version. Alternativet är att ha en delad lista, men ibland (av behörighetskäl eller av behovet för smärre anpassningar av enskilda listor), ligger det i separata listor/sajter.

För enkelhetens skull, har jag följande demouppsättning:

  • Två enkla listor i samma sajt:
    • ShapesA
    • ShapesB
  • Listorna innehåller två exakt likadana kolumner
    • Shape
    • Color
Första listan
Andra listan

Målet är att addera raderna ifrån två listorna till en större lista.

Power BI

Att läsa in SharePoint-listor in i Power BI är ganska enkelt. Starta Power BI Desktop på din Windows-dator.

Man börjar med Get Data…
Filtrera på SharePoint och välj SharePoint Online List
Klistra in länken till din sajt
Välj listorna
Viktigt! Tryck på “Transform Data” först
Radera alla kolumner du inte behöver. Gör det på båda listorna.

Nästa steg är det vikigaste i den här guiden: Kombinera de här två listorna: Append Queries

Append Queries
Välj “Append Queries as New”
Välj lista 1 och lista 2 och klicka på OK
När den är färdig, klicka på Close & Apply

Den kombinerade datan ligger i “Append1”

Append1

Resten är “bara” visualisering. “Bara”, eftersom det hårdaste jobbet (Tranformation, kombinering) är redan bakom oss. Testa olika alternativ.

Visualisering

Ett exempel är Treemap:

Treemap
För att få till Treemap i det här exemplet behövde jag ha Title (=Shape) både i Group och Values

Summering och reflektion

Den här guiden visar hur man kombinerar (lägger på) två listor och jobbar med dem som om de vore en lista. Hör av dig om det har varit till hjälp eller om du har frågor eller funderingar.

Själva behovet är verkligt. I SharePoint har det alltid varit en utmaning att samla ihop datan från olika sajter och listor. Vi har använt oss av söken (med Sökcenter och DisplayTemplates) och andra tekniker. Visst vore det enklare att ha det samlat i en central databas eller en lista. Det är dock en viss frihet att låta olika organisatoriska enheter “äga” sina delar (användare kan justera sina vyer, ordna formatering, lägga till extrakolumner och annat trevligt). Aggregeringsbehovet kan nu, med Power BI, lösas på ett relativt smidigt och användarvänligt sätt.

Add a security group as hubbers using PowerShell

Today I needed to add a security group to “People who can associate sites with this hub” through PowerShell. Here is quick how-to. I usually say “hubbers” instead of the long “People who….”. By the way, if you want to know what prerequisites there are for being a hubber, read my other blog post

An example of how to add a security group is missing in the MS Docs:

This is easy, all you have to do is to find out the security group id, prepend it with “c:0t.c|tenant|” and run the command.

$url = "https://tenant.sharepoint.com/sites/hubsite1"
$groupId = "0b2d5885-158a-4aa1-bea3-cd28f71fcc37"
$securityGroup = "c:0t.c|tenant|$groupId"
Grant-SPOHubSiteRights $url -Principals $securityGroup -Rights Join

That’s it. I’ll try to provide this example to the MSDocs github.

Own praise badges in Teams

While reading the Teams News recap from December 2020, I found one thing that caught my attention particularly: Custom Praise Badges in Teams. Let’s try this. But first, those badges are the default ones:

They are good starting point. But to take it a step further, to really engage people and praise, you need some specific badges that mean something for your company.

To add a new badge all you have to do is to go to Teams Admin (obviously it requires you having the Teams Administrator Role):

  1. Click on Teams Apps –> Manage Apps
  2. Search for “Praise” and open it
  3. Click on Settings and scroll down to Custom Badges
  4. Click on “Create a custom badge”
  5. Give it a name, upload a picture, define the text color and the background color.
That’s how you add a new custom badge as a Teams Administrator.

That’s it! When I added my little construction worker badge, it took seconds (but be patient if takes more time).

You can select the new badge when you are about to praise someone.

Custom badges might be this little “extra” that make difference for better adoption and better collaboration in your team and your organization.

Here I praise my colleague Adele Vance. Well you know Adele Vance, she works in every demo Office 365 environment 😜

Of course, for those who has not tried the Praise feature in Teams yet, I highly recommend it. Next time you want to say “Thanks” to someone or show your appreciation, just hit this tiny badge button within the “New Conversation” in Teams.

The size

The badge image should be 216×216 px.

Learn more on MSDocs:

Manage the Praise app in the Teams admin center – Microsoft Teams | Microsoft Docs

Two workarounds for overcoming the listview threshold

These are two workarounds to see documents / list items in a view that exceeds the listview threshold of 5000 items.

Overcoming the listview threshold is as fun as succeeding in the limbo games.

This is changing all the time. When you read this, it might have changed. Today, 2021-01-05, me and my colleague found following two workarounds for listing over 5000 items in a list view in SharePoint Online:

  1. Sorting by name in a view
  2. Adding a shortcut to OneDrive

Both methods require the Modern UI in SharePoint Online.

In our case we have a migrated document library with many items. In the source, the threshold was much higher, in SharePoint Online some folders didn’t show anything. It showed only “Something went wrong”:

The classic view had a better error message, but no solution for that:

This view cannot be displayed because it exceeds the list view threshold (5000 items) enforced by the administrator.
To view items, try selecting another view or creating a new view. If you do not have sufficient permissions to create views for this list, ask your administrator to modify the view so that it conforms to the list view threshold.

Sorting by Name

We will split those big folders into smaller ones. But while trying things out, we found that having Sorting by name, suddenly showed the documents in the big folders.

As a user you have to scroll a lot to find your document (because of the infinite scroll), but still, now you are able to see your documents!

Also, you can start selecting files and re-organizing them by using “MoveTo”.

Adding shortcut to OneDrive

Another workaround (or I’d rather say trick) is to open the folder from within your personal OneDrive by adding a shortcut:

With that you’ll get the folder linked in your OneDrive.

Even with a big number of files, OneDrive will list the folder. Why is that? Maybe, Microsoft treats personal OneDrives differently, more gently, in a more forgiving way.

On your computer you’ll see the linked OneDrive folder, too.

Teams incoming webhooks for performance monitoring

Incoming webhooks in Teams are great, indeed. Last week I saw this tweet, which inspired me to share one of our webhooks.

There are many scenarios where it can be used, I would also like to share one of our scenarios: notifying our DevOps team about performance issues in SharePoint Online, detected through Exoprise.

This adaptive card (in Swedish) shows an alarm from Exoprise about login problems.

It’s simple and scalable:

  • On the Exoprise end you can define any combination of larms and connect it to a Teams Incoming Webhook.
  • On the Teams end, you can invite all the people who need to be notified, and you can also set up push notifications if you have Teams installed on your mobile.

This is just a little “demo”. We use Exoprise Larms, Teams and Adaptive Cards. I can share more technical details if you are interested, but in that case, leave a comment below.

MagicMirror2 and Chuvash

This post is about my setup of the popular MagicMirror2 application. I show the steps needed to set it up on a Raspberry Pi Zero W and connect it to a TV set. As a bonus, I share my thoughts on the Chuvash localization work.

MagicMirror2 is a DIY project and an open-source application, voted to number one of the best Raspberry Pi Projects. In essence, it shows information of your choice (weather, calendar, news) on a screen that is embedded in a mirror.

I was introduced to MagicMirror2 by my colleague, who uses it in another way: rather than having it in a mirror, he has it on his smart tv. The customisability is the beauty of the whole DIY and the Raspberry Pi.

Installation

I also decided to use it on my tv, through the built-in web browser. My raspberry pi zero w required some special steps due to its processor architecture. Fortunately, I found a guide for MagicMirror2 and Raspberry Pi Zero W which I used as a starting point.

After trying it, I found that I needed those steps for my server-only magic:

  • Install node and npm for armv6l
  • Clone the MagicMirror
  • Install npm packages targeting the armv7l
  • Run the server
# node from apt won't work
wget https://nodejs.org/dist/v10.16.0/node-v10.16.0-linux-armv6l.tar.xz
tar xvf node-v10.16.0-linux-armv6l.tar.xz
cd node-v10.16.0-linux-armv6l/
sudo cp -R * /usr/local/
sudo reboot
git clone https://github.com/MichMich/MagicMirror
cd MagicMirror
# it needs to be armv7l, not armv6l
npm install -arch=armv7l
# if errors arise: rm -rf nod_modules. Then npm install again
npm run server .
view raw magicmirror.sh hosted with ❤ by GitHub
Creating a bookmark on the TV browser.

My tv screen with MagicMirror2 localized into Chuvash

Chuvash

Every time I work with software I try to think: can I help to translate/localize it to Chuvash. Chuvash is a “little” language, it is only spoken by 1.5 million people. Almost in all cases, it is hard to even to register the Chuvash as such (like in Windows, or macOS), and it is even harder to localize applications, because they tend to have thousands or sometimes millions of strings to translate (and maintain!).

With MagicMirror I experienced how work from the past can help today, how small pieces can become connected parts of a bigger picture.

MagicMirror uses momentjs, a javascript library for datetime. In matter of fact, that’s exactly something that I already have worked on:

That was nice! I changed language to “cv” in the config/config.js and the most of the interface turned into Chuvash. (cv – is the iso code for Chuvash language). What a feeling of joy!

Why that excitement? Well, almost everything in the Chuvash IT (UX, localization, keyboard layouts, speech recognition, machine translation etc) is driven and sponsored by volunteers and a community. That’s why it is a special joy to see pieces come together, even though it is a small DIY project.

The MagicMirror2 itself does a minimal set of strings to translate. I translated it and submitted a Pull Request, which has been already merged to the develop branch.

I also created a list of Compliments in Chuvash:

DIY: Integrating Trådfri lights with Teams presence

It seems that Work from Home (WFH) is here to stay, it’s okay. I’d say, Work from a Smart Home is even more okay. To me, Home Automation (HA) and Work from Home (WFH) are really two peas in a pod.

Today’s “guest” is a tiny application that I’ve set up on my raspberry pi to listen to my presence (status) in Teams and show it with colors of my smart RGB light (IKEA Trådfri).

The code

You can find the application on github:

I’ll try to provide the needed documentation on the github repo and focus more on the story part in this blog post.

The story

I stumbled upon Elio Struyf’s blog post and I was amazed:

Wow! I thought immediately: that would be a cool challenge, I wanted to set up this, too. Although, with some adjustments for my smart home:

  • I wanted to run the whole application on one raspberry pi only, because I don’t have the second one, neither I have HomeBridge installation (maybe something for future projects, though).
  • I wanted to have as little code as possible, maintenance should be kept to minimal.
  • I wanted to use python in order to learn more python and because python seems to be the most supported language on the pi.
  • I wanted to use IKEA Trådfri lights (with a gateway and a remote) that I already have invested into.

I omit the configuration steps for Trådfri lights and Raspberry Pi, you can see them in my previous blog post:

Why show Teams presence with a smart light

Elio wrote his blog post in April this year – in the times of the lockdown in Belgium. In Sweden, we hadn’t a real lockdown, but it seems that it might come times when my children would need to be at home more while I work. In that case a superclear system that shows when I have important meetings is just awesome. Maybe, with that I am prepared for such times.

But to be really honest, the main driving factor is the fact that it is very satisfying to tinker around with this DIY stuff 😜😎

Lessons learned

There is a python wrapper for MSGraph which is awesome, but it needs more contributors:

In your Azure AD App Registration you can specify auth flow type as public, with that you don’t need to store a client secret for delegated access. That was a kind of a new thing to me.

Presence endpoint in MSGraph is in beta, make sure you call the beta endpoint. The scope is ‘https://graph.microsoft.com/Presence.Read’ and you need an admin consent for that permission grant.

Mass remove live photo videos

While importing images from my iPhone using Image Capture on my mac, I discovered that almost all pictures had corresponding videos. They had the same name, only the file extension was different:

  • IMG_2829.JPG
  • IMG_2829.MOV

For archiving I don’t to have live photo videos, that’s why I needed a script for that. I found a good start in an answer on SuperUser.stackexchange.com:

I altered it a bit to remove the corresponding .MOV-file (rm):

find . -type f -name "*.JPG" | \
while read -r f;do [ -e "${f%.JPG}.MOV" ] && rm "${f%.JPG}.MOV";done

Flashing Trådfri lights on Azure Alerts

What if you put together Work From Home and Home Automation? Well, removing the common denominator (HOME) would mean Work Automation (sic!). I want to tell you about a tiny hobby project I have had at home, still related to work of mine: Whenever an Azure alert is triggered, my Trådfri smart light from IKEA flashes for a couple of seconds.

Summary (if you want to skip the long story below): The solution is a tiny web application. The publicly accessible url, exposed using ngrok, is registered as a webhook in an Azure Alert. It’s on Github, you’re welcome to use it as you please 😎:

How I did this (the long story)

The github repo (linked above) is self-expaining, but here comes the story. I used the same setup for Azure Alerts as described in my previous blog post:

When I was done setting up an alert, I thought: besides a notification in a Teams channel, I thought: what if I could show the alert visually using some LED or similar? Then I thought about Home Automation and a Trådfri RGB bulb I’ve got. That’s the beauty of the mentioned equation: Work From Home and Home Automation. We can pick the best parts of it and combine to something unique.

Since I have a kit from IKEA containing a gateway, a remote, and an RGB lamp, I wanted to do something with that. Unfortunately I didn’t find any routines (Google Home), applets (IFTTT) or automations (Home app in iOS) that could do it.

Luckily, there is a way of controlling the Trådfri lights, best described in this tutorial:

As in this tutorial I also used a Raspberry Pi Zero W, and it went very well, except one thing: Trådfri team introduced a change for the security code, I needed an additional step that was missing, more on that later.

The flow from an Azure Alert to the flashing light.

The tutorial says: the world is your lobster. My “lobster” is a webhook that makes lights flash on an alert, so I needed to have a simple web server (http.server) and a tunnel to my network (ngrok). It was best to take one step at a time.

Step 1. Connect

First, I wanted to make sure I could have a simple web server that could host my webhook. I followed the advice from that tutorial and used http.server python module:

I didn’t need to install any additional modules, you have this already on the Raspberry Pi OS. Just create a simple file like this:

from http.server import BaseHTTPRequestHandler, HTTPServer
host_name = '192.168.0.193'
host_port = 8000
class MyServer(BaseHTTPRequestHandler):
def do_HEAD(self):
self.send_response(200)
self.send_header('Content-Type', 'text/html')
self.end_headers()
def do_GET(self):
self.do_HEAD()
self.wfile.write("hej".encode("utf-8"))
if __name__ == '__main__':
http_server = HTTPServer((host_name, host_port), MyServer)
print("Server Starts – %s:%s" % (host_name, host_port))
try:
http_server.serve_forever()
except KeyboardInterrupt:
http_server.server_close()

Start it:

python3 alert-step1-server.py
view raw alert-step1-start.sh hosted with ❤ by GitHub

I opened that page, (192.168.0.193:8000), and I could see “hej”, time to go further.

Step 2. Connect World

Next step was to open up this “web app” for the world, to make it accessible from outside my local network. ngrok is the best solution for that. I followed that guide to install ngrok on my Raspberry Pi Zero W.

The installation process was pretty straight forward, for the record I tried to install ngrok as a snap, it did not work.

cd ~
wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-arm.tgz
tar -xvzf ngrok-stable-linux-arm.tgz
view raw alert-step2-ngrok.sh hosted with ❤ by GitHub

I also fetched the authtoken and registered it locally

Then I started the ngrok tunnel:

And my web app went online:

Step 3. Harness the lights

Now to the core of this hobby solution: controlling Trådfri lights.

I installed, configured and built the libcoap client, as described in the blog post I already mentioned:

But I also installed git, because my Raspberry Pi OS installation didn’t have it.

sudo apt-get install build-essential autoconf automake libtool git -y
git clone –recursive https://github.com/obgm/libcoap.git
cd libcoap
git checkout dtls
git submodule update –init –recursive
./autogen.sh
./configure –disable-documentation –disable-shared
make
sudo make install

Next, I found the IP Address and the security code of the IKEA Trådfri Gateway, using my router:

Then I created a new preshared key (that’s the news I mentioned above). With just the security code, you will get 4.01 “Unauthorized” when you try to control the lights, as described:

# -k = Security Code, that you can find on the back of the gateway
# 9090: xxx, your new client identity, you decide, in my case TOLLERASP0
# coaps: the ip address is the one of your gateway
coap-client -m post -u "Client_identity" -k "OHsfKxV0UaJu81" -e '{"9090":"TOLLERASP0"}' "coaps://192.168.0.120:5684/15011/9063"

I got the pre-shared key that I saved for later use:

With this information you can harness the IKEA lights:

# off
coap-client -m put -u "TOLLERASP0" -k "{presharedkey}" -e '{ "3311": [{ "5850": 0 }] }' "coaps://192.168.0.120:5684/15001/65537"
# on
coap-client -m put -u "TOLLERASP0" -k "{presharedkey}" -e '{ "3311": [{ "5850": 1 }] }' "coaps://192.168.0.120:5684/15001/65537"
view raw alert-step3-onoff.sh hosted with ❤ by GitHub

5850:0 is off, 5850:1 is on. Easy-peasy, right?

Want to know how to control the brightness, the colors etc, just check this documentation (already mentioned):

Step 4. Put everything together

When I knew I could have a simple webhook service, locally (step 1) and on the WWW (step 2), and that I could control the smart light I’ve got from IKEA using code running on my raspberry pi, then connecting everything was easy. I created a repo for that and you can see that it is a very simple one:

The main part is in the server.py. When it gets invoked, it calls the flash function. It uses os.system to call the libcoap-client and time.sleep for delay parts needed in the flash action. The configuration is parsed using configparser and the server is a simple http.server.

In the end I registered the ngrok endpoint in my Azure Alert Rule Action Group:

Then I triggered my test logic app that failed reliably 🙂

After 1-2 minutes my smart light started flash:

Success 🎯🎯🎯🎯

Words of caution and Tips

Security

http.server does not provide the right level of security, it’s most for prototyping. For this tiny hobby project I have, it’s exactly what I need. Don’t use it as it is for production.

Treat the security code your preshared key appropriately, you don’t want to be hacked.

Flashing lights reacting to alerts is cool, but think about the work-life balance. Don’t have it in your bedroom 😎.

Inspect ngrok from other computer

By default the ngrok web inspect interface is only available from localhost (127.0.0.1), make it available across your network by configuring ngrok:

# ip address of your raspberry pi
echo "web_addr: 192.168.0.193:4040" >> ~/.ngrok2/ngrok.yml

Reserve your local ip addresses

The router can assign new ip addresses to your devices. Reserve the ip addresses of your raspberry pi and your IKEA Trådfri Gateway. It will make your life easier.

Start ngrok closer to you and in the background

EU is closer to me, but also running the background is nice when you only have one terminal:

# -region eu
# >/dev/null & for running in the background
~/ngrok http 192.168.0.193:8000 -region eu > /dev/null &

Replay ngrok calls

This is a game changer: rather than wait for an alert to be triggered, you can just Replay it over and over again while you mickle-muckle your python code locally.

Keep running your server after logout

You just need to to have “nohup” when you start your server, ngrok has already what’s needed: nohup python3 server.py. With that the server will run even when you log out or, your ssh connection disappears.

Next steps

I’d like to end this post also by saying: The world is your lobster. Try out the flashing lights on Azure Alerts, or why not to replace Azure Alerts with Exoprise Alarms, or some triggers in Power Automate, perhaps, when a new site has popped up 🙂 Or maybe you want to elaborate the flashing behaviour, why not to use Morse code to send a message? Or maybe color-code the different types of alarms/alerts. Once again, the world is your lobster 🦞(or oyster 🦪, well whatever) .

Daniel Chronlund Cloud Tech Blog

News, tips and thoughts for Microsoft cloud fans

Вула Чăвашла

VulaCV - Чăвашла вулаттаракан сайт

Discovering SharePoint

And going crazy doing it

Bram de Jager - Architect, Speaker, Author

Microsoft 365, SharePoint and Azure

SharePoint Dragons

Nikander & Margriet on SharePoint

Mai Omar Desouki

PFE @ Microsoft

Cameron Dwyer

Office 365, SharePoint, Azure, OnePlace Solutions & Life's Other Little Wonders

paul.tavares

Me and My doings!

Share SharePoint Points !

By Mohit Vashishtha

Jimmy Janlén "Den Scrummande Konsulten"

Erfarenheter, synpunkter och raljerande om Scrum från Jimmy Janlén

Aryan Nava

DevOps, Cloud and Blockchain Consultant

SPJoel

SharePoint for everyone

SharePointRyan

Ryan Dennis is a SharePoint Solution Architect with a passion for SharePoint and PowerShell

SharePoint 2020

The Vision for a Future of Clarity

Aharoni in Unicode

Treacle tarts for great justice

... And All That JS

JavaScript, Web Apps and SharePoint

blksthl

Mostly what I know and share about...