Powershell and Uptimerobot

Uptimerobot can be quite tedious when you need to update many monitors at once.
For example say you bought the license for Uptimerobot and now you want to change 50 monitors to check status every 60 seconds instead of the free modes 300 second interval. 

Luckily Uptimerobot have a great API that we can work with powershell against.

getmonitors.ps1 is method to export all monitors to local file. For easier import to another account, or just a small backup just in case. All monitors on account is exported to file note.txt. Using https://api.uptimerobot.com/v2/newMonitor/getMonitors

addmonitors.ps1 is example of how to add new monitor using https://api.uptimerobot.com/v2/newMonitor

editMonitors.ps1  is method for updating all monitors in account to use 60seconds interval.
Using https://api.uptimerobot.com/v2/editMonitor

Sharepoint 2019 and mysites

 I had some problems with the old familiar "Working on it" for mysites in Sharepoint 2019. It was difficult to find information about this, Im guessing the installation base for world wide on-premise installations is dropping.

Key information when troubleshooting

- On User Profile, attribute "Personal Site Capabilities" is set after user visits mysite for the first time. Default behaviour seems to be setting this to 4. Meaning only storage.
- To reset user mysite, remove the mysite and attributes for mysite and Personal Site Capabilites are automaticly removed from User Profile
- To check ULSViewer for Personal Site Capabilites settings, filter for eventid =  aj1lz
- FeedServiceIdentifier is based on Microfeeds-list on user mysite, without site feature MySiteSocialDeployment no microfeeds is created.
- User Profile Service Application_ActivityFeedJob is running at 10min interval.
- To track mysite creating in ULSViewer, filter on Category = Personal Site Instantation

To solve the problem

After verifying all the user profile service properties and mysite settings I ended up with what might be a bug or might be design. The MySiteSocialDeployment seems to be missing on the default mysite template. So using the solution from Trevors site below i built some scripts for easier deployment in case Microsoft updates onet.xml i future updates.

So basicly first update onet.xml configuration site with ID 10 to include the missing MySiteSocialDeployment-feature
Then deploy the updated file to all members with copyupdatefile.ps1
Finally update sitemaster with newsitemaster.ps1
Remove previously created mysites to trigger mysite-creation with correct features.

References:
Details about PersonalSiteCapabilites
the fix from Trevor
Good details about Fast Site Creating
Another site with same fix
German site using the same fix

Handle multiple versions of powershellPnp with Sharepoint

 This is a nice way to handle accessing different versions of PNPPowershell. 
I use this when switching between onpremise environments of different version and online.

Note that env-variables can't be used until session, like ISE is restarted. Otherwise can variable be accessed immediatly using 
[System.Environment]::GetEnvironmentVariable('SharePointPnPPowerShell2019','machine')

setupEnv.ps1 downloads named modules in $array to $saveFolder. 
Then locates psd1-file and adds this as a enviroment variable with name in arrayobject. 
To use the module access env-variable with $env:name as showed in importModule_Example.ps1

References:
https://www.erwinmcm.com/running-the-various-versions-of-pnp-powershell-side-by-side/

Converting csv to excel

Using the useful powershell module importExcel

I could use this to import folder with csv-files to a excelfil for more easy viewing for common users.

Use commented install-module line for first run.

References:
https://github.com/dfinke/ImportExcel

Using powershell with UptimeRobots API

I got a chance to test Uptimerobots API with Powershell and I used it with a csv-list of urls with keywords to update my uptimerobot account.
Pretty good stuff where you also can output all your monitors with uptimelogs directly to PowerBI or equivalent.

Code:


To get UptimeRobot status from PowerBI the following queries can be used:

Example 1 - Simple query

let

   body = Text.ToBinary("api_key=#myAPIKey#&format=json&logs=1"),

   url= "https://api.uptimerobot.com/v2/getMonitors",

   Options = [

   Headers=[#"Content-type"="application/x-www-form-urlencoded", #"cache-control"="no-cache"],

   Content=body

   ],

   result = Web.Contents(url, Options)

in

    result


Example 2 - Query using records

let

   content= [

      #"api_key"="#myAPIKey#",

      #"format" = "json",

      #"logs" = "1"

   ],

   query = Text.ToBinary(Uri.BuildQueryString(content)),

   Url= "https://api.uptimerobot.com/v2/getMonitors",

   Options = [

   Headers=[

      #"Content-type"="application/x-www-form-urlencoded",

      #"cache-control"="no-cache"],

      Content=query

   ],

   result = Web.Contents(Url, Options)

in

    result

 

Example 3 - Query using json
 

let

   content = "{

  ""api_key"": ""#myAPIKey#"",

  ""format"": ""json"",

      ""logs"": ""1""        

    }",

   query = Text.ToBinary(Uri.BuildQueryString(Json.Document(content))),

   Url = "https://api.uptimerobot.com/v2/getMonitors",

   Options = [

   Headers=[

      #"Content-type"="application/x-www-form-urlencoded",

      #"cache-control"="no-cache"],

      Content=query

   ],

   result = Web.Contents(Url, Options)  

in

    result

SelfService sitecollections in Sharepoint 2019

With Sharepoint 2019 we now have the ability to allow users to create site collections outside their own mysite themselves. This allows for less administration.
The drawback of this solution is that you can only create managed path sites on primary Alternate Access for webapplication. 
Microsofts documentation leaves some holes in regards to how to activate selfservicing functions with powershell in Sharepoint 2019 so I digged for the related values and found settings below. Most are obvious, one wasn't.


Script below




References:
https://docs.microsoft.com/en-us/sharepoint/sites/configure-self-service-site-creation-in-sharepoint-server-2019

Blobcache issues

I had some weird problem with images in Sharepoint 2013.
Issues were for example

• Updated images didn't work, old images still showed for clients. 
• Change imageproperties didn't work either
• Changing website logo didn't work

BlobCache quickly became as suspect, so we tried to flush the blobcache the proper way with flushBlobCache.ps1 but the folders in the blobcache-directory didn't get recreated.
Expected behaviour is when flushblob has run on farm, the folder under siteID in blob-directory will get a new created date and new name.
For me, they just left the old folder without change on certain farm-members.

After searching for a official solution I landed on my own fix which was as follows

1. Redirect trafik from webfrontend or do this during servicewindow
2. Update url and run setBlobDisabled.ps1 (or disable blob manually in web.config)
3. This will cause a reset to Application Pool and blobfolder will stop updating change.bin file in blob temp-directory
4. Rename or delete folder with SiteID, for example E:\Blob\14\354006434 to 354006434_old
5.  Update url and run setBlobEnabled.ps1
6. Problem solved.

 

Checking Dotnet version on server

I needed to check running dotnet version on a few servers to verify support for TLS 1.2
I used script below, which I though was easy to update and using hash-tables in an effective way.

Using powershell to check company information

I had a long list of companynames but no homepage adress.
Using the swedish yellow pages service Eniro I could use powershell to get the information.
This script only gets the first result with homepage attribut added, but depending on what you put the results may vary.

Checkup list of websites

To determine a range of websites dns, registrar and IPOwner and what technology is used the following script can be used.
Note that whois64.exe referred to in script needs to be downloaded seperatly from Sysinternals/Microsoft



References:
https://docs.microsoft.com/en-us/sysinternals/downloads/whois

enumerate columns from subsites

This case is something which I've came back to a few times.
Customer wants to output columns from certain content types from a number of sites.
Its always a hassle to get the correct names of columns so I've built a method to output the values more dynamically.

1. First getFields.ps1 reads columns containg values from a list item, outputs to gridview, chosen values gets saved to customfields.csv
2. Second getSitecontenttypefields.ps1 enumerates all sites on chosen destination and loops all items of specified content types and outputs chosen fields from customfields.ps1 to an object which is saved to a csv-file.

CertAuto

For a developer environment I had a few certificates that needed automatic renewal at certain intervals. The script below uses the PSPKI module to check if certificates are expiring on IIS-Site in X days and if so, renews the agains a local certification authority.
The PSPKI -module is really god for sending and approving certification requests  via Powershell when Microsofts own tools leaves room for improvement.

This is really only for a testenviroment and offline-enviroments.
Folder structure needs to created to look below




Certini contains the template for the certificate like below for example
 [Version]
Signature = "$Windows NT$"
[NewRequest]
Subject = "C=US,S=CA,L=OHIO,O=Fabrikam,OU=IT,CN=ajax.aspnetcdn.com"
Exportable = TRUE
KeyLength = 4096
KeySpec = 1
KeyUsage = 0xa0
MachineKeySet = True
ProviderName = "Microsoft RSA SChannel Cryptographic Provider"
ProviderType = 12
Silent = True
SMIME = False
RequestType = PKCS10
FriendlyName = "ajax.aspnetcdn.com"



References:
https://www.sysadmins.lv/projects/pspki/default.aspx  - PSPKI Module

Export IIS bindings

I find the cmdlet for exporting cmdlets a bit lacking so I built my own export function for fun and profit.
The function in the toolsfile collects all info and return a populated object that can be used to export list to file or do something else with.

Digging through a lot of files

I have a scenario where I need to scan a lot of xmlfiles from elmah-logs in the size of 300 000 files.

Powershell was a fun project for this but couldn't really do the job due to performance issues.
When reading 300 000 files using powershell like example below, the script ran for 368 minutes.


I tried running parallell jobs and using dotnet to read files instead but nothing could complete with using MS Logparser.
So enter Log Parser Studio.
This neat tool managed to comb through 300 000 files in 47 minutes instead!
It is a bit tricky to formulate the queries however. Heres an example of getting elmah logs where a variable named HTTP_REFERER contains a key value

select * FROM '[LOGFILEPATH]' where  string like '%http://www.mycompany.com/subsite%' and name like 'HTTP_REFERER'

So in conclusion for same set of 300 000 files
Powershell took 368 minutes
Log Parser Studio took 47 minutes

References:

https://www.microsoft.com/en-us/download/details.aspx?id=24659  - Log Parser 

https://gallery.technet.microsoft.com/office/Log-Parser-Studio-cd458765

https://blogs.technet.microsoft.com/exchange/2013/06/17/log-parser-studio-2-0-is-now-available/

Using hashtables to combine values

For a job I needed a good way of combining a primary value with optional subvalues in a xml-file.
Below is an example of how to extract attributes from a xmlelement into a hashtable and then join two hashtables into one. I couldn't find a method for extracting elements on the web, folks usually goes with subnodes instead of attributes. I'm partial to attributes and had to get creative, but the solution was quite simple. The Attributes from the element only show the actual elements from the xmlfile. Its not obvious when browsing the object that -name property can be lifted, but here it's used for creating a new hashtable-row



Samplescript for proof of concept


This eventually led to createwebsFromStructure.ps1 that uses powershell splatting to build commands with the resulting hashtable.

References:
https://powershell.org/2013/01/23/join-powershell-hash-tables/ - source for join-hashtable function
https://technet.microsoft.com/en-us/library/gg675931.aspx - source for details on splatting

Powershell and jobs

Powershell jobs are a bit of blunt instrument.

Here's two options for sending inputs for them
Scenario 1: Sending a string with all of our values and splitting them to an array when inside the scriptblock.


Scenario2: Sending an object as input. Handy for configs and saves the need to define variablenames.

powershell assisting with mails

ExtractAttachedMails.ps1 is used for extracted alla attached mails in a outlook .msg-file to specified folder.
CatalogueMails.ps1 is used for indexing the mails to more easily filter on timestamps and senders.

This came in handy when a happy user sent me 45 attached mails with the same subjectline so the extractionprocess also throws in an indexnumber to avoid namingconflicts.

Remove iislogs

Built a nice script to remove older files from IISLogs.
It has two retentionmodes. Either clear based on number of last changed files or for specific days.

Installing workflow manager on sharepoint 2016

Scenario:
Sharepoint 2016 needs workflow supports since 2010 workflows now seems dead.
This needs to be done offline

Overview of steps

1. Download all programs from internet-enabled client with getOfflinePackages
2. Install packages on server in correct order
3.  Configure workflow manager
4.  Register workflow service in sharepoint

Step 1 - Download necessary files for offlineinstallationen

On client with webaccess

1. Install WebPlattformInstaller
2. Download packages with getOfflinePackages.ps1

Step 2 - Install packages on server in correct order

1. Install packages with InstallWFM.ps1 to get correct order
   

Step 3 - Configure Workflow Manager

1. Update wfmconfig.xml with credentials and databasename
2. Configure with configureWFM.ps1
   
3. Update siteaddress and run registerWFOnsite
   

References:
https://www.helloitsliam.com/2014/11/06/sharepoint-2013-workflow-manager-woes/ - source for the order of installation
http://blog.robgarrett.com/2014/05/12/install-workflow-manager-with-powershell/ - original script for configuring workflow manager

https://blogs.msdn.microsoft.com/laleh/2014/09/03/sharepoint-2013-workflow-troublehsooting/ - troubleshooting workflow manager
https://social.technet.microsoft.com/wiki/contents/articles/34407.sharepoint-2016-step-by-step-installation-of-workflow-manager.aspx - another installationguide
https://knowledge.zomers.eu/SharePoint/Pages/How-to-install-and-configure-Workflow-Manager-on-Windows-2012-R2.aspx - more updated information on how to configure worklow manager
https://docs.microsoft.com/en-us/iis/install/web-platform-installer/web-platform-installer-direct-downloads - webplattforminstaller

Sharepoint and alternate languages

So what happens when a column is created and then changed.
And then, possible, changed again in an alternate language?

First created, staticName is set and the cultured name

When names is changed, only the culturename is changed

When and alternate language is added, and copy of the culturename appears.

When name is changed again to doktor2, only the current language version is changed. Swedish 1053.

Now we have 3 version to manage.
When we choose overwrite language specific version the values dont change.

When we now change the column to Doktor3, only the 1053 version change when we’re using the alternate languagebrowser.

But if we change using the default language (1033) to doktor4, all local variants are overwritten

But then a local editor is at it again, setting doktor5 in swedish and the values are off.

But what happens if we now remove local variants? Then we're down to only the default columns

Changing from swedish gui to doktor6 and the english 1033 is changed.

But what happens if language is enabled again? Well, the old value is still there…

The lesson here?
Stay away from alternate languages!
And also a fun script for checking all the versions of a fields on a list.