Wednesday, September 27, 2017

Patching Sharepoint with the homebrew-method

Patchning Sharepoint is not something to taken lightly.
It's a high risk not to patch but a guaranteed risk to apply the patches.

Last year I had a difficult experience when I patched a semilarge(7 members) farm for the first time with Cumulative Updates.
This had 5 webapplications and +200 databases. And it took some time. Like 30 hours, most of the time being lost when running psconfig on each host.
Supposedly, sharepoint should only upgrade the databases on the first member run, but something still takes a lot of time, even its only to check if upgrade is required.

So a more effective method was needed.
I looked into the impressive SPPatchify-project but found the results to be too inconsistent. Sometimes the script ran through it all and said everything was green, when actually two servers had failed psconfigs and a lot of other issues giving half-finished state.
So I deviced my own hybrid method using the best parts of the sppatchify-idea and russmax exe-installationscript.

The method took the installationwindow down to 4 hours in total, where only 2 hours was complete downtime.
Its a bit more manual than others, but therefore also gives more control.

The steps
Preperations

  1. Take whole farm offline and take snapshots, or use regular backups. Whichever method you trust most.
  2. Backup All databases
Installation
  1. Stop all sharepointrelated-services
    1.  For AppServers 1. App-Prepatching
    2.  For Frontends  1. WFE-PrePatching
  2. Run exe on all members
  3. Start all services 
    1. For AppServers 2. App-PostPatching
    2. For frontends  2. WFE-PostPatching
  4. Export all databasinformation to file.
    1. ExportAllDbs.ps1
  5. Dismount all databases (after making sure you have all correct info on file from previousStep) This will take everything offline.
    1. DismountAllDbs.ps1
  6. Run psconfig on all members starting with AppServers
    1. RunPSConfig.ps1 on each member 
  7. Mount all databases
    1. MountAllDbs.ps1  - Update the filename to your exported databases
  8. Upgrade databases in parallel. While each db is upgrading website on it will be unavailable. If you're sql supports it you can mitigate this using -usesnapshot  How many concurrent jobs is depending on hardware. I found the sweetspot to be 4 concurrent jobs, after that it started taking longer time that running them in sequence. But this might vary. 
    1. UpgradeDatabasesFaster.ps1
  9. Check on warnings in Health Analyzer, make repairs if needed. 
  10. Make sure User Profile Synchronizer service is running (if your using FIM-sync)


Below all relevant scripts. Use at your own risk.



References:
https://blogs.msdn.microsoft.com/russmax/2013/04/01/why-sharepoint-2013-cumulative-update-takes-5-hours-to-install/ - Russmax excellent script for halting services before running exe. Saving hours
http://www.spjeff.com/2016/05/16/sppatchify-cu-patch-entire-farm-from-one-script/ - SpJeffs
https://blogs.technet.microsoft.com/stefan_gossner/2015/08/20/why-i-prefer-psconfigui-exe-over-psconfig-exe/ - Excellent resource for patching

Powershell and Uptimerobot

Uptimerobot can be quite tedious when you need to update many monitors at once. For example say you bought the license for Uptimerobot and n...