Steady Nerves! Powershell for Peace of Mind

Steady Nerves! Powershell for Peace of Mind

Powershell Photo

Can Powershell help me with mundane tasks?  Absolutely!  I work for an enterprise financial entity that supports thousands of clients.  Currently I am working on an internal web application that is critical for our customer support representatives (CSRs) to service our clients.  This application has been up and running for a few years now, but there have been many times when I receive that dreaded IM from one of our business partners:

“Hey Rick, is something wrong with {web application}?  It seems like its not working…”

What?!  Yikes, STOP everything and check!

Server 1 – up and alive, Server 2, up.  Server 3, not responding!  Argh, why didn’t operations find this issue?

As a developer or IT professional, I am sure you can relate to this dreaded exchange.  This is the type of disruption that can quickly derail a work day, kill your “flow”, and make you question your line of work.  Being in a line of work that supports some long running legacy applications, failures only have to happen 1 or 2 times before you become weary of ANY messaging around your systems.

If it happens enough to the applications your boss is responsible for, guess what, you better start knowing your servers are down before your users do!  Granted, we do have a great operations support team that should catch these failures, but sometimes their health checks are not sophisticated enough to catch the nuances that we may be aware of.  This year, our leadership wants extra focus on stability, on par with quick responses in the face of an issue.

I found myself bumping into a frustrating routine.  Start work, check the site, is it up?  About lunchtime, check the site, API sources returning data?  Done for the day, check one last time.  Its now time to get this thing automated so I don’t have to check the site, but will still have the peace of mind I need!

Enter my new good friend, Windows Powershell!  Our enterprise is mainly windows based, and I had heard rumblings of our operations team being required to take some Powershell classes.  After some really quick research, guess what?  Powershell can make HTTP requests and API calls (System.Net.WebRequest), just like my browser is doing on these daily check points!  In addition, it can also email me if it finds an issue…perfect, time to dig in.  I figure, spend an hour or two writing some automation scripts and THEY will update me on the state of the application!  Chaos to clarity…

Elementary Powershell

Below is a sample of a quick and rather simple web server health check script.  The script sends a web request to each URI in an array ($uri_array) of URI’s.  An email is generated and sent with the results of the web request.  I also added a priority indicator on the message to set a high priority if we get no response, which gives a good visual indicator in your inbox.

Once you get this script running, you can use your Windows Task Scheduler to create a schedule and run it as you see fit.

$message = "<b>Web Server Health Check</b>";
$msg_priority = "Low";
foreach ($element in $uri_array) {  #$uri_array is an array of strings, the strings are intended to be URI's
   $d = $element.StartsWith("http");
   if($d)
   {  
      $exmessage = $null;
      try
      {
         $proxyURL = ([System.Net.WebRequest]::GetSystemWebproxy()).GetProxy($element);
         $res = Invoke-WebRequest -URI $element -Method Get -Credential $theCred -Proxy $proxyURL -ProxyUseDefaultCredentials;
      }
      catch
      {
         $exmessage = $_.Exception.Message;
      }
      finally {
         if(!$exmessage) {
            $message += $element + " " + $exmessage + ": Status Code: " + $res.StatusCode + ", " + $res.StatusDescription + "";
         }
         else
         {
            $message += $element + " <span style="color: red;"><b>" + $exmessage + "</b></span>";
            $msg_priority = "High";
         }
      }
   }
   else
   {
      $message += "" + $element + "";
   }
};
Send-ToEmail -body $message -msg_priority $msg_priority;

Extend It

Powershell scripts are quite powerful, and the simple example above is capable of being extended to do much more.  For my needs, I went with 3 scripts, each run on different schedules, and each emailing different groups of people.  The first utilizes code very similar to the above sample, but the other 2 dig a bit deeper.

  • Script #2 checks the health of many of our partner API’s.  If one of these API’s is failing, our application may not appear to work properly.  Operations might alert us if our servers are having a problem, but will they if our partners are having problems?  It is not likely.
  • Script #3 utilizes our own application’s service layer and inspects for specific data conditions. There is a good chance that something is awry when the specific conditions are not found.

Next Steps

Think about your own applications and how they have failed in the past.  What kind of monitoring would bring you some piece of mind?  For me, it was really easy, having had supported the application for a number of years.  Likely, when we design from scratch, we take this experience and incorporate it into our new applications, but going back and refactoring a legacy application is not always a possibility or even practical.  Dig deeper into powershell, or bash scripts,  and help yourself out!  I encourage you, use the tools that are available to you and start to automate the mundane!