Add a Slack alert to our alert channel
Add a Slack alert to our alert channel
Overview
This document will serve as a guide on how to add an alert to our Slack alerts channel below:
- #operations-engineering-alerts
There are three types of alerts at time of writing:
- RSS Feed - third party tools usually offer an RSS feed we can subscribe to which will alert us if they are having issues
- Pingdom - end point checking, Pingdom will simply ping an end point every X seconds and report if it responds with bad return code
- GitHub Actions - a snippet of code we add at the end of certain GitHub Action files to report to us when they fail.
RSS Feeds
Requirements
- The RSS feed - this can be retrieved from the third party tools website, an example is the Get Updates button on this page
- Access to the Slack RSS application from MoJ (https://mojdt.slack.com/services/B642DDZG9)
Adding a feed
- Navigate to the bottom of the Slack RSS application
- Copy the feed address (Atom/RSS both work) into the "Feed URL" field
- Select #operations-engineering-alerts in the "Post to Channel" field
- Select "Subscribe to feed"
Once this has been done, every time the selected feed reports an issue, it will let us know on Slack.
Pingdom
Requirements
- Access to the Operations Engineering tools account (it's in 1Password)
- The end point you wish monitor
Adding a Pingdom alert
- Navigate to the Uptime section on Pingdom
- Select "Add new" at the top left of the page
- Enter a relevant name for the "Name of check" field - (We usually go with the application name)
- Enter the end point you wish to monitor to the "URL/IP" field
- Ensure "http/https" is selected correctly
- Select "Europe" for the "Test from" field
- Add "operationsengineering" to the "Tags" field
- Add any other relevant tags to the "Tags" field - example "decomission" would be relevant for Operations Engineering Decomission Pages end points
- Select the option you want for "Check importance" - we usually go with Low
- Untick "MoJ Digital" on "Who to alert?"
- Tick "Operations Engineering Alerts" in the "Connect Integrations" section
- Ensure the test is working with the "Test check" button
- Select "Create check"
Once this has been done, every time the selected end point is down, it will let us know on Slack.
GitHub Actions
Requirements
- SLACK_WEBHOOK_URL secret configured on target repository (secure notes in LastPass)
- Access to push to target repository
Adding GitHub Actions alert
-
Clone the repository containing the GitHub action file you want to add the alert to
-
Open the GitHub action/workflow file
-
As the last step of the action, add the following code:
- name: Report failure to Slack if: always() uses: ravsamhq/notify-slack-action@v1 with: status: ${{ job.status }} notify_when: 'failure' notification_title: 'Failed GitHub Action Run' env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} -
An example of a complete workflow with this step is here
-
Pull request this change into
main
Once this has been done, every time the selected GitHub Action fails, it will let us know on Slack.
Was this page useful?