Scouter is a simple containerized API that allows you to perform various troubleshooting utilities remotely.

It runs common Linux based troubleshooting utilities such as ping, traceroute and a few others that are documented further on. These utilities support multiple options, so creating specific types of tests is very easy.

It deploys easily to any container infrastructure.

The most powerful aspect of Scouter is its automation and monitoring potential.

These instructions cover building and using the docker container.

You'll need docker installed in order to run this container.

• Windows
• macOS
• Linux

You'll also need to sign up for a GeoLite2 account and generate a license key. License keys can be generated free of charge and are used to authorize access to download MaxMind databases.

The GeoLite2-ASN MaxMind database is used to translate source IP addresses in both the traceroute and dns_traceroute utilities into their associated Autonomous System Numbers.

Run the following to build the Scouter container from source.

git clone https://github.com/stackpath/scouter.git
docker build -t scouter .

If you'd like to skip building the container from source then pull the latest version of Scouter from Docker Hub:

Coming soon!

• SCOUTER_API_SECRET - Specify the shared API secret to be used to authenticate every API call.
• MMDB_LICENSE_KEY - Specify one of your valid GeoLite2 license keys to be used when pulling the latest GeoLite2-ASN.mmdb.

• SCOUTER_MAX_TEST_COUNT - Specify the maximum number of individual tests in a single test payload. Defaults to 10.
• SCOUTER_MAX_PROCESS_COUNT - Specify the maximum number of parallel processes to be used in test execution. Defaults to 10.
• API_PORT - Specify the port that Nginx will be listening on. Defaults to 8000.
• UWSGI_WORKERS - Specify the number of Uwsgi worker processes to use. Defaults to 3.
• UWSGI_CACHE_ITEMS - Specify the maximum number of Uwsgi cache items. Defaults to 100.
• UWSGI_CACHE_BLOCKSIZE - Specify the maximum Uwsgi cache size. Defaults to 1000000.
• BUP_PROXY_TTL - Specify the maximum amount of time that a BUP proxy is allowed to live. Defaults to 300 seconds.
• BUP_PROXY_PORT_RANGE - Specify the range of ports reserved for BUP proxies. Also serves a pseudo rate limiter. Defaults to 9001-9005.

First, start the Scouter container:

$ docker run --rm --name "scouter" -e "SCOUTER_API_SECRET=secret" -p 8000:8000 scouter

Once started, ensure that everything is working correctly by checking the /api/v1.0/status endpoint:

$ curl -X GET -H "Authorization: secret" "http://localhost:8000/api/v1.0/status" | jq

This produces a response that contains internal information about the Scouter service.

{
  "total_requests": 0,
  "worker_status": [
    {
      "avg_rt": 0,
      "delta_requests": 0,
      "exceptions": 0,
      "id": 1,
      "last_spawn": 1569856797,
      "pid": 41,
      "requests": 0,
      "respawn_count": 0,
      "rss": 0,
      "running_time": 0,
      "signals": 0,
      "status": "busy",
      "tx": 0,
      "vsz": 0
    },
    {
      "avg_rt": 0,
      "delta_requests": 0,
      "exceptions": 0,
      "id": 2,
      "last_spawn": 1569856797,
      "pid": 42,
      "requests": 0,
      "respawn_count": 0,
      "rss": 0,
      "running_time": 0,
      "signals": 0,
      "status": "idle",
      "tx": 0,
      "vsz": 0
    },
    {
      "avg_rt": 0,
      "delta_requests": 0,
      "exceptions": 0,
      "id": 3,
      "last_spawn": 1569856797,
      "pid": 43,
      "requests": 0,
      "respawn_count": 0,
      "rss": 0,
      "running_time": 0,
      "signals": 0,
      "status": "idle",
      "tx": 0,
      "vsz": 0
    }
  ]
}

Once you've confirmed the API is up and running you can execute tests. Scouter supports the ability to send multiple tests in a single payload. These tests are executed in parallel, the number of which is configured with the SCOUTER_MAX_THREAD_COUNT environment variable.

Here's an example of how to create a new test to perform both an http_request and a dns_lookup:

First start by creating a test via POST:

$ curl -X POST \
 -H "Authorization: secret" \
 -H "Content-Type: application/json" \
 -d '{"http_request": [{"url": "example.com"}], "dns_lookup": [{"qname": "example.com"}]}' \
 "http://localhost:8000/api/v1.0/tests" | jq

This command returns a receipt that's used to retrieve test results:

{
  "receipt": "78e473ed3397c8fb02b9c9c9b21a9ae1"
}

Pass the test ID via the receipt parameter in a subsequent API call:

$ curl -X GET \
 -H "Authorization: secret" \
 "http://localhost:8000/api/v1.0/tests?receipt=78e473ed3397c8fb02b9c9c9b21a9ae1" | jq

The response contains the test result's details:

{
  "is_running": false,
  "receipt": "c37f83382242675804820562d2a44210",
  "results": {
    "dns_lookup": [
      {
        "failed": false,
        "id": "38e618",
        "message": null,
        "result": {
          "answer": [
            {
              "rclass": "IN",
              "rdata": "93.184.216.34",
              "rdlen": null,
              "rrname": "example.com.",
              "ttl": 86400,
              "type": "A"
            }
          ],
          "elapsed_time": 0.06239771842956543,
          "failed": false,
          "ns": "169.254.169.254",
          "question": {
            "qclass": "IN",
            "qname": "example.com.",
            "qtype": "A"
          },
          "rcode": "ok",
          "timeout_count": 0
        }
      }
    ],
    "http_request": [
      {
        "failed": false,
        "id": "4051b9",
        "message": null,
        "result": {
          "comment": null,
          "failed": false,
          "headers": {
            "accept-ranges": "bytes",
            "age": "325974",
            "cache-control": "max-age=604800",
            "content-encoding": "gzip",
            "content-length": "648",
            "content-type": "text/html; charset=UTF-8",
            "date": "Wed, 05 Feb 2020 19:57:20 GMT",
            "etag": "\"3147526947\"",
            "expires": "Wed, 12 Feb 2020 19:57:20 GMT",
            "last-modified": "Thu, 17 Oct 2019 07:18:26 GMT",
            "server": "ECS (oxr/830C)",
            "x-cache": "HIT"
          },
          "method": "HEAD",
          "reason": "OK",
          "speed_download": 0,
          "status": 200,
          "time_appconnect": "0.000",
          "time_connect": "0.018",
          "time_namelookup": "0.016",
          "time_starttransfer": "0.020",
          "time_total": "0.023",
          "url": "example.com",
          "version": "1.1"
        }
      }
    ]
  }
}

There is a lot of data from these two tests. Use a tool like jq to manually parse the output.

Test results can also be deleted. Results automatically expire in 10 minutes if not deleted manually:

$ curl -X DELETE \
 -H "Authorization: secret" \
 "http://localhost:8000/api/v1.0/tests?receipt=78e473ed3397c8fb02b9c9c9b21a9ae1" | jq

Test deletions should produce a success message:

{
  "message": "Provided 'receipt' has been successfully deleted."
}

• Docker
• Python 3
• Nginx
• Circus
• Flask
• Scapy
• PycURL
• Selenium
• BrowserUp Proxy

• @aaroncouch - Initial work

This project is written in Python Version 3.7.3 and uses the PEP 8 style guide. The use of a code linter such as Pylint is highly recommended to keep the code as consistent as possible. All docstrings and comments follow the Chromium Python Style Guidelines.