pylot

Pylot is a free open source tool for testing performance and scalability of web services. It runs HTTP load tests, which are useful for capacity planning, benchmarking, analysis, and system tuning.
Pylot generates concurrent load (HTTP Requests), verifies server responses, and produces reports with metrics. Tests suites are executed and monitored from a GUI or shell/console.

Getting Started Guide

Platforms:

Console and Blocking Mode

Console and Blocking modes run on all platforms where Python 3.6+ can be installed. Tested on Windows XP, Vista, Ubuntu 8.04/8.10, Eee PC, Mac OS.

GUI Mode

Pylot GUI will run on all platforms that support Python and wxWidgets. The GUI has mostly been developed and tested on Windows, but looks decent on Linux and Mac. The application code is pure Python and uses a cross-platform toolkit.

Installing Pylot

Step 1: Download and unzip the latest Pylot release Get the latest release here: Download Pylot

Step 2: Install Python 3.5+ Get installer from here: http://www.python.org/download

Step 3: Install wxPython (optional - used for GUI mode) Get the installer from here: http://www.wxpython.org/download.php

Step 4: Install NumPy (optional - used for report graphs) Get the installer from here: http://sourceforge.net/projects/numpy

Step 5: Install Matplotlib (optional - used for report graphs) Get the installer from here: http://sourceforge.net/projects/matplotlib

Step 6: Install tenjin Get the installer from here: https://pypi.org/project/Tenjin/

pip install Tenjin

Step 7: Run Pylot

GUI Mode:

python run.py -g

Web Mode:

python run.py -w 0.0.0.0:8080

ui/web/config_web_prod.py cusomte config for ldap etc.

Console and Blocking Mode

Command Line Options:

usage: run.py [options] args
  -a, --agents=NUM_AGENTS     :  number of agents
  -d, --duration=DURATION     :  test duration in seconds
  -r, --rampup=RAMPUP         :  rampup in seconds
  -i, --interval=INTERVAL     :  interval in milliseconds
  -x, --xmlfile=TEST_CASE_XML :  test case xml file
  -o, --output_dir=PATH       :  output directory
  -n, --name=TESTNAME         :  name of test
  -l, --log_msgs              :  log messages
  -b, --blocking              :  blocking mode
  -g, --gui                   :  start GUI
  -p, --port=PORT             :  xml-rpc listening port  
  -w, --web=0.0.0.0:8080      :  web server like 0.0.0.0 or 0.0.0.0:8080

Starting Pylot Remotely:

Pylot contains an XML-RPC server that can be launched so you can start tests with a remote client.

Configuration Options:

The file /core/config.py contains some global configuration options. You can set certain defauls and alter certain behavior here. These options here are overridden if specified on the command line.

AGENTS = 1
DURATION = 60  # secs
RAMPUP = 0  # secs
INTERVAL = 0  # millisecs
TC_XML_FILENAME = 'testcases.xml'
OUTPUT_DIR = None
TEST_NAME = None
LOG_MSGS = False

GENERATE_RESULTS = True
SHUFFLE_TESTCASES = False  # randomize order of testcases per agent
WAITFOR_AGENT_FINISH = True  # wait for last requests to complete before stopping
SMOOTH_TP_GRAPH = 1  # secs.  smooth/dampen throughput graph based on an interval
SOCKET_TIMEOUT = 300  # secs
COOKIES_ENABLED = True

HTTP_DEBUG = False  # only useful when combined with blocking mode  
BLOCKING = False  # stdout blocked until test finishes, then result is returned as XML
GUI = False

Using Pylot

Step 1: Create Test Cases

Test cases are declared in an XML file named "testcases.xml", or a different XML file specified on the command line. This is the format that the test engine understands.

A test case is defined using the following syntax. Only the URL element is required.

<case tenjin="true|false">
  <url>URL</url>
  <method>HTTP METHOD</method>
  <body >REQUEST BODY CONTENT</body>
  <add_header>ADDITIONAL HTTP HEADER</add_header>
  <add_header_tenjin>ADDITIONAL HTTP HEADER Tenjin Template</add_header_tenjin>
  <verify>STRING OR REGULAR EXPRESSION</verify>
  <verify_negative>STRING OR REGULAR EXPRESSION</verify_negative>
  <timer_group>TIMER GROUP NAME</timer_group>
</case>

Below is an example of the simplest possible test case file. It contains a single test case which will be executed continuously during the test run. The test case contains a URL for the service under test. Since no method or body defined, it will default to sending an HTTP GET to this resource. Since no verifications are defined, it will pass/fail the test case based on the HTTP status code it receives (pass if status is < 400).

<testcases>
  <case>
    <url>http://www.example.com/foo</url>
  </case>
</testcases>

We can add positive and negative verifications. A positive verification is a string or regular expression that must be contained in the response body. A negative verification is a string or regular expression that must not be contained in the response body.

<case>
    <url>http://www.goldb.org/foo</url>
    <verify>Copyright.*Corey Goldberg</verify>
    <verify_negative>Error</verify_negative>
<case>

Tenjin: body and url context support Tenjin tenamplate. Example:

<case tenjin="true">
  <url><![CDATA[
  <?py import random ?>
  <?py ids = [10047,10048,10049,10050,10052,10053,10055,10057,10058,10059,10060,10062,10063,10064,10066,10070,10071,10072,10074,10075,10076,10077,10079,10080,10081,10082,10083,10084,10089,10090,10092,10093,10094,10095,10096,10097,10099,10101,10102] ?>
  <?py _random = ids[random.randint(0, len(ids)-1)] ?>
  http://172.18.21.56:31941/node-config/v1/node/#{_random}]]></url>
  <method>POST</method>
  <add_header>Content-type: application/json</add_header>
  <add_header_tenjin><![CDATA[
<?py import random ?>
<?py ids = ["t1","t2","t3","t4","t5","t6","t7","t8","t9","t0"] ?>
<?py _random = ids[random.randint(0, len(ids)-1)] ?>
X-TENANT-ID: #{_random}]]></add_header_tenjin>
</case>

Cookies: Cookies are handled automatically. If a response is received with a "Set-cookie" header, the cookie will be set and passed back in the header of subsequent requests.

Example: Yahoo! Search Web Services (REST API) Yahoo offers various REST Web Services to access search results. In this example, I will show how to create Pylot test cases to interact with the REST API.

Here is a simple GET request against the service:

http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=foo

A Pylot test case for this request would look like this:

<case>
  <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=foo</url>
</case>

Notice that the ampersand (&) in the URL was escaped with the code: "&"

This is done becasue certain characters ("<" and "&") are illegal in XML documents. Since we are definig test cases within an XML doc, we must either escape these with ampersand codes, or place them within a CDATA section.

Yahoo also allows the query parameters to be passed in the POST data block. In this case we must also change the "Content-type" HTTP header to: "application/x-www-form-urlencoded". (Pylot defaults to "text/xml")

Here is a POST request against the service:

<case>
  <url>http://search.yahooapis.com/WebSearchService/V1/webSearch</url>
  <method>POST</method>
  <body><![CDATA[appid=YahooDemo&query=webinject]]></body>
  <add_header>Content-type: application/x-www-form-urlencoded</add_header>
</case>

Now that we know how to create individual cases, we can create a test case file containing several of these. In this example, our test case file contains Yahoo web search queries for: "foo", "bar", "baz"

<testcases>
  <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=foo</url>
  </case>
    <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=bar</url>
  </case>
    <case>
    <url>http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&amp;query=baz</url>
  </case>
</testcases>

Example: SOAP API

We can model our test cases to talk to any HTTP API. This example shows how you could send requests to a SOAP service. The SOAP envelope we need to send will be enclosed in the HTTP POST body.

<case>
  <url>http://www.example.org/StockPrice</url>
  <method>POST</method>
  <add_header>Content-Type: application/soap+xml; charset=utf-8</add_header>
  <body><!
    [CDATA[
    
      <!-- This is the SOAP Envelope  -->  
      <?xml version="1.0"?>
      <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
        soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
        <soap:Body xmlns:m="http://www.example.org/stock">
          <m:GetStockPrice>
            <m:StockName>IBM</m:StockName>
          </m:GetStockPrice>
        </soap:Body>
      </soap:Envelope>
      
    ]]>
  </body>
</case>

Example: Setting Static Variables/Parameters

You can define global parameters in your test case file. This is useful if you have a value shared among several test cases that you change often. In the example below, we define an "http_server" parameter and then use that token in a test case.

<testcases>
  <param name="http_server" value="http://www.example.com" />
  <case>
    <url>${http_server}/foo</url>
  </case>
</testcases>

Example: File-based HTTP Payloads

You may want to store POST data in an external file rather than declaring it directly in your testcase XML file. This is useful if you have very large POST BODYs or want to send binary data which can not be embedded in XML. Use the syntax below to pull data from a file and POST it at runtime.

<case>
  <url>http://www.example.com/foo</url>
  <method>POST</method>
  <body file="./myfile.dat"></body>
</case>

Step 2: Model Workload Scenario

Define a workload using the controls on the UI. Using the options below. you can create a steady-state or increasing load test.

Agents: number of agents (virtual users) to run

Rampup: time span over which agents are started. They will be evenly distributed throughout this time span. (see note below) Interval: interval at which each user sends requests. The requests from each user agent are paced at even intervals (unless the respone time is slower thean the interval defined) Duration: time span of the test

Step 3: Execute and Monitor

Run Modes

Console Mode: During the test, you can view real-time stats on the UI

Blocking Mode: STDOUT is blocked until test finishes, results are returned as XML

GUI Mode: Manage and view running tests with the GUI interface At the end of a test run, an HTML report is automatically generated, showing test results and graphs.

Step 4: View Results

When a test is finished, a results directory is created and a report is automatically generated to summarize the test results. It includes various statistics and graphs for response times and throughput. A sample of the results report can be seen here:

Sample Report

Pylot also writes results to CSV text files so you can import them into your favorite spreadsheet to crunch numbers, generate statistics, and create graphs.

develop doc

pip3 install virtualenv

# windows
python3 -m virtualenv venv_win
.\venv_win\Scripts\activate.bat

# Unix/linux/macos
python3 -m virtualenv venv_unix
# python3 -m virtualenv venv_unix --system-site-packages
source venv_unix/bin/activate


# opt for ubuntu
apt install -y libgtk-3-dev
apt install -y libldap2-dev libsasl2-dev
# opt for centos
yum install gtk3-devel
# ldap dependence
yum install openldap-devel

pip install -r requirement.txt

docker build -t pylot:v1 .

docker run -p 5000:5000 -it -d --name pylot -v /data/pylot/tasks:/workspace/tasks  -v /data/pylot/results:/workspace/results  -v /data/pylot/tasks_insts:/workspace/tasks_insts pylot:v1

Name		Name	Last commit message	Last commit date
Latest commit History 20 Commits
core		core
docker_config		docker_config
test_py		test_py
ui		ui
util		util
.gitignore		.gitignore
Dockerfile		Dockerfile
README.md		README.md
__init__.py		__init__.py
requirement.txt		requirement.txt
run.py		run.py
testcases.xml		testcases.xml

qintang/pylot

Folders and files

Latest commit

History

Repository files navigation

pylot

Getting Started Guide

Console and Blocking Mode

GUI Mode

Installing Pylot

GUI Mode:

Web Mode:

Console and Blocking Mode

Starting Pylot Remotely:

Configuration Options:

Using Pylot

Step 1: Create Test Cases

Step 2: Model Workload Scenario

Step 3: Execute and Monitor

Step 4: View Results

develop doc

About

Resources

Stars

Watchers

Forks

Languages