Load testing sites using Curl Loader


Often (most always) at the design stage it is impossible to predict the system bottlenecks. This is largely due to the inaccuracy of the projected peak load and in the context of specific modules. Even if you follow all the basic principles for developing systems of heavy duty, you are likely to be problems with specific subsystems with increasing loads. Pre-defined places and prepare a strategy to help scale load testing.

For stress testing a number of tools and today we look at one of them – simple and powerful Curl Loader.

Installation

Installing Curl Loader is very easy (the main requirement – Linux / Unix environment). First you need to download the source code with fresh symye the developer’s site:
http://sourceforge.net/projects/curl-loader/files/

After that, unpack and install:

tar-xvf curl-loader-.tar.gz
cd curl-loader-
make
make install

By default, Curl Loader is compiled in debug mode. In order to disable debug, and improve the performance of tests you can do the following:

make cleanall
make optimize = 1 debug = 0
make install

Run Tests

Running the tests performed using the utility curl-loader and transfer it to any configuration file with option-f:

curl-loader-f bulk.conf

Create a configuration file (named test.conf) for a very simple test as follows:

########### GENERAL SECTION ################################

BATCH_NAME = test
CLIENTS_NUM_MAX = 50
CLIENTS_NUM_START = 10
CLIENTS_RAMPUP_INC = 5
INTERFACE = eth0
NETMASK = 16
IP_ADDR_MIN = 192.168.0.192
IP_ADDR_MAX = 192.168.0.192
IP_SHARED_NUM = 1
CYCLES_NUM = -1
URLS_NUM = 1

########### URL SECTION ####################################

URL = http://google.com/
URL_SHORT_NAME = "Google.com"
REQUEST_TYPE = GET
TIMER_URL_COMPLETION = 5000
TIMER_AFTER_URL_SLEEP = 500

In this file we have configured the test to test the main page of google.com with 50 simultaneous clients. Begin testing with 10 clients. Every second, adds another 5 clients until they reach 50. After saving the file run the Curl Loader:

sudo curl-loader-f test.conf

In addition, after testing, Curl Loader creates three files:

test.log – error information and trace
test.txt – download statistics
test.ctx – statistics in the context of virtual client

In unpacked the archive you can find the folder conf-examples, which are examples of configuration files to perform the exercise test.

Configuration files

The configuration files allow you to create fairly complex logic caesarean tests (authorization, send the form, download files, etc.). Files consist of two sections: GENERAL, and URL.

The parameter list of configuration files:

Section GENERAL

  • BATCH_NAME – name of the test, is used as a name for report files
  • CLIENTS_NUM_MAX – maximum number of clients
  • CLIENTS_NUM_START – the initial number of customers (immediately after the test run)
  • CLIENTS_RAMPUP_INC – the number of customers that will be added every second
  • IP_ADDR_MIN, IP_ADDR_MAX – a lot of IP addresses that belong to customers. If the values ??in these two parameters are the same, all customers will receive the same IP address
  • CYCLES_NUM – the number of cycles during testing. If set to -1, then the tests are repeated endlessly (interruption of the execution of Ctrl + C)
  • URLS_NUM – the number of URLs in the Description section addresses – see below

Section URL

  • URL – the first parameter for each sub-section, specifies the address to test
  • URL_SHORT_NAME – optional short name of the link (up to 12 characters)
  • URL_USE_CURRENT – used when the option URL = "" (blank) to indicate that it is necessary to use the URL from the previous operation. It is useful when you want to send a POST request to the resulting URL from the previous GET request
  • REQUEST_TYPE – the type of HTTP request: GET, POST or PUT
  • UPLOAD_FILE – full path to the file for download via the link
  • HEADER – used to install / re-HTTP / FTP headlines
  • USERNAME, PASSWORD – these parameters are used for the operation authorization. Their functionality is controlled by the parameter
  • FORM_USAGE_TYPE (see full list)
  • FORM_STRING – form template to submit data POST request. Contains template parameters to be passed. Values ??can be populated from an external file specified in the parameter FORM_RECORDS_FILE.
  • WEB_AUTH_METHOD, WEB_AUTH_CREDENTIALS – used for HTTP authentication
  • LOG_RESP_HEADERS, LOG_RESP_BODIES – enables / disables the logging of headers and bodies replies

As examples, be sure to look at the configuration examples that come with sources. There are a lot of use cases that can be taken as the basis for test development for the project.

Full list of options can be found on the official site

Testing of the really big loads

  1. When testing with hundreds of thousands of customers, be sure to note the following:
  2. Need to increase the limit for the socket to the required (for example, ulimit-n 10000)
    You can also enable the reuse of the socket (echo 1> / proc/sys/net/ipv4/tcp_tw_recycle and / or; echo 1> / proc/sys/net/ipv4/tcp_tw_reuse)

See Also

    Advertising

    Archives