Package test :: Module regrtest |
[hide private]
[frames] | no frames]

# Module regrtest

source code

    Usage:

    python -m test.regrtest [options] [test_name1 [test_name2 ...]]
    python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]

    If no arguments or options are provided, finds all files matching
    the pattern "test_*" in the Lib/test subdirectory and runs
    them in alphabetical order (but see -M and -u, below, for exceptions).

    For more rigorous testing, it is useful to use the following
    command line:

    python -E -tt -Wd -3 -m test.regrtest [options] [test_name1 ...]

    Options:

    -h/--help       -- print this text and exit

    Verbosity

    -v/--verbose    -- run tests in verbose mode with output to stdout
    -w/--verbose2   -- re-run failed tests in verbose mode
    -W/--verbose3   -- re-run failed tests in verbose mode immediately
    -q/--quiet      -- no output unless one or more tests fail
    -S/--slow       -- print the slowest 10 tests
       --header     -- print header with interpreter info

    Selecting tests

    -r/--random     -- randomize test execution order (see below)
       --randseed   -- pass a random seed to reproduce a previous random run
    -f/--fromfile   -- read names of tests to run from a file (see below)
    -x/--exclude    -- arguments are tests to *exclude*
    -s/--single     -- single step through a set of tests (see below)
    -u/--use RES1,RES2,...
                    -- specify which special resource intensive tests to run
    -M/--memlimit LIMIT
                    -- run very large memory-consuming tests

    Special runs

    -l/--findleaks  -- if GC is available detect tests that leak memory
    -L/--runleaks   -- run the leaks(1) command just before exit
    -R/--huntrleaks RUNCOUNTS
                    -- search for reference leaks (needs debug build, v. slow)
    -j/--multiprocess PROCESSES
                    -- run PROCESSES processes at once
    -T/--coverage   -- turn on code coverage tracing using the trace module
    -D/--coverdir DIRECTORY
                    -- Directory where coverage files are put
    -N/--nocoverdir -- Put coverage files alongside modules
    -t/--threshold THRESHOLD
                    -- call gc.set_threshold(THRESHOLD)
    -F/--forever    -- run the specified tests in a loop, until an error happens

    Additional Option Details:

    -r randomizes test execution order. You can use --randseed=int to provide a
    int seed value for the randomizer; this is useful for reproducing troublesome
    test orders.

    -s On the first invocation of regrtest using -s, the first test file found
    or the first test file given on the command line is run, and the name of
    the next test is recorded in a file named pynexttest.  If run from the
    Python build directory, pynexttest is located in the 'build' subdirectory,
    otherwise it is located in tempfile.gettempdir().  On subsequent runs,
    the test in pynexttest is run, and the next test is written to pynexttest.
    When the last test has been run, pynexttest is deleted.  In this way it
    is possible to single step through the test files.  This is useful when
    doing memory analysis on the Python interpreter, which process tends to
    consume too many resources to run the full regression test non-stop.

    -f reads the names of tests from the file given as f's argument, one
    or more test names per line.  Whitespace is ignored.  Blank lines and
    lines beginning with '#' are ignored.  This is especially useful for
    whittling down failures involving interactions among tests.

    -L causes the leaks(1) command to be run just before exit if it exists.
    leaks(1) is available on Mac OS X and presumably on some other
    FreeBSD-derived systems.

    -R runs each test several times and examines sys.gettotalrefcount() to
    see if the test appears to be leaking references.  The argument should
    be of the form stab:run:fname where 'stab' is the number of times the
    test is run to let gettotalrefcount settle down, 'run' is the number
    of times further it is run and 'fname' is the name of the file the
    reports are written to.  These parameters all have defaults (5, 4 and
    "reflog.txt" respectively), and the minimal invocation is '-R :'.

    -M runs tests that require an exorbitant amount of memory. These tests
    typically try to ascertain containers keep working when containing more than
    2 billion objects, which only works on 64-bit systems. There are also some
    tests that try to exhaust the address space of the process, which only makes
    sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
    which is a string in the form of '2.5Gb', determines howmuch memory the
    tests will limit themselves to (but they may go slightly over.) The number
    shouldn't be more memory than the machine has (including swap memory). You
    should also keep in mind that swap memory is generally much, much slower
    than RAM, and setting memlimit to all available RAM or higher will heavily
    tax the machine. On the other hand, it is no use running these tests with a
    limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
    to use more than memlimit memory will be skipped. The big-memory tests
    generally run very, very long.

    -u is used to specify which special resource intensive tests to run,
    such as those requiring large file support or network connectivity.
    The argument is a comma-separated list of words indicating the
    resources to test.  Currently only the following are defined:

        all -       Enable all special resources.

        audio -     Tests that use the audio device.  (There are known
                    cases of broken audio drivers that can crash Python or
                    even the Linux kernel.)

        curses -    Tests that use curses and will modify the terminal's
                    state and output modes.

        largefile - It is okay to run some test that may create huge
                    files.  These tests can take a long time and may
                    consume >2GB of disk space temporarily.

        network -   It is okay to run tests that use external network
                    resource, e.g. testing SSL support for sockets.

        bsddb -     It is okay to run the bsddb testsuite, which takes
                    a long time to complete.

        decimal -   Test the decimal module against a large suite that
                    verifies compliance with standards.

        cpu -       Used for certain CPU-heavy tests.

        subprocess  Run all tests for the subprocess module.

        urlfetch -  It is okay to download files required on testing.

        gui -       Run tests that require a running GUI.

        xpickle -   Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to
                    test backwards compatibility. These tests take a long time
                    to run.

    To enable all resources except one, use '-uall,-<resource>'.  For
    example, to run all the tests except for the bsddb tests, give the
    option '-uall,-bsddb'.

Classes | [hide private]
| saved_test_environment Save bits of the test environment and restore them at block exit.
| _ExpectedSkips

Functions | [hide private]
usage(code, msg=`'````'`) | source code
main(tests=None, testdir=None, verbose=0, quiet=False, exclude=False, single=False, randomize=False, fromfile=None, findleaks=False, use_resources=None, trace=False, coverdir=`'``coverage``'`, runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, random_seed=None, use_mp=None, verbose3=False, forever=False, header=False) Execute a test suite. | source code
findtests(testdir=None, stdtests=`[``'``test_grammar``'``,` `'``test_opcodes``'``,` `'``test_dict``'``,` `'``test_builtin``'``,` `...`, nottests=`set([``'``test_future1``'``,` `'``test_future2``'``,` `'``test_support``'``])`) Return a list of all applicable test modules. | source code
runtest(test, verbose, quiet, huntrleaks=False, use_resources=None) Run a single test. | source code
runtest_inner(test, verbose, quiet, huntrleaks=False) | source code
cleanup_test_droppings(testname, verbose) | source code
dash_R(the_module, test, indirect_test, huntrleaks) Run a test multiple times, looking for reference leaks. | source code
dash_R_cleanup(fs, ps, pic, zdc, abcs) | source code
findtestdir(path=None) | source code
removepy(names) | source code
count(n, word) | source code
printlist(x, width=70, indent=4) Print the elements of iterable x to stdout. | source code

Variables | [hide private]
| newsoft = `min(hard, max(soft, 1024* 2048))`
| PASSED = `1`
| FAILED = `0`
| ENV_CHANGED = `-1`
| SKIPPED = `-2`
| RESOURCE_DENIED = `-3`
| INTERRUPTED = `-4`
| RESOURCE_NAMES = ``(``'``audio``'``,` `'``curses``'``,` `'``largefile``'``,` `'``network``'``,` `'``...``
| TEMPDIR = ``'``/tmp``'``
| STDTESTS = ``[``'``test_grammar``'``,` `'``test_opcodes``'``,` `'``test_dict``'``,` `'``test``...``
| NOTTESTS = ``set([``'``test_future1``'``,` `'``test_future2``'``,` `'``test_support``'``])``
| _expectations = ``{``'``aix5``'``:` `'``\n test_bsddb\n test_b``...``
| __package__ = ``'``test``'``
| module = `None`
| path = ``'``/usr/lib/pymodules/python2.7/docutils/parsers/rst/dire``...``

Function Details | [hide private]

### main(tests=None, testdir=None, verbose=0, quiet=False, exclude=False, single=False, randomize=False, fromfile=None, findleaks=False, use_resources=None, trace=False, coverdir=`'``coverage``'`, runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, random_seed=None, use_mp=None, verbose3=False, forever=False, header=False) | source code
Execute a test suite. This also parses command-line options and modifies its behavior accordingly. tests -- a list of strings containing test names (optional) testdir -- the directory in which to look for tests (optional) Users other than the Python test suite will certainly want to specify testdir; if it's omitted, the directory containing the Python test suite is searched for. If the tests argument is omitted, the tests listed on the command-line will be used. If that's empty, too, then all *.py files beginning with test_ will be used. The other default arguments (verbose, quiet, exclude, single, randomize, findleaks, use_resources, trace, coverdir, print_slow, and random_seed) allow programmers calling main() directly to set the values that would normally be set by flags on the command line.

### runtest(test, verbose, quiet, huntrleaks=False, use_resources=None) | source code
    Run a single test.

    test -- the name of the test
    verbose -- if true, print more messages
    quiet -- if true, don't print 'skipped' messages (probably redundant)
    test_times -- a list of (time, test_name) pairs
    huntrleaks -- run multiple times to test for leaks; requires a debug
                  build; a triple corresponding to -R's three arguments
    Returns one of the test result constants:
        INTERRUPTED      KeyboardInterrupt when run under -j
        RESOURCE_DENIED  test skipped because resource denied
        SKIPPED          test skipped for some other reason
        ENV_CHANGED      test failed because it changed the execution environment
        FAILED           test failed
        PASSED           test passed

### dash_R(the_module, test, indirect_test, huntrleaks) | source code
    Run a test multiple times, looking for reference leaks.

    Returns:
        False if the test didn't leak references; True if we detected refleaks.

### printlist(x, width=70, indent=4) | source code
Print the elements of iterable x to stdout. Optional arg width (default 70) is the maximum line length. Optional arg indent (default 4) is the number of blanks with which to begin each line.

Variables Details | [hide private]

  ### RESOURCE_NAMES Value:
      ('audio',
       'curses',
       'largefile',
       'network',
       'bsddb',
       'decimal',
       'cpu',
       'subprocess',
      ...

  ### STDTESTS Value:
      ['test_grammar',
       'test_opcodes',
       'test_dict',
       'test_builtin',
       'test_exceptions',
       'test_types',
       'test_unittest',
       'test_doctest',
      ...

  ### _expectations Value:
      {'aix5': '''
              test_bsddb
              test_bsddb185
              test_bsddb3
              test_bz2
              test_dl
              test_epoll
              test_gdbm
      ...

  ### path Value:
      '/usr/lib/pymodules/python2.7/docutils/parsers/rst/directives'