diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 70a4e3c2..bd647c58 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -21,7 +21,7 @@ Changelog means that on the class keyword it will now return the docstring of Python's builtin function ``help('class')``. - The API documentation is now way more readable and complete. Check it out - under https://jedi.readthedocs.io. + under https://jedi.readthedocs.io. A lot of it has been rewritten. - Removed Python 3.4 support - Many bugfixes diff --git a/docs/docs/testing.rst b/docs/docs/testing.rst index 0c666a13..1aa4a1f1 100644 --- a/docs/docs/testing.rst +++ b/docs/docs/testing.rst @@ -28,8 +28,8 @@ simple and readable testing structure. .. _blackbox: -Blackbox Tests (run.py) -~~~~~~~~~~~~~~~~~~~~~~~ +Integration Tests (run.py) +~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: test.run diff --git a/test/refactor.py b/test/refactor.py index bfd7a4a7..2a997b19 100644 --- a/test/refactor.py +++ b/test/refactor.py @@ -1,7 +1,8 @@ #!/usr/bin/env python """ -Refactoring tests work a little bit similar to Black Box tests. But the idea is -here to compare two versions of code. +Refactoring tests work a little bit similar to integration tests. But the idea +is here to compare two versions of code. If you want to add a new test case, +just look at the existing ones in the ``test/refactor`` folder and copy them. """ from __future__ import with_statement import os diff --git a/test/run.py b/test/run.py index 3a0c08dd..4ea8c14f 100755 --- a/test/run.py +++ b/test/run.py @@ -1,14 +1,11 @@ #!/usr/bin/env python """ -|jedi| is mostly being tested by what I would call "Blackbox Tests". These -tests are just testing the interface and do input/output testing. This makes a -lot of sense for |jedi|. Jedi supports so many different code structures, that -it is just stupid to write 200'000 unittests in the manner of -``regression.py``. Also, it is impossible to do doctests/unittests on most of -the internal data structures. That's why |jedi| uses mostly these kind of -tests. +|jedi| is mostly being tested by what I would call "integration tests". These +tests are testing type inference with the public API. This makes a +lot of sense for |jedi|. Also, it is hard to write doctests/unittests for +the internal data structures. -There are different kind of tests: +There are different kinds of tests: - completions / inference ``#?`` - goto: ``#!`` @@ -18,29 +15,31 @@ How to run tests? +++++++++++++++++ Jedi uses pytest_ to run unit and integration tests. To run tests, -simply run ``pytest``. You can also use tox_ to run tests for -multiple Python versions. +simply run ``pytest``. .. _pytest: http://pytest.org -.. _tox: http://testrun.org/tox -Integration test cases are located in ``test/completion`` directory -and each test case is indicated by either the comment ``#?`` (completions / -inference), ``#!`` (goto), or ``#<`` (references). +Most integration test cases are located in the ``test/completion`` directory +and each test case starts with one of these comments: + +- ``#?`` (completions / inference) +- ``#!`` (goto) +- ``#<`` (references) + There is also support for third party libraries. In a normal test run they are not being executed, you have to provide a ``--thirdparty`` option. -In addition to standard `-k` and `-m` options in pytest, you can use the -`-T` (`--test-files`) option to specify integration test cases to run. +In addition to pytest's ``-k`` and ``-m`` options, you can use the +``-T`` (``--test-files`) option to specify which test cases should run. It takes the format of ``FILE_NAME[:LINE[,LINE[,...]]]`` where ``FILE_NAME`` is a file in ``test/completion`` and ``LINE`` is a line -number of the test comment. Here is some recipes: +number of the test comment. Here are some examples: -Run tests only in ``basic.py`` and ``imports.py``:: +Run tests only in ``completion/basic.py`` and ``completion/imports.py``:: pytest test/test_integration.py -T basic.py -T imports.py -Run test at line 4, 6, and 8 in ``basic.py``:: +Run test at line 4, 6, and 8 in ``completion/basic.py``:: pytest test/test_integration.py -T basic.py:4,6,8 @@ -57,38 +56,30 @@ that you can start by running ``./run.py``. The above example could be run by:: ./run.py basic 4 6 8 50-80 The advantage of this runner is simplicity and more customized error reports. -Using both runners will help you to have a quicker overview of what's -happening. +Auto-Completion Tests ++++++++++++++++++++++ -Auto-Completion +Uses a comment to specify a test on the next line. The comment defines the +expected completions. The comment always begins with `#?`. The last row +symbolizes the cursor. For example:: + + #? ['upper'] + a = 'foo'; a.upp + +Inference Tests +++++++++++++++ -Uses comments to specify a test in the next line. The comment says which -results are expected. The comment always begins with `#?`. The last row -symbolizes the cursor. - -For example:: - - #? ['real'] - a = 3; a.rea - -Because it follows ``a.rea`` and a is an ``int``, which has a ``real`` -property. - -Inference -+++++++++ - -Inference tests use the same symbols like completion tests. This is -possible because the completion tests are defined with a list:: +Inference tests look very simliar. The difference is that inference tests don't +use brackets:: #? int() ab = 3; ab -Goto -++++ +Goto Tests +++++++++++ -Tests look like this:: +Goto Tests look like this:: abc = 1 #! ['abc=1'] @@ -100,13 +91,13 @@ describes the position of the test (otherwise it's just the end of line):: #! 2 ['abc=1'] abc -References -++++++++++ +Reference Tests ++++++++++++++++ Tests look like this:: abc = 1 - #< abc@1,0 abc@3,0 + #< (1,0), (3,0) abc """ import os