1..N ok 1 Description # Directive # Diagnostic .... ok 47 Description ok 48 Description more tests....
For example, a test file's output might look like:
1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet
A harness must only read TAP output from standard output and not from standard error. Lines written to standard output matching "/^(not )?ok\b/" must be interpreted as test lines. All other lines must not be considered test output.
The plan is usually the first line of TAP output and it specifies how many test points are to follow. For example,
means you plan on running 10 tests. This is a safeguard in case your test file dies silently in the middle of its run. The plan is optional but if there is a plan before the test points it must be the first non-diagnostic line output by the test file.
In certain instances a test file may not know how many test points it will ultimately be running. In this case the plan can be the last non-diagnostic line in the output.
The plan cannot appear in the middle of the output, nor can it appear more than once.
This tells whether the test point passed or failed. It must be at the beginning of the line. "/^not ok/" indicates a failed test point. "/^ok/" is a successful test point. This is the only mandatory part of the line.
Note that unlike the Directives below, "ok" and "not ok" are case-sensitive.
TAP expects the "ok" or "not ok" to be followed by a test point number. If there is no number the harness must maintain its own counter until the script supplies test numbers again. So the following test output
1..6 not ok ok not ok ok ok
has five tests. The sixth is missing. Test::Harness will generate
FAILED tests 1, 3, 6 Failed 3/6 tests, 50.00% okay
Any text after the test number but before a "#" is the description of the test point.
ok 42 this is the description of the test
Descriptions should not begin with a digit so that they are not confused with the test point number.
The harness may do whatever it wants with the description.
The test point may include a directive, following a hash on the test line. There are currently two directives allowed: "TODO" and "SKIP". These are discussed below.
not ok 13 # TODO bend space and time
Note that if the TODO has an explanation it must be separated from "TODO" by a space.
These tests represent a feature to be implemented or a bug to be fixed and act as something of an executable ``things to do'' list. They are not expected to succeed. Should a todo test point begin succeeding, the harness should report it as a bonus. This indicates that whatever you were supposed to do has been done and you should promote this to a normal test point.
ok 23 # skip Insufficient flogiston pressure.
Similarly, one can include an explanation in a plan line, emitted if the test file is skipped completely:
1..0 # Skipped: WWW::Mechanize not installed
to standard output. Any message after these words must be displayed by the interpreter as the reason why testing must be stopped, as in
Bail out! MySQL is not running.
... ok 18 - Closed database connection # End of database section. # This starts the network part of the test. # Daemon started on port 2112 ok 19 - Opened socket ... ok 47 - Closed socket # End of network tests
1..6 # # Create a new Board and Tile, then place # the Tile onto the board. # ok 1 - The object isa Board ok 2 - Board size is zero ok 3 - The object isa Tile ok 4 - Get possible places to put the Tile ok 5 - Placing the tile produces no error ok 6 - Board size is 1
ok 1 - retrieving servers from the database # need to ping 6 servers ok 2 - pinged diamond ok 3 - pinged ruby not ok 4 - pinged saphire ok 5 - pinged onyx not ok 6 - pinged quartz ok 7 - pinged gold 1..7
1..573 not ok 1 - database handle Bail out! Couldn't connect to database.
1..5 ok 1 - approved operating system # $^0 is solaris ok 2 - # SKIP no /sys directory ok 3 - # SKIP no /sys directory ok 4 - # SKIP no /sys directory ok 5 - # SKIP no /sys directory
1..0 # skip because English-to-French translator isn't installed
1..4 ok 1 - Creating test program ok 2 - Test program runs, no error not ok 3 - infinite loop # TODO halting problem unsolved not ok 4 - infinite loop 2 # TODO halting problem unsolved
ok - created Board ok ok ok ok ok ok ok # +------+------+------+------+ # | |16G | |05C | # | |G N C | |C C G | # | | G | | C +| # +------+------+------+------+ # |10C |01G | |03C | # |R N G |G A G | |C C C | # | R | G | | C +| # +------+------+------+------+ # | |01G |17C |00C | # | |G A G |G N R |R N R | # | | G | R | G | # +------+------+------+------+ ok - board has 7 tiles + starter tile 1..9
The following sections are provided by their maintainers, and may not be up-to-date.
More information about libtap, including download links, checksums, anonymous access to the Subersion repository, and a bug tracking system, can be found at:
(Nik Clayton, April 17, 2006)
Right now, it's got a fair bit of the basics needed to emulate Test::More, and I think it's easy to add more stuff --- just like Test::Builder, there's a singleton that you can get at easily.
I need to better identify and finish implementing the most basic tests. I am not a Python guru, I just use it from time to time, so my aim may not be true. I need to write tests for it, which means either relying on Perl for the tester tester, or writing one in Python.
Here's a sample test, as found in my Subversion:
from TAP.Simple import * plan(15) ok(1) ok(1, "everything is OK!") ok(0, "always fails") is_ok(10, 10, "is ten ten?") is_ok(ok, ok, "even ok is ok!") ok(id(ok), "ok is not the null pointer") ok(True, "the Truth will set you ok") ok(not False, "and nothing but the truth") ok(False, "and we'll know if you lie to us") isa_ok(10, int, "10") isa_ok('ok', str, "some string") ok(0, "zero is true", todo="be more like Ruby!") ok(None, "none is true", skip="not possible in this universe") eq_ok("not", "equal", "two strings are not equal");
(Ricardo Signes, April 17, 2006)
(David Wheeler, April 17, 2006)
Outputs TAP by default as of the yet-to-be-released PEAR 1.5.0
Has a TAP logger (since 2.3.4)
There's a third-party TAP reporting extension for SimpleTest
Apache-Test's PHP writes TAP by default and includes the standalone test-more.php
(Geoffrey Young, April 17, 2006)
The basis for the TAP format was created by Larry Wall in the original test script for Perl 1. Tim Bunce and Andreas Koenig developed it further with their modifications to Test::Harness.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.