From 89cf8634e10bda9a9f836af2913a5adc137742ca Mon Sep 17 00:00:00 2001 From: mierle Date: Fri, 6 Jun 2008 01:10:05 +0000 Subject: [PATCH] Edited wiki page through web user interface. --- wiki/GoogleTestAdvancedGuide.wiki | 1075 ++++++++++++++++++++++++++++- 1 file changed, 1068 insertions(+), 7 deletions(-) diff --git a/wiki/GoogleTestAdvancedGuide.wiki b/wiki/GoogleTestAdvancedGuide.wiki index fd95a201..8d77e992 100644 --- a/wiki/GoogleTestAdvancedGuide.wiki +++ b/wiki/GoogleTestAdvancedGuide.wiki @@ -1,13 +1,1074 @@ #summary Google Test Advanced Guide += Google Test Advanced Guide = -= Introduction = +Now that you have read the Google Test Primer and learned how to write tests +using Google Test, it's time to learn some new tricks. This document will show you +more assertions as well as how to construct complex failure messages, propagate +fatal failures, reuse and speed up your test fixtures, and use various flags +with your tests. -Add your content here. +== More Assertions == + +This section covers some less frequently used, but still significant, assertions. + +=== Explicit Success and Failure === + + These three assertions do not actually test a value or expression. Instead, +they generate a success or failure directly. Like the macros that actually +perform a test, you may stream a custom failure message into the them. + +{{{ +SUCCEED(); +}}} + +Generates a success. This does NOT make the overall test succeed. A test is +considered successful only if none of its assertions fail during its execution. + +Note: `SUCCEED()` is purely documentary and currently doesn't generate any +user-visible output. However, we may add `SUCCEED()` messages to Google Test's +output in the future. + +|| `FAIL();` || `ADD_FAILURE();` || + +`FAIL*` generates a fatal failure while `ADD_FAILURE*` generates a nonfatal +failure. These are useful when control flow, rather than a Boolean expression, +deteremines the test's success or failure. For example, you might want to write +something like: + +{{{ +switch(expression) { + case 1: ... some checks ... + case 2: ... some other checks + ... + default: FAIL() << "We shouldn't get here."; +} +}}} + +_Availability_: Linux, Windows, Mac. + +=== Windows HRESULT assertions === + +These assertions test for `HRESULT` success or failure. + +|| `ASSERT_HRESULT_SUCCEEDED(`_`expression`_`);` || `EXPECT_HRESULT_SUCCEEDED(`_`expression`_`);` || +|| `ASSERT_HRESULT_FAILED(`_`expression`_`);` || `EXPECT_HRESULT_FAILED(`_`expression`_`);` || + +`ASSERT_HRESULT_SUCCEEDED` and `EXPECT_HRESULT_SUCCEEDED` fail if _`expression`_ +is a failure `HRESULT`. `ASSERT_HRESULT_FAILED` and `EXPECT_HRESULT_FAILED` +fail if _`expression`_ is a success `HRESULT`. + +In either case, the generated output contains the human-readable error message +associated with the `HRESULT` code returned by _expression_. + +You might use them like this: + +{{{ +CComPtr shell; +ASSERT_HRESULT_SUCCEEDED(shell.CoCreateInstance(L"Shell.Application")); +CComVariant empty; +ASSERT_HRESULT_SUCCEEDED(shell->ShellExecute(CComBSTR(url), empty, empty, empty, empty)); +}}} + +_Availability_: Windows. + +=== Predicate Assertions === + +Even though Google Test has a rich set of assertions, they can never be +complete, as it's impossible (nor a good idea) to anticipate all the scenarios +a user might run into. Therefore, sometimes a user has to use `EXPECT_TRUE()` +to check a complex expression, for lack of a better macro. This has the problem +of not showing you the values of the parts of the expression, making it hard to +understand what went wrong. As a workaround, some users choose to construct the +failure message by themselves, streaming it into `EXPECT_TRUE()`. However, this +is awkward especially when the expression has side-effects or is expensive to +evaluate. + +To solve this problem, we provide a family of generic _predicate assertions_: +|| `ASSERT_PRED1(`_`pred1, val1`_`);` || `EXPECT_PRED1(`_`pred1, val1`_`);` || +|| `ASSERT_PRED2(`_`pred2, val1, val2`_`);` || `EXPECT_PRED2(`_`pred2, val1, val2`_`);` || +|| ... || ... || + +In the above, _`predn`_ is an _`n`_-ary predicate function or functor, where +_`val1`_, _`val2`_, ..., and _`valn`_ are its arguments. The assertion succeeds +if the predicate returns true when applied on the given arguments, and fails +otherwise. When the assertion fails, it prints the value of each arguments. In +either case, the arguments are evaluated exactly once. + +Currently we only provide predicate assertions of arity <= 5. If you need a +higher-arity assertion, let us know. + +Here's an example. Given + +{{{ +// Returns true iff m and n have no common divisors except 1. +bool MutuallyPrime(int m, int n) { ... } +const int a = 3; +const int b = 4; +const int c = 10; +}}} + +the assertion `EXPECT_PRED2(MutuallyPrime, a, b);` will succeed, while the +assertion `EXPECT_PRED2(MutuallyPrime, b, c);` will fail with the message + +{{{ +MutuallyPrime(b, c) is false, +where + b is 4 + c is 10 +}}} + +If you find the default message generated by `(ASSERT|EXPECT)_PRED*` not +satisfactory, or some arguments to your predicate do not support streaming to +`ostream`, you can instead use the following assertions to customize how the +message is formatted: + +|| `ASSERT_PRED_FORMAT1(`_`pred_format1, val1`_);` || `EXPECT_PRED_FORMAT1(`_`pred_format1, val1`_); || +|| `ASSERT_PRED_FORMAT2(`_`pred_format2, val1, val2`_`);` || `EXPECT_PRED_FORMAT2(`_`pred_format2, val1, val2`_); || +|| ... || ... || + +The difference between this and the previous group of macros is that instead of +a predicate, `(ASSERT|EXPECT)_PRED_FORMAT*` take a _predicate-formatter_ +(`_pred_formatn_`), which is a function or functor that has the signature: + +`testing::AssertionResult PredicateFormattern(const char* `_`expr1`_`, const char* `_`expr2`_`, ... const char* `_`exprn`_`, T1 `_`val1`_`, T2 `_`val2`_`, ... Tn `_`valn`_`);` + +where _`val1`_, _`val2`_, ..., and _`valn`_ are the values of the predicate +arguments, and _`expr1`_, _`expr2`_, ..., and _`exprn`_ are the corresponding +expressions as they appear in the source code. The types `T1`, `T2`, ..., and +`Tn` can be either value types or reference types. For example, if an +argument has type `Foo`, you can declare it as either `Foo` or `const Foo&`, +whichever is appropriate. + +A predicate-formatter returns a `testing::AssertionResult` object to indicate +whether the assertion has succeeded or not. The only way to create such an +object is to call one of these factory functions: + +{{{ +namespace testing { + +// Returns an AssertionResult object to indicate that an assertion has +// succeeded. +AssertionResult AssertionSuccess(); + +// Returns an AssertionResult object to indicate that an assertion has +// failed with the given failure message. +AssertionResult AssertionFailure(const Message& msg); + +} // namespace testing +}}} + +As an example, let's improve the failure message in the previous example, which uses `EXPECT_PRED2()`: + +{{{ +// Returns the smallest prime common divisor of m and n, +// or 1 when m and n are mutually prime. +int SmallestPrimeCommonDivisor(int m, int n) { ... } + +// A predicate-formatter for asserting that two integers are mutually prime. +testing::AssertionResult AssertMutuallyPrime(const char* m_expr, const char* n_expr, int m, int n) { + if (MutuallyPrime(m, n)) + return testing::AssertionSuccess(); + testing::Message msg; + msg << m_expr << " and " << n_expr << " (" << m << " and " << n + << ") are not mutually prime, " << "as they have a common divisor " + << SmallestPrimeCommonDivisor(m, n); + return testing::AssertionFailure(msg); +} +}}} + +With this predicate-formatter, we can use +`EXPECT_PRED_FORMAT2(AssertMutuallyPrime, b, c);` to generate the message +`b and c (4 and 10) are not mutually prime, as they have a common divisor 2.` + +As you may have realized, many of the assertions we introduced earlier are +special cases of `(EXPECT|ASSERT)_PRED_FORMAT*`. In fact, most of them are +indeed defined using `(EXPECT|ASSERT)_PRED_FORMAT*`. + +_Note:_ If you see a compiler error "no matching function to call" when using +`ASSERT_PRED*` or `EXPECT_PRED*`, please see the GoogleTestFAQ for how to +resolve it. + +_Availability_: Linux, Windows, Mac. -= Details = +=== Floating-Point Comparison === -Add your content here. Format your content with: - * Text in *bold* or _italic_ - * Headings, paragraphs, and lists - * Automatic links to other wiki pages \ No newline at end of file +Comparing floating-point numbers is tricky. Due to round-off errors, it is +very unlikely that two floating-points will match exactly. Therefore, +`ASSERT_EQ` 's naive comparison usually doesn't work. And since floating-points +can have a wide value range, no single fixed error bound works. It's better to +compare by a fixed relative error bound, except for values close to 0 due to +the loss of precision there. + +In general, for floating-point comparison to make sense, the user needs to +carefully choose the error bound. If they don't want or care to, comparing in +terms of Units in the Last Place (ULPs) is a good default, and Google Test +provides assertions to do this. Full details about ULPs are quite long; if you +want to learn more, see +[http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm this +article on float comparison]. + +==== Floating-Point Macros ==== + +|| `ASSERT_FLOAT_EQ(`_`expected, actual`_`);` || `EXPECT_FLOAT_EQ(`_`expected, actual`_`);` || +|| `ASSERT_DOUBLE_EQ(`_`expected, actual`_`);` || `EXPECT_DOUBLE_EQ(`_`expected, actual`_`);` || + +Verifies that two `float` or `double` values are almost equal, meaning they are +within 4 ULP's from each other. + +The following assertions allow you to choose the acceptable error bound: + +|| `ASSERT_NEAR(`_`val1, val2, abs_error`_`);` || `EXPECT_NEAR`_`(val1, val2, abs_error`_`);` || + +Verifies that the difference between _`val1`_ and _`val2`_ doesn't exceed the +given absolute error. + +_Availability_: Linux, Windows, Mac. + +==== Floating-Point Predicate-Format Functions ==== + +Some floating-point operations are useful, but not that often used. In order +to avoid an explosion of new macros, we provide them as predicate-format +functions that can be used in predicate assertion macros (e.g. +`EXPECT_PRED_FORMAT2`, etc). + +{{{ +EXPECT_PRED_FORMAT2(testing::FloatLE, val1, val2); +EXPECT_PRED_FORMAT2(testing::DoubleLE, val1, val2); +}}} + +Verifies that _`val1`_ is less than, or almost equal to, _`val2`_. You can +replace `EXPECT_PRED_FORMAT2` in the above table with `ASSERT_PRED_FORMAT2`. + +_Availability_: Linux, Windows, Mac. + +=== Assertion Placement === + +You can include assertions in any C++ function's code. The one constraint is +that assertions that generate a fatal failure (`FAIL*` and `ASSERT_*`) can only +be used by void-returning functions. This is a consequence of Google Test not +using exceptions. By placing it in a non-void function you'll get a confusing +compile error like `"error: void value not ignored as it ought to be"`. + +If you need to use assertions in a function that returns non-void, one option +is to make the function return the value in an out parameter instead. For +example, you can rewrite `T2 Foo(T1 x)` to `void Foo(T1 x, T2* result)`. You +need to make sure that `*result` contains some sensible value even when the +function returns prematurely. As the function now returns `void`, you can use +any assertion inside of it. + +If changing the function's type is not an option, you should just use +assertions that generate non-fatal failures, such as `ADD_FAILURE*` and +`EXPECT_*`. + +_Note_: Constructors and destructors are not considered void-returning +functions, according to the C++ language specification, and so you may not use +fatal assertions in them. You'll get a compilation error if you try. A simple +workaround is to transfer the entire body of the constructor or destructor to a +private void-returning method. However, you should be aware that a fatal +assertion failure in a constructor does not terminate the current test, as your +intuition might suggest; it merely returns from the constructor early, possibly +leaving your object in a partially-constructed state. Likewise, a fatal +assertion failure in a destructor may leave your object in a +partially-destructed state. Use assertions carefully in these situations! + +== Using Assertions in Sub-routines == + +=== Propagating Fatal Failures === + +A common pitfall when using `ASSERT_*` and `FAIL*` is not understanding that +when they fail they only abort the _current function_, not the entire test. For +example, the following test will segfault: +{{{ +void Subroutine() { + // Generates a fatal failure and aborts the current function. + ASSERT_EQ(1, 2); + // The following won't be executed. + ... +} + +TEST(FooTest, Bar) { + Subroutine(); + // The intended behavior is for the fatal failure + // in Subroutine() to abort the entire test. + // The actual behavior: the function goes on after Subroutine() returns. + int* p = NULL; + *p = 3; // Segfault! +} +}}} + +Since we don't use exceptions, it is technically impossible to implement the +intended behavior here. + +To alleviate this, Google Test provides the `HasFatalFailure()` function in the +`testing::Test` class, which returns `true` if an assertion in the current test +has suffered a fatal failure. This allows functions to catch fatal failures in +a sub-routine and return early. + +{{{ +class Test { + public: + ... + static bool HasFatalFailure(); +}; +}}} + +The typical usage, which basically simulates the behavior of a thrown +exception, is: + +{{{ +TEST(FooTest, Bar) { + Subroutine(); + // Aborts if Subroutine() had a fatal failure. + if (HasFatalFailure()) + return; + // The following won't be executed. + ... +} +}}} + +If `HasFatalFailure()` is used outside of `TEST()` , `TEST_F()` , or a test +fixture, you must add the `testing::Test::` prefix, as in: + +{{{ +if (testing::Test::HasFatalFailure()) + return; +}}} + + +_Availability:_ Linux, Windows, Mac. + +== Logging Additional Information == + +In your test code, you can call `RecordProperty("key", value)` to log +additional information, where `value` can be either a C string or a 32-bit +integer. The _last_ value recorded for a key will be emitted to the XML output +if you specify one. For example, the test + +{{{ +TEST_F(WidgetUsageTest, MinAndMaxWidgets) { + RecordProperty("MaximumWidgets", ComputeMaxUsage()); + RecordProperty("MinimumWidgets", ComputeMinUsage()); +} +}}} + +will output XML like this: + +{{{ +... + +... +}}} + +_Note_: + * `RecordProperty()` is a static member of the `Test` class. Therefore it needs to be prefixed with `testing::Test::` if used outside of the `TEST` body and the test fixture class. + * `key` must be a valid XML attribute name, and cannot conflict with the ones already used by Google Test (`name`, `status`, `time`, and `classname`). + +_Availability_: Linux, Windows, Mac. + +== Sharing Resources Between Tests in the Same Test Case == + +Google Test creates a new test fixture object for each test in order to make +tests independent and easier to debug. However, sometimes tests use resources +that are expensive to set up, making the one-copy-per-test model prohibitively +expensive. + +If the tests don't change the resource, there's no harm in them sharing a +single resource copy. So, in addition to per-test set-up/tear-down, Google Test +also supports per-test-case set-up/tear-down. To use it: + + # In your test fixture class (say `FooTest` ), define as `static` some member variables to hold the shared resources. + # In the same test fixture class, define a `static void SetUpTestCase()` function (remember not to spell it as *`SetupTestCase`* with a small `u`!) to set up the shared resources and a `static void TearDownTestCase()` function to tear them down. + +That's it! Google Test automatically calls `SetUpTestCase()` before running the +_first test_ in the `FooTest` test case (i.e. before creating the first +`FooTest` object), and calls `TearDownTestCase()` after running the _last test_ +in it (i.e. after deleting the last `FooTest` object). In between, the tests +can use the shared resources. + +Remember that the test order is undefined, so your code can't depend on a test +preceding or following another. Also, the tests must either not modify the +state of any shared resource, or, if they do modify the state, they must +restore the state to its original value before passing control to the next +test. + +Here's an example of per-test-case set-up and tear-down: +{{{ +class FooTest : public testing::Test { + protected: + // Per-test-case set-up. + // Called before the first test in this test case. + // Can be omitted if not needed. + static void SetUpTestCase() { + shared_resource_ = new ...; + } + + // Per-test-case tear-down. + // Called after the last test in this test case. + // Can be omitted if not needed. + static void TearDownTestCase() { + delete shared_resource_; + shared_resource_ = NULL; + } + + // You can define per-test set-up and tear-down logic as usual. + virtual void SetUp() { ... } + virtual void TearDown() { ... } + + // Some expensive resource shared by all tests. + static T* shared_resource_; +}; + +T* FooTest::shared_resource_ = NULL; + +TEST_F(FooTest, Test1) { + ... you can refer to shared_resource here ... +} +TEST_F(FooTest, Test2) { + ... you can refer to shared_resource here ... +} +}}} + +_Availability:_ Linux, Windows, Mac. + +== Global Set-Up and Tear-Down == + +Just as you can do set-up and tear-down at the test level and the test case +level, you can also do it at the test program level. Here's how. + +First, you subclass the `testing::Environment` class to define a test +environment, which knows how to set-up and tear-down: + +{{{ +class Environment { + public: + virtual ~Environment() {} + // Override this to define how to set up the environment. + virtual void SetUp() {} + // Override this to define how to tear down the environment. + virtual void TearDown() {} +}; +}}} + +Then, you register an instance of your environment class with Google Test by +calling the `testing::AddGlobalTestEnvironment()` function: + +{{{ +Environment* AddGlobalTestEnvironment(Environment* env); +}}} + +Now, when `RUN_ALL_TESTS()` is called, it first calls the `SetUp()` method of +the environment object, then runs the tests if there was no fatal failures, and +finally calls `TearDown()` of the environment object. + +It's OK to register multiple environment objects. In this case, their `SetUp()` +will be called in the order they are registered, and their `TearDown()` will be +called in the reverse order. + +Note that Google Test takes ownership of the registered environment objects. +Therefore *do not delete them* by yourself. + +You should call `AddGlobalTestEnvironment()` before `RUN_ALL_TESTS()` is +called, probably in `main()`. If you use `gtest_main`, you need to call +this before `main()` starts for it to take effect. One way to do this is to +define a global variable like this: + +{{{ +testing::Environment* const foo_env = testing::AddGlobalTestEnvironment(new FooEnvironment); +}}} + +However, we strongly recommend you to write your own `main()` and call +`AddGlobalTestEnvironment()` there, as relying on initialization of global +variables makes the code harder to read and may cause problems when you +register multiple environments from different translation units and the +environments have dependencies among them (remember that the compiler doesn't +guarantee the order in which global variables from different translation units +are initialized). + +_Availability:_ Linux, Windows, Mac. + +== Testing Private Code == + +If you change your software's internal implementation, your tests should not +break as long as the change is not observable by users. Therefore, per the +_black-box testing principle_, most of the time you should test your code +through its public interfaces. + +If you still find yourself needing to test internal implementation code, +consider if there's a better design that wouldn't require you to do so. If you +absolutely have to test non-public interface code though, you can. There are +two cases to consider: + + * Static functions (_not_ the same as static member functions!) or unnamed namespaces, and + * Private or protected class members + +=== Static functions === + +Both static functions and definitions/declarations in an unnamed namespace are +only visible within the same translation unit. To test them, you can `#include` +the entire `.cc` file being tested in your `*_test.cc` file. (#including `.cc` +files is not a good way to reuse code - you should not do this in production +code!) + +However, a better approach is to move the private code into the +`foo::internal` namespace, where `foo` is the namespace your project normally +uses, and put the private declarations in a `*-internal.h` file. Your +production `.cc` files and your tests are allowed to include this internal +header, but your clients are not. This way, you can fully test your internal +implementation without leaking it to your clients. + +=== Private class members === + +Private class members are only accessible from within the class or by friends. +To access a class' private members, you can declare your test fixture as a +friend to the class and define accessors in your fixture. Tests using the +fixture can then access the private members of your production class via the +accessors in the fixture. Note that even though your fixture is a friend to +your production class, your tests are not automatically friends to it, as they +are technically defined in sub-classes of the fixture. + +Another way to test private members is to refactor them into an implementation +class, which is then declared in a `*-internal.h` file. Your clients aren't +allowed to include this header but your tests can. Such is called the Pimpl +(Private Implementation) idiom. + +Or, you can declare an individual test as a friend of your class by adding this +line in the class body: + +{{{ +FRIEND_TEST(TestCaseName, TestName); +}}} + +For example, +{{{ +// foo.h +#include "testing/production_stub/public/gtest_prod.h" + +// Defines FRIEND_TEST. +class Foo { + ... + private: + FRIEND_TEST(FooTest, BarReturnsZeroOnNull); + int Bar(void* x); +}; + +// foo_test.cc +... +TEST(FooTest, BarReturnsZeroOnNull) { + Foo foo; + EXPECT_EQ(0, foo.Bar(NULL)); + // Uses Foo's private member Bar(). +} +}}} + +Pay special attention when your class is defined in a namespace, as you should +define your test fixtures and tests in the same namespace if you want them to +be friends of your class. For example, if the code to be tested looks like: + +{{{ +namespace my_namespace { + +class Foo { + friend class FooTest; + FRIEND_TEST(FooTest, Bar); + FRIEND_TEST(FooTest, Baz); + ... + definition of the class Foo + ... +}; + +} // namespace my_namespace +}}} + +Your test code should be something like: + +{{{ +namespace my_namespace { +class FooTest : public testing::Test { + protected: + ... +}; + +TEST_F(FooTest, Bar) { ... } +TEST_F(FooTest, Baz) { ... } + +} // namespace my_namespace +}}} + +== Death Tests == + +In many applications, there are assertions that can cause application failure +if a condition is not met. These sanity checks, which ensure that the program +is in a known good state, are there to fail at the earliest possible time after +some program state is corrupted. If the assertion checks the wrong condition, +then the program may proceed in an erroneous state, which could lead to memory +corruption, security holes, or worse. Hence it is vitally important to test +that such assertion statements work as expected. + +Since these precondition checks cause the processes to die, we call such tests +_death tests_. More generally, any test that checks that a program terminates +in an expected fashion is also a death test. + +=== How to Write a Death Test === + +Google Test has the following macros to support death tests: + +|| `ASSERT_DEATH(`_`statement, regex`_`); || `EXPECT_DEATH(`_`statement, regex`_`); || +|| `ASSERT_EXIT(`_`statement, predicate, regex`_`); || `EXPECT_EXIT(`_`statement, predicate, regex`_`); || + +where _`statement`_ is a statement that is expected to cause the process to +die, _`predicate`_ is a function or function object that evaluates an integer +exit status, and _`regex`_ is a regular expression that the stderr output of +_`statement`_ is expected to match. Note that _`statement`_ can be _any valid +statement_ (including _compound statement_) and doesn't have to be an +expression. + +As usual, the `ASSERT` variants abort the current test function, while the +`EXPECT` variants do not. + +A predicate here must accept an `int` and return a `bool`. The death test +succeeds only if the predicate returns `true`. Google Test defines a few +predicates that handle the most common cases: + +{{{ +testing::ExitedWithCode(exit_code) +}}} + +This expression is `true` if the program exited normally with the given exit +code. + +{{{ +testing::KilledBySignal(_signal_number_) +}}} + +This expression is `true` if the program was killed by the given signal. + +The `*_DEATH` macros are convenient wrappers for `*_EXIT` that use a predicate +that verifies that the process either exited with a nonzero exit code or was +killed by a signal. + +To write a death test, simply use one of the above macros inside your test +function. For example, + +{{{ +TEST(My*DeathTest*, Foo) { + // This death test uses a compound statement. + ASSERT_DEATH({ int n = 5; Foo(&n); }, "Error on line .* of Foo()"); +} +TEST(MyDeathTest, NormalExit) { + EXPECT_EXIT(NormalExit(), testing::ExitedWithCode(0), "Success"); +} +TEST(MyDeathTest, KillMyself) { + EXPECT_EXIT(KillMyself(), testing::KilledBySignal(SIGKILL), "Sending myself unblockable signal"); +} +}}} + +verifies that: + + * calling `Foo(5)` causes the process to die with the given error message, + * calling `NormalExit()` causes the process to print `"Success"` to stderr and exit with exit code 0, and + * calling `KillMyself()` kills the process with signal `SIGKILL`. + +The test function body may contain other assertions and statements as well, if +necessary. + +_Important:_ We strongly recommend you to follow the convention of naming your +test case (not test) `*DeathTest` when it contains a death test, as +demonstrated in the above example. The `Death Tests And Threads` section below +explains why. + +_Availability:_ Linux + +=== How It Works === + +Under the hood, `ASSERT_EXIT()` calls `fork()` to spawn a new process. What +happens next in the child process depends on the value of the variable +`testing::FLAGS_gtest_death_test_style` (which is initialized from the +command-line flag `--gtest_death_test_style`). + + * If the variable's value is `"fast"`, the death test statement is immediately executed. + * If the variable's value is `"threadsafe"`, the child process re-executes the unit test binary just as it was originally invoked, but with some extra flags to cause just the single death test under consideration to be run. + +Other values for the variable are illegal and will cause the death test to +fail. Currently, the flag's default value is `"fast"`. However, we reserve the +right to change it in the future. Therefore, your tests should not depend on +this. + +In either case, the parent process waits for the child process to complete, and checks that + + # the child's exit status satisfies the predicate, and + # the child's stderr matches the regular expression. + +If the death test statement runs to completion without dying, the child +process will nonetheless terminate, and the assertion fails. + +=== Death Tests And Threads === + +The reason for the two death test styles has to do with thread safety. Due to +well-known problems with forking in the presence of threads, death tests should +be run in a single-threaded context. Sometimes, however, it isn't feasible to +arrange that kind of environment. For example, statically-initialized modules +may start threads before main is ever reached. Once threads have been created, +it may be difficult or impossible to clean them up. + +Google Test has two features intended to raise awareness of threading issues. + + # A warning is emitted if multiple threads are running when a death test is encountered. + # Test cases with a name ending in "!DeathTest" are run before all other tests. + +It's perfectly fine to create threads inside a death test statement; they are +executed in a separate process and cannot affect the parent. + +=== Death Test Styles === + +The "threadsafe" death test style was introduced in order to help mitigate the +risks of testing in a possibly multithreaded environment. It trades increased +test execution time (potentially dramatically so) for improved thread safety. +We suggest using the faster, default "fast" style unless your test has specific +problems with it. + +You can choose a particular style of death tests by setting the flag +programmatically: + +{{{ +testing::FLAGS_gtest_death_test_style = "threadsafe"; +}}} + +You can do this in `main()` to set the style for all death tests in the +binary, or in individual tests. Recall that flags are saved before running each +test and restored afterwards, so you need not do that yourself. For example: + +{{{ +TEST(MyDeathTest, TestOne) { + testing::FLAGS_gtest_death_test_style = "threadsafe"; + // This test is run in the "threadsafe" style: + ASSERT_DEATH(ThisShouldDie(), ""); +} + +TEST(MyDeathTest, TestTwo) { + // This test is run in the "fast" style: + ASSERT_DEATH(ThisShouldDie(), ""); +} + +int main(int argc, char** argv) { + ParseGTestFlags(argv[0], &argc, &argv, true); + testing::FLAGS_gtest_death_test_style = "fast"; + return RUN_ALL_TESTS(); +} +}}} + +=== Caveats === + +The _`statement`_ argument of `ASSERT_EXIT()` can be any valid C++ statement +except that it can not return from the current function. This means +_`statement`_ should not contain `return` or a macro that might return (e.g. +`ASSERT_TRUE()` ). If _`statement`_ returns before it crashes, Google Test will +print an error message, and the test will fail. + +Since _`statement`_ runs in the child process, any in-memory side effect (e.g. +modifying a variable, releasing memory, etc) it causes will _not_ be observable +in the parent process. In particular, if you release memory in a death test, +your program will fail the heap check as the parent process will never see the +memory reclaimed. To solve this problem, you can + + # try not to free memory in a death test; + # free the memory again in the parent process; or + # do not use the heap checker in your program. + +Due to an implementation detail, you cannot place multiple death test +assertions on the same line; otherwise, compilation will fail with an unobvious +error message. + +Despite the improved thread safety afforded by the "threadsafe" style of death +test, thread problems such as deadlock are still possible in the presence of +handlers registered with `pthread_atfork(3)`. A future implementation will +address this. + +Because the child process in an threadsafe-style death test re-executes the +test binary by passing `argv[0]` to `execve(2)`, the test binary may not be +found if the program was executed via a relative path and any preceding program +code changes the program's working directory. + +== Getting the Current Test's Name == + +Sometimes a function may need to know the name of the currently running test. +For example, you may be using the `SetUp()` method of your test fixture to set +the golden file name based on which test is running. The `testing::TestInfo` +class has this information: + +{{{ +namespace testing { + +class TestInfo { + public: + // Returns the test case name and the test name, respectively. + // + // Do NOT delete or free the return value - it's managed by the + // TestInfo class. + const char* test_case_name() const; + const char* name() const; +}; + +} // namespace testing +}}} + + + To obtain a `TestInfo` object for the currently running test, call +`current_test_info()` on the `UnitTest` singleton object: + +{{{ +// Gets information about the currently running test. +// Do NOT delete the returned object - it's managed by the UnitTest class. +const testing::TestInfo* const test_info = + testing::UnitTest::GetInstance()->current_test_info(); +printf("We are in test %s of test case %s.\n", + test_info->name(), test_info->test_case_name()); +}}} + +`current_test_info()` returns a null pointer if no test is running. In +particular, you cannot find the test case name in `TestCaseSetUp()`, +`TestCaseTearDown()` (where you know the test case name implicitly), or +functions called from them. + +_Availability:_ Linux, Windows, Mac. + +== Running Test Programs: Advanced Options == + +Google Test test programs are ordinary executables. Once built, you can run +them directly and affect their behavior via the following environment variables +and/or command line flags. For the flags to work, your programs must call +`testing::ParseGTestFlags()` before calling `RUN_ALL_TESTS()`. + +If an option is specified both by an environment variable and by a flag, the +latter takes precedence. + +=== Turning Assertion Failures into Break-Points: `GTEST_BREAK_ON_FAILURE` and `--gtest_break_on_failure` === + +When running test programs under a debugger, it's very convenient if the +debugger can catch an assertion failure and automatically drop into interactive +mode. Google Test's _break-on-failure_ mode supports this behavior. + +To enable it, set the `GTEST_BREAK_ON_FAILURE` environment variable to a value +other than `0` . Alternatively, you can use the `--gtest_break_on_failure` +command line flag. + +_Availability:_ Linux, Windows, Mac. + +=== Suppressing Pop-ups Caused by Exceptions: `GTEST_CATCH_EXCEPTIONS` and `--gtest_catch_exceptions` === + +On Windows, Google Test may be used with exceptions enabled. Even when +exceptions are disabled, an application can still throw structured exceptions +(SEH's). If a test throws an exception, by default Google Test doesn't try to +catch it. Instead, you'll see a pop-up dialog, at which point you can attach +the process to a debugger and easily find out what went wrong. + +However, if you don't want to see the pop-ups (for example, if you run the +tests in a batch job), set the `GTEST_CATCH_EXCEPTIONS` environment variable to +a non- `0` value, or use the `--gtest_catch_exceptions` flag. Google Test now +catches all test-thrown exceptions and logs them as failures. + +_Availability:_ Windows. `GTEST_CATCH_EXCEPTIONS` and +`--gtest_catch_exceptions` have no effect on Google Test's behavior on Linux or +Mac, even if exceptions are enabled. It is possible to add support for catching +exceptions on these platforms, but it is not implemented yet. + +=== Running a Subset of the Tests: `GTEST_FILTER` and `--gtest_filter` === + +By default, a Google Test program runs all tests the user has defined. +Sometimes, you want to run only a subset of the tests (e.g. for debugging or +quickly verifying a change). If you set the `GTEST_FILTER` environment variable +or the `--gtest_filter` flag to a filter string, Google Test will only run the +tests whose full names (in the form of `TestCaseName.TestName`) match the +filter. + +The format of a filter is a '`:`'-separated list of wildcard patterns (called +the positive patterns) optionally followed by a '`-`' and another +'`:`'-separated pattern list (called the negative patterns). A test matches the +filter if and only if it matches any of the positive patterns but does not +match any of the negative patterns. + +A pattern may contain `'*'` (matches any string) or `'?'` (matches any single +character). For convenience, the filter `'*-NegativePatterns'` can be also +written as `'-NegativePatterns'`. + +For example: + + * `./foo_test` Has no flag, and thus runs all its tests. + * `./foo_test --gtest_filter=*` Also runs everything, due to the single match-everything `*` value. + * `./foo_test --gtest_filter=FooTest.*` Runs everything in test case `FooTest`. + * `./foo_test --gtest_filter=*Null*:*Constructor*` Runs any test whose full name contains either `"Null"` or `"Constructor"`. + * `./foo_test --gtest_filter=-*DeathTest.*` Runs all non-death tests. + * `./foo_test --gtest_filter=FooTest.*-FooTest.Bar` Runs everything in test case `FooTest` except `FooTest.Bar`. + +_Availability:_ Linux, Windows, Mac. + +=== Listing Test Names: `--gtest_list_tests` === + +Sometimes it is necessary to list the available tests in a program before +running them so that a filter may be applied if needed. Including the flag +`--gtest_list_tests` overrides all other flags and lists tests in the following +format: +{{{ +TestCase1. + TestName1 + TestName2 +TestCase2. + TestName +}}} + +None of the tests listed are actually run if the flag is provided. There is no +corresponding environment variable for this flag. + +_Availability:_ Linux, Windows, Mac. + +=== Generating an XML Report: GTEST_OUTPUT and `--gtest_output` === + +Google Test can emit a detailed XML report to a file in addition to its normal +textual output. The report contains the duration of each test, and thus can +help you identify slow tests. + +To generate the XML report, set the `GTEST_OUTPUT` environment variable or the +`--gtest_output` flag to the string `"xml:_path_to_output_file_"`, which will +create the file at the given location. You can also just use the string +`"xml"`, in which case the output can be found in the `test_detail.xml` file in +the current directory. + +If you specify a directory (for example, `"xml:output/directory/"` on Linux or +`"xml:output\directory\"` on Windows), Google Test will create the XML file in +that directory, named after the test executable (e.g. `foo_test.xml` for test +program `foo_test` or `foo_test.exe`). If the file already exists (perhaps left +over from a previous run), Google Test will pick a different name (e.g. +`foo_test_1.xml`) to avoid overwriting it. + +The report uses the format described here. Since that format was originally +intended for Java, a little interpretation is required to make it apply to +Google Test tests, as shown here: + +{{{ + + + + + + + + + +}}} + + * The unnamed outer `` element corresponds to the entire test program. + * Inner elements correspond to Google Test test cases. + * elements correspond to Google Test tests. + +For instance, the following program + +{{{ +TEST(MathTest, Addition) { ... } +TEST(MathTest, Subtraction) { ... } +TEST(LogicTest, NonContradiction) { ... } +}}} + +could generate this report: + +{{{ + + + + + + + + + + + + + + + +}}} + +Things to note: + + * The `tests` attribute of a `` element tells how many tests the Google Test test case or program contains, while the `failures` attribute tells how many of them failed. + * The `time` attribute expresses the duration of the test, test case, or entire test program in milliseconds. + * Each `` element corresponds to a single failed Google Test assertion. + * Some JUnit concepts don't apply to Google Test, yet we have to conform to the DTD. Therefore you'll see some dummy elements and attributes in the report. You can safely ignore these parts. + +_Availability:_ Linux, Windows, Mac. + +=== Colored Terminal Output: `GTEST_COLOR` and `--gtest_color` === + +Google Test can use colors in its terminal output to make it easier to spot +the separation between tests, and whether tests passed. + +You can set the GTEST_COLOR environment variable or set the `--gtest_color` +command line flag to `yes`, `no`, or `auto` (the default) to enable colors, +disable colors, or let Google Test decide. When the value is `auto`, Google +Test will use colors if and only if the output goes to a terminal and (on +non-Windows platforms) the `TERM` environment variable is set to `xterm` or +`xterm-color`. + +_Availability:_ Linux, Windows, Mac. + +=== Repeating the Tests: `GTEST_REPEAT` and `--gtest_repeat` === + +Once in a while you'll run into a test whose result is hit-or-miss. Perhaps it +will fail only 1% of the time, making it rather hard to reproduce the bug under +a debugger. This can be a major source of frustration. + +The `--gtest_repeat` flag allows you to repeat all (or selected) test methods +in a program many times. Hopefully, a flaky test will eventually fail and give +you a chance to debug. Here's how to use it: + +|| `$ foo_test --gtest_repeat=1000` || Repeat foo_test 1000 times and don't stop at failures. || +|| `$ foo_test --gtest_repeat=-1` || A negative count means repeating forever. || +|| `$ foo_test --gtest_repeat=1000 --gtest_break_on_failure` || Repeat foo_test 1000 times, stopping at the first failure. This is especially useful when running under a debugger: when the testfails, it will drop into the debugger and you can then inspect variables and stacks. || +|| `$ foo_test --gtest_repeat=1000 --gtest_filter=FooBar` || Repeat the tests whose name matches the filter 1000 times. || + +You can also specify the repeat count by setting the `GTEST_REPEAT` environment +variable. + +_Availability:_ Linux, Windows, Mac. + +=== Temporarily Disabling Tests === + +If you have a broken test that you cannot fix right away, you can add the +`DISABLED_` prefix to its name. This will exclude it from execution. This is +better than commenting out the code or using `#if 0`, as disabled tests are +still compiled (and thus won't rot). + +If you need to disable all tests in a test case, you can either add `DISABLED_` +to the front of the name of each test, or alternatively add it to the front of +the test case name. + +For example, the following tests won't be run by Google Test, even though they +will still be compiled: + +{{{ +// Tests that Foo does Abc. +TEST(FooTest, DISABLED_DoesAbc) { ... } + +class DISABLED_BarTest : public testing::Test { ... }; + +// Tests that Bar does Xyz. +TEST_F(DISABLED_BarTest, DoesXyz) { ... } +}}} + +_Note:_ This feature should only be used for temporary pain-relief. You still +have to fix the disabled tests at a later date. As a reminder, Google Test will +print a banner warning you if a test program contains any disabled tests. + +_Tip:_ You can easily count the number of disabled tests you have using `gsearch` +and/or `grep`. This number can be used as a metric for improving your test +quality. + +_Availability:_ Linux, Windows, Mac. + +== Where to Go from Here == + +Congratulations! You've now learned more advanced Google Test tools and are +ready to tackle more complex testing tasks. If you want to dive even deeper, you +can read the GoogleTestFAQ. \ No newline at end of file