summaryrefslogtreecommitdiff
path: root/test
Commit message (Collapse)AuthorAge
* Add tests for author name reordering in search resultsDirk Hohndel2010-04-26
| | | | | | This should be required for all patches :-) Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
* test: Comment why we need to set TZMichal Sojka2010-04-26
|
* fix expected test result to include Bcc linesDirk Hohndel2010-04-23
| | | | | | | | | | | | | | | | this test actually tests behavior that I consider as broken. The Bcc should be to the same address as used in the From line, otherwise we are creating a potential information leak as email that is related to one email account (say, work) is copied to a different account Signed-off-by: Dirk Hohndel <hohndel@infradead.org> Reviewed-by: Carl Worth <cworth@cworth.org> These tests don't actually pass yet, since the feature being tested has not been merged. But gettting these tests in first will let us more easily test that the feature actually works, (and will help us ensure we don't forget the feature before the next release).
* add From guessing tests to test suiteDirk Hohndel2010-04-23
| | | | | | | | right now these are not trying to be overly fancy simply one test per strategy that we apply to figure out the best from address - including the fallback if there's nothing to go on Signed-off-by: Dirk Hohndel <hohndel@infradead.org>
* test: Set fixed time zoneMichal Sojka2010-04-23
| | | | | | | | | | | | | | | | | | | | | | | When the test suite is run in a different time zone that where Carl lives, some tests may fail depending on the time when the test suite is run. For example, just now I get: Search for all messages ("*"):... FAIL --- test-031.expected 2010-04-23 09:33:47.898634822 +0200 +++ test-031.output 2010-04-23 09:33:47.898634822 +0200 @@ -1,5 +1,5 @@ -thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #6 (inbox unread) -thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Test message #14 (inbox unread) +thread:XXX 2001-01-06 [1/1] Notmuch Test Suite; Test message #6 (inbox unread) +thread:XXX 2001-01-06 [1/1] Notmuch Test Suite; Test message #14 (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (inbox unread) thread:XXX 2000-01-01 [1/1] searchbyfrom; search by from (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread) By setting a fixed time zone in the test script, these problems should be eliminated. Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
* test: Put the json tests into their own section.Carl Worth2010-04-22
| | | | | Not that the sections actually mean anything yet, but it makes for clean output.
* First tests for JSON output and UTF-8 in mail body and subjectGregor Hoffleit2010-04-22
| | | | | | | | | | | | | | | | | | | | The test suite doesn't yet cover --format=json output nor UTF-8 in subject or body. This patch starts with test cases for 'search --format=json' and 'show --format=json'. Furthermore, it has test cases for a search for a UTF-8 string in a mail body for a UTF-8 string in a mail subject. Finally, it has a test case for --format=json with UTF-8 messages, demonstrating the fix in 1267697893-sup-4538@sam.mediasupervision.de. Reviewed-by: Carl Worth <cworth@cworth.org> Updated tests to current implementation of the test suite. These tests demonstrate a bug in the current implementation of "notmuch show --format=json", (timestamp output is changed depending on current timezone).
* test: Make existing "notmuch show" test more resilientCarl Worth2010-04-22
| | | | | | | If future updates to the test suite add more messages to the database before this "notmuch show" test, then the message-ID numbers in the expected output will all change. But we can at least compute the numbers so that this test will continue to pass.
* test: Add a test to demonstrate message-sorting regression.Carl Worth2010-04-22
| | | | | | | | | | | In the recent change to rename threads based on changing subject lines, I broke message ordering within "notmuch show" output. But our test suite didn't catch that regressions, because we didn't have any tests of "notmuch show". This adds one "notmuch show" test along with the thread-naming tests. It's not a whole suite of "notmuch show" testing, but it does catch this regression at least.
* test: When a test fails, show diff only (save complete output to a file)Carl Worth2010-04-22
| | | | | | | | We're starting to get test output that's fairly long, so it's much kinder to just show a diff rather than displaying the complete expected and actual output. To allow the user to investigate things after the fact, we save the expected and actual output to files named test-${test_number}.expected and test-${test_number}.output .
* test: Improve grammar in test-message subjects.Carl Worth2010-04-22
| | | | | Before codifying these in any expected test output, I'd like them to at least resemble English.
* test: Add tests for naming threads with changing subjects.Carl Worth2010-04-21
| | | | | | | | We recently added a feature to name threads based on the messages that actually matched the search, (as opposed to simply the oldest or newest message in the thread whether it matched or not). So add tests for that, and (surprise, surprise!) the feature does not entirely work.
* make test: Actually count and report on failures.Carl Worth2010-04-15
| | | | | | | | Hurrah---no more manual verification of that PASS column. This means that "make test" can actually be a useful part of the release process now, (since it will exit with non-zero status if there are any failures).
* test: Unify all tests to use the pass_if_equal function.Carl Worth2010-04-15
| | | | | | | | | | | Previously some tests (dump/restore) were doing ad-hoc verification of values and their own printing of PASS/FAIL, etc. This made it impossible to count test pass/fail rates in a single place. The only reason these tests were written that way was because the old execute_expecting function only worked if one could directly test the stdout output of a notmuch command. The recent switch to pass_if_equal means that all tests can use it.
* test: Align the PASS/FAIL column.Carl Worth2010-04-15
| | | | Just to make things a little more pleasant to look at.
* test: Add a test for "notmuch search '*'"Carl Worth2010-04-15
| | | | | | | | | This feature was added recently and should have gotten a new test at the time. As this test demonstrates, the code is broken, ("notmuch search '*' returns bogus dates of the Unix epoch for any threads where the term "and" does not appear in any messages).
* test: Use an older date for the generated messages.Carl Worth2010-04-15
| | | | | | Using a date in the current year makes the test suite fragile since the search output will include a date of "January 05" for now, but will start doing "2010-01-05" in the future.
* test: Remove test-message filenames from generated messagesCarl Worth2010-04-15
| | | | | | | | | | The filenames aren't predictable (including the current directory) nor stable from one run to the next (including the PID). This makes it hard to predict the output from a search command that returns such a message (such as "*"). The original goal was simply to ensure that each generated message was distinguishable somehow. So just use the message counter instead.
* test: Disentangle execution of notmuch from the examination of the results.Carl Worth2010-04-15
| | | | | | | | | | | | | | The old execute_expecting function was doing far too much for its own good. One of the worst aspects of this was that it introduced shell-quoting challengers where the caller could not easily control the precise invocation of the command to be executed. I personally couldn't find a way to test "notmuch search '*'" without the shell expanding * against files in the current directory, or having bogus quotation marks appearing in the search string, (defeating the recognition of "*" as a special search term). Hopefully this aspect of the test suite will be much easier to maintain now.
* test: Fix reply tests to track insignificant change in output.Carl Worth2010-04-14
| | | | | | | | | | The recent fix to properly decode encoded headers made the expected output of "notmuch reply" differ by a single space, (previously, there were two spaces before the References: value and now there is just one). Fix the test suite so that these are all noted as correct results again.
* test: Add new tests for out-of-order messages.James Westby2010-04-12
| | | | | | | | | | | | | | | | | | | | | These new tests demonstrate a bug as follows: Multiple messages are added to the database All of these message references a common parent The parent message does not exist in the databas In this scenario, the messages will not be recognized as belonging to the same thread. We consider this a bug, and the new tests treat this as a failure. Edited by Carl Worth <cworth@cworth.org>: Split these tests into their own commit (before the fix of the bug). This lets me see the actual failure in the test suite, before the fix is applied. Also fix the alignment of new messages from test suite, (so that the PASS portions all line up---which is important while we're still manually verifying test-suite results).
* test: Fix phrase-search tests.Carl Worth2010-03-31
| | | | | With some extra qutotation marks, we are now doing actual phrase searches so these tests pass.
* test: Add some negative results for the phrase searches.Carl Worth2010-03-31
| | | | | | | | | These results have all the same terms as the target phrase, but not in the expected order. They are designed to ensure that we actually test phrase searches. And as it turns out, we're not currently quoting the search terms properly, so the phrase-search tests now fail with this commit.
* Switch from random to sequential thread identifiers.Carl Worth2010-02-09
| | | | | | | | | | | | | The sequential identifiers have the advantage of being guaranteed to be unique (until we overflow a 64-bit unsigned integer), and also take up half as much space in the "notmuch search" output (16 columns rather than 32). This change also has the side effect of fixing a bug where notmuch could block on /dev/random at startup (waiting for some entropy to appear). This bug was hit hard by the test suite, (which could easily exhaust the available entropy on common systems---resulting in large delays of the test suite).
* notmuch-test: Add basic tests for "notmuch dump" and "notmuch restore"Carl Worth2010-02-05
| | | | Our test-suite coverage keeps improving, (slowly).
* notmuch-test: Fix misalignment in output.Carl Worth2010-02-05
| | | | Clean output is just so much easier to read (and validate).
* notmuch-test: Add test to verify that uuencoded data is not indexed.Carl Worth2010-02-05
| | | | | As recently promised, no new features are being accepted to notmuch without corresponding new tests for the test suite.
* notmuch-test: Add several tests of "notmuch search"Carl Worth2010-02-05
| | | | | | These tests were surprisingly simple to write---not much code at all and most of them worked the first time even with hand-prepared versions of the expected output.
* notmuch-test: Add a new add_message function.Carl Worth2010-02-04
| | | | | | | The previous generate_message function is what's needed when testing "notmuch new". But after that, we never want to generate a message without also adding it to the index. So create a new add_message function with this convenience.
* notmuch-test: Test "notmuch reply" with a reply-to-munged mailCarl Worth2010-02-04
| | | | | | | | This is a test for the recently added feature where we detect that the reply-to address already exists in the To: or Cc: header so will already be replied to. In this case we want to include the From: address in our reply, (where, otherwise we would use the Reply-To address *instead* of the address in the From header).
* notmuch-test: Test "notmuch reply" with multiple recipientsCarl Worth2010-02-04
| | | | | | The feature tested here is that we reply to both the sender and to others addresses on the To: line of the original message, but that we don't reply to our own address.
* notmuch-test: Add test for "notmuch reply" with a CC header.Carl Worth2010-02-04
| | | | A simple test to help round out the set of tested features.
* notmuch-test: Add "notmuch reply" test for reply-to support.Carl Worth2010-02-04
| | | | | | | This is the standard support of reply-to, (replying to that address rather than the from address). It has nothing to do with the proposed feature for extra-clever handling of a mail from a mailing-list that has munged the reply-to header.
* notmuch-test: Test "notmuch reply" from alternate address.Carl Worth2010-02-04
| | | | | | When reply to a message addresses to an address configured in the other_email setting in the configuration file, the reply should use that address in the From header. Test this.
* notmuch-test: Add a basic test of "notmuch reply"Carl Worth2010-02-04
| | | | Simply ensuring that the reply template is formatted as expected.
* notmuch-test: Eliminate sleeps to speed up test suite runCarl Worth2010-02-04
| | | | | | | | | | | | We were sleeping merely to ensure that our updates to the mail store would result in the mtime of the appropriate directories being updated. We make the test suite run faster by not sleeping, but instead explicitly updating the mtime of the directory to a future time with touch. We're careful to ensure that the time is not merely in the future compared to the current time, but also later than any previous update to the same directory mtime.
* notmuch-test: Allow custom headers when generating messagesCarl Worth2010-02-04
| | | | | | This provides the control that future tests will need, (for example, adding a CC field to ensure proper handling with "notmuch reply", etc.)
* notmuch-test: Use named-parameters for generate_message functionCarl Worth2010-02-04
| | | | | | | | | | This makes the test suite bash-specific, but that's not much of an issue for me, (if somebody else would prefer some other language then they can rewrite the test suite and maintain it). The advantage here is that we'll now be able to easily generate custom messages for testing operations that depend on the message content, (such as "notmuch reply", etc.).
* Add actual testing to notmuch-testCarl Worth2010-02-04
| | | | | | We verify that each command creates output exactly as expected (after ignoring variable output such as the report of how long an operation took).
* Add a very rough beginning of a test suite.Carl Worth2010-01-24
This notmuch-test script simply runs a few different notmuch operations, (things that I found were useful while testing the rename-support code). It's not useful as a test suite yet, since it doesn't actually check the results of any operation, (the user of the suite has to know what the results should be and must manually verify them. So there's no integration with the build system yet, (no "make test" target). But I didn't want to lose what I had so far, so here it is.
generated by cgit v1.2.3 (git 2.39.1) at 2024-05-20 11:03:52 +0000