El RIDO 66cee9dbd2 adding icon generator comparison test script for reference in #148 3 days ago
..
Data 02f3cc739f documentation on fnv1a64 is lacking, but tests show it was only introduced with PHP 5.6 1 month ago
Persistence 4a35428499 cleanup of PurgeLimiter #342 10 months ago
Bootstrap.php 09162a3c57 fix display of v2 pastes in JS, fixing parsing of comments in PHP, avoid exposing expiration date (we provide time_to_live, would allow calculation of creation date of paste) 1 month ago
ConfigurationTest.php 478cf288b4 implementing StyleCI recommendations 1 year ago
ConfigurationTestGenerator.php 700f8a0ea7 made all php unit tests pass again 1 month ago
ControllerTest.php d785ada3fe apply StyleCI patch 1 month ago
ControllerWithDbTest.php 09162a3c57 fix display of v2 pastes in JS, fixing parsing of comments in PHP, avoid exposing expiration date (we provide time_to_live, would allow calculation of creation date of paste) 1 month ago
FilterTest.php 823adb78ef bumping required PHP to 5.4, removing unneccessary code, resolves #186 2 years ago
FormatV2Test.php 1e44902340 apply StyleCI patch 1 month ago
I18nTest.php be1e7babc0 removing dead code and improving code coverage 1 month ago
IconTest 66cee9dbd2 adding icon generator comparison test script for reference in #148 3 days ago
JsonApiTest.php cc1c55129f switching to full JSON API without POST array use, ensure all JSON operations are done with error detection 1 month ago
ModelTest.php be1e7babc0 removing dead code and improving code coverage 1 month ago
README.md dfb16ea61d switching to JS code coverage nyc, maintained fork of istanbul 1 month ago
RequestTest.php cc1c55129f switching to full JSON API without POST array use, ensure all JSON operations are done with error detection 1 month ago
ViewTest.php d6f203dc4c Removed option to hide clone button on expiring pastes, since this requires reading the paste for rendering the template, which leaks information on the pastes state 1 year ago
Vizhash16x16Test.php 1f46823942 applying patch based on StyleCI ruleset 2 years ago
phpunit.xml fdef8bc5be starting to work on JSVerify & Mocha based unit tests for our JS code base 2 years ago

README.md

Running all unit tests

Since it is non-trivial to setup all dependencies for our unit testing suite, we provide a docker image that bundles all of them into one container, both phpunit for PHP and mocha for JS.

You can fetch and run the image from the docker hub like this:

docker run --rm --read-only -v ~/PrivateBin:/srv:ro privatebin/unit-testing

The parameters in detail:

  • -v ~/PrivateBin:/srv:ro - Replace ~/PrivateBin with the location of the checked out PrivateBin repository on your machine. It is recommended to mount it read-only, which guarantees that your repository isn't damaged by an accidentally destructive test case in it.
  • --read-only - This image supports running in read-only mode. Only /tmp may be written into.
  • -rm - Remove the container after the run. This saves you doing a cleanup on your docker environment, if you run the image frequently.

You can also run just the php and javascript test suites instead of both:

docker run --rm --read-only -v ~/PrivateBin:/srv:ro privatebin/unit-testing phpunit
docker run --rm --read-only -v ~/PrivateBin:/srv:ro privatebin/unit-testing mocha

We also provide a Janitor image that includes the Cloud9 and Theia WebIDEs as well as the integrated unit testing utilities. See our docker wiki page for further details on this.

Running PHP unit tests

In order to run these tests, you will need to install the following packages and their dependencies:

  • phpunit
  • php-gd
  • php-sqlite3
  • php-xdebug (for code coverage reports)

Example for Debian and Ubuntu:

$ sudo apt install phpunit php-gd php-sqlite3 php-xdebug

To run the tests, change into the tst directory and run phpunit:

$ cd PrivateBin/tst
$ phpunit

Additionally there is the ConfigurationTestGenerator. Based on the configurations defined in its constructor, it generates the unit test file tst/ConfigurationCombinationsTest.php, containing all possible combinations of these configurations and tests for (most of the) valid combinations. Some of combinations can't be tested with this method, i.e. a valid option combined with an invalid one. Other very specific test cases (i.e. to trigger multiple errors) are covered in tst/PrivateBinTest.php. Here is how to generate the configuration test and run it:

$ cd PrivateBin/tst
$ php ConfigurationTestGenerator.php
$ phpunit ConfigurationCombinationsTest.php

Note that it can take an hour or longer to run the several thousand tests.

Running JavaScript unit tests

In order to run these tests, you will need to install the following packages and its dependencies:

  • npm

Then you can use the node package manager to install the latest stable release of mocha and nyc (for code coverage reports) globally and jsVerify, jsdom and jsdom-global locally:

$ npm install -g mocha nyc
$ cd PrivateBin/js
$ npm install jsverify jsdom@9 jsdom-global@2 mime-types node-webcrypto-ossl

Example for Debian and Ubuntu, including steps to allow the current user to install node modules globally:

$ sudo apt install npm
$ sudo mkdir /usr/local/lib/node_modules
$ sudo chown -R $(whoami) $(npm config get prefix)/{lib/node_modules,bin,share}
$ ln -s /usr/bin/nodejs /usr/local/bin/node
$ npm install -g mocha nyc
$ cd PrivateBin/js
$ npm install jsverify jsdom@9 jsdom-global@2 mime-types node-webcrypto-ossl

To run the tests, just change into the js directory and run istanbul:

$ cd PrivateBin/js
$ nyc mocha

Property based unit testing

In the JavaScript unit tests we use the JSVerify library to leverage property based unit testing. Instead of artificially creating specific test cases to cover all relevant paths of the tested code (with the generated coverage reports providing means to check the tested paths), property based testing allows us to describe the patterns of data that are valid input.

With each run of the tests, for each jsc.property 100 random inputs are generated and tested. For example we tell the test to generate random strings, which will include empty strings, numeric strings, long strings, unicode sequences, etc. This is great for finding corner cases that one might not think of when explicitly writing one test case at a time.

There is another benefit, too: When an error is found, JSVerify will try to find the smallest, still failing test case for you and print this out including the associated random number generator (RNG) state, so you can reproduce it easily:

[...]

  30 passing (3s)
  1 failing

  1) Helper getCookie returns the requested cookie:
     Error: Failed after 30 tests and 11 shrinks. rngState: 88caf85079d32e416b; Counterexample: ["{", "9", "9", "YD8%fT"]; [" ", "_|K:"]; 

[...]

Of course it may just be that you need to adjust a test case if the random pattern generated is ambiguous. In the above example the cookie string would contain two identical keys "9", something that may not be valid, but that our code could encounter and needs to be able to handle.

After you adjusted the code of the library or the test you can rerun the test with the same RNG state as follows:

$ nyc mocha test --jsverifyRngState 88caf85079d32e416b