I already had axios installed for a Node.
js HTTP client.
I was using Jest for test assertions.
Ultimately I was just using SuperTest as a HTTP client only.
SuperTest is a great module and I suggest you give it a try if you think it is what you are after, however I didn’t need it.
I ditched SuperTest and decided to build my black-box integration tests using only axios as the request client.
The Final SolutionIt took me a few hours to get all the pieces working together.
When I finally got the testing process working it involved nine different files.
For anyone wanting to duplicate this process you will find all the code you need to repeat it below.
Again, this is not a tutorial however with a little bit of gumption you will ascend to black-box testing.
1.
Testing WorkflowBefore discussing the files and configuration it is essential to understand the testing workflow.
Here are the steps I am using for black-box testing my Web API:Black-box Testing WorkflowThere is a lot going on in the above flow diagram so I’ll expand the steps below.
Open this article in another browser window if you would like to reference the diagram whilst reading the steps:Integration tests are initiated by running npm test.
The Jest setup process starts and reads the jest.
config.
js file.
The global-setup.
js module is executed as referenced in the jest.
config.
js file.
This module is rather simple for now and only executes db-setup.
js.
db-setup.
js launches an in-memory MongoDB instance, connects to the database, and adds required documents for authentication.
Now the Jest setup process spawns a child process for environment and test execution.
The jest.
config.
js file includes a testEnvironment key that points to the mongo-environment.
js module.
The Jest child process runs the setup() function from that module.
Jest scans the project looking for .
test.
js testing suite files.
Each test suite module includes a beforeAll() function which is executed before any tests.
From within beforeAll() the custom database driver is connected to the MongoDB in-memory instance.
Also from within beforeAll() the http-setup.
js module is executed.
This module launches the http server listener and authenticates against the user account added in db-setup.
js.
It sets the authentication cookie as an axios.
defaults.
We finally get to running API tests.
Any tests within the testing suite are carried out within the Jest child process.
At the end of the test suite file is the afterAll() function which is executed.
This function closes the http server listener and the database driver is disconnected.
The final task of the Jest child process is to execute the teardown() function from the mongo-environment.
js module.
Lastly the parent Jest process runs the gloabl-teardown.
js module which stops the MongoDB in-memory instance.
Now that you have an understanding of the testing process, lets have a look at how it’s done.
2.
DependenciesUse npm to install the following dependencies prior to building the testing solution:jestjest-environment-nodemongodb-memory-serveraxiosThere are many more dependencies such as express, however they should already be installed for your application.
Save them as devDependencies unless you wish to use axios in production.
3.
NPM Test ScriptYou will need to create or update the test script in your package.
json file.
Here is a simple example:This is not the package.
json file I am using.
I generated it just to show the test script on line 8.
Notice the environment variables I have set.
You don’t need these.
You could edit or delete them so that line 8 looks like this:“test”: “jest –watch”4.
Jest ConfigurationYou will need to create a Jest config file in the root of your project if you don’t already have one.
Here is the one I am using:The three files referenced in the Jest config will be discussed below.
You will need to make sure your globalSetup, globalTeardown, and testEnvironment paths are correct.
5.
Global SetupThe global-setup.
js module is executed by the Jest setup process.
Jest knows where to find the module file from reading the jest.
config.
js file.
For my use case the global-setup.
js module is rather simple.
It only executes the db-setup.
js module.
Here is the contents of the global-setup.
js file:One thing to note about the Jest global setup process is that you can’t set global variable values for use in test suites.
Tests are executed in a child process, not the Jest parent process.
This is the reason for the testEnvironment Jest configuration which you will see below.
6.
Database SetupThis is an important step in the setup process.
The db-setup.
js module gets executed by the global-setup.
js module.
Before taking a look at the database setup step it is worth turning back to Yoni’s best practice article.
In number 9.
Avoid global test fixtures and seeds, add data per-test he recommends avoiding database initialization tasks.
This is impossible in this example because each API call is required to be authenticated and authorized.
The authentication is carried out against a user account document in the database and the authorization is based on policy documents.
Due to this limitation the database needs some content for the tests to function.
Here is the contents of the db-setup.
js file:Line 10 creates a new in-memory MongoDB instance without starting it.
Line 14 exports the dbSetup function.
Line 19 to 22 creates a config object that holds the MongoDB connection URI.
This URI is generated and is different for every test run.
Line 24 connects the custom database driver to the MongoDB in-memory instance.
Line 25 executes an internal module called initialize-db.
js.
This module is required as the Web API requires authentication and authorization to access it.
The database needs to contain a test user account and access policy documents.
I’m not going to show the contents of this module because it is specific to this application.
Line 28 writes the MongoDB configuration to a JSON file ,globalConfig.
json, on the system.
The global setup process executes in the context of the Jest setup process.
To connect to the database in the test files, which are in a child process, the test suites will need to know the MongoDB connection URI.
This config file persists the URI for later retrieval by the test suites.
Line 31 sets a global variable __MONGOD__ to the MongoDB instance object.
This is used later in the global-teardown.
js module to stop the process.
7.
Test EnvironmentThe Jest setup process is complete at this point.
Now the mongo-environment.
js module gets executed in a child process.
This module contains a setup(), teardown(), and runScript() function that all get called from Jest.
Here is the content of the mongo-environment.
js file:I haven’t changed this file at all.
It is straight from the Jest example.
Here is the link to the document for reference: “Using with MongoDB”Line 18 and 19 load the saved globalConfig from the disk and set global variables.
This module is executed in the Jest child process so the global variables are exposed to each test suite.
These variables are used by the test suites to be able to connect to the in-memory database.
8.
Integration TestsJest scans your project files for file names containing .
test.
js and others.
Once it finds them it executes them in the child process.
Here is a contrived example of one test file:Line 9 to 12 is the beforeAll() function which is called before any tests.
Line 10 connects the custom database driver to the in-memory MongoDB database.
Note that it is using the global URI variable set from the mongo-environment.
js module.
Line 11 calls the http-setup.
js module.
This is explained below in more detail however it enables the http server listener and authenticates axios against the API.
A reference to the listener is kept for the afterAll() function.
Line 14 is the first integration test which is a PUT call against the API.
Line 24 is the winner.
It is the purpose for this whole article.
What we are doing here is acting like a standard http client and making a HTTP PUT request to the /products endpoint passing in the contrived product.
Line 25 is an example assertion.
This simple example test suite should be expanded with many more assertions.
Line 29 is the second integration test which is a GET call against the API.
Line 31 is the second winner here.
It is making a HTTP GET request to the /products endpoint to retrieve a list of products.
Line 32 and 33 are some simple test assertions.
Again this should be expanded on.
Line 37 to 40 is the afterAll() function which is called after all the tests are complete.
Line 38 uses the http server listner to close the server.
Line 39 disconnects the custom database driver from the in-memory MondoDB instance.
9.
HTTP Client SetupWhen Jest executes a test suite the axios client is used to make calls to the Web API.
That means we need the API or app in a listening state.
The HTTP client, axios, also needs to be authenticated to be able to access the API.
All of this is achieved through the http-setup.
js module.
Here is the content of the http-setup.
js file:Line 2 imports a shared axios client.
It is important to realize this is a shared instance of axios.
When we set axios.
defaults it will apply to any module that imports axios within the Jest child process.
Line 3 imports our express application objectLine 6 exports the httpSetup() function which is the remainder of the file.
Line 8 starts the http server.
A random high order TCP port is used.
Line 9 gets a reference to the generated http server port number.
Line 10 constructs the Web API endpoint URL.
Line 11 assigns a base URL to the shared axios client.
This will be used as the http address for all future client requests unless it is overridden.
Line 13 is the authentication call to the Web API.
This may be different in your application however I am using simple email and password authentication.
A HAProxy will be used for SSL offloading in production.
Line 14 gets a reference to the valid authentication cookie.
Line 16 assigns the authentication cookie to the shared axios client HTTP header.
As above, we are setting axios.
defaults and it will apply to any axios client calls unless overridden.
Line 18 completes the function by returning the http server listener so the test suites can close the server on test completion.
10.
Global TeardownFinally after all test suite files have been run Jest executes the global-teardown.
js module in the parent process.
The global variables set in the global-setup.
js module are available here.
All we are doing here is stopping the in-memory MongoDB instance.
SummaryThere is a lot going on to achieve true black-box testing of a Node.
js Web API.
I believe the benefits outweigh the time required to set it up.
The major benefit I am seeing right away is being able to build a new endpoint in the back-end of the application without needing to deal with the front-end.
This helps focus your effort on one application design element, preventing multitasking from causing me to make more mistakes or perform tasks slowly.
The final solution section of this article is quite specific to Jest however the concepts will translate to any testing framework.
Thank you for taking the time to read this article.
I hope you have learned something new from my discoveries.
AcknowledgementsI would like to thanks a few people and projects who made this work and article possible:Yoni Goldberg, Albert Gao, and the many other people in this industry that make developing fun.
Node Weekly: A weekly email news letter I look forward to.
Thanks Peter Cooper and staff.
This is my main tool for finding articles like Yoni’s.
Medium: This is my first post on Medium.
It seems like a great platform.
Sub-lists would be helpful however all-in-all it is a great experience.
draw.
io: What a brilliant tool.
I used this web based drawing application to make the images in this article.
Jest: The guys working on Jest have and are doing a fantastic job.
Keep it up.
axios: This project makes working with external HTTP endpoints a delight.
MongoDB: My expectations on document databases are high due to my work with RethinkDB.
MongoDB comes close which is saying something.
Well done.
mongodb-memory-server: Truly a brilliant target for application integration testing.
Great work Pavel Chertorogov and the community.
Node.
js: Finally Node.
js and all the community involved in making a free open JavaScript platform that is a joy to work with.
.