Testfully CLI
Testfully CLI is a multi-platform CLI tool that allows you to work with requests from the command line, and integrate with your CI/CD pipeline.
Pre-requisites
- Node.js version 16 or higher
- A Node Package Manager such as
npmoryarn.
Installation
Install Testfully CLI globally using npm or yarn.
# Install using npm
npm install -g @testfully/cli
# or install using yarn
yarn global add @testfully/cli
How It Works
To use Testfully CLI, you need to export your requests via the Testfully App. Once exported, you can source the exported files to the Testfully CLI, and run your requests from the command line. To learn more about exporting requests, please refer to the Export Workspace Data page.
run command
The testfully run command executes one or more requests. By default, it runs all requests in the supplied file(s). You can specify one or more requests or folder IDs to run specific requests. Use the following parameters to customize the command.
--request-id
By default, the testfully run command runs all requests in the supplied file(s). You can specify one or more request IDs to run specific requests.
# Run requests with ID 1 and 2 only
testfully run --request-id 1 --request-id 2 rick-and-morty.json
--folder-id
By default, the testfully run command runs all requests in the supplied file(s). You can specify one or more folder IDs to run specific requests.
# Run requests in folder with ID 1 and 2 only
testfully run --folder-id 1 --folder-id 2 rick-and-morty.json
--logs
Display logs in the console.
# Display logs in the console
testfully run --logs rick-and-morty.json
--verbose
Display raw requests and responses in the console.
testfully run --verbose rick-and-morty.json
--environment-id
Specify the environment ID to use for the requests.
testfully run --environment-id 1 rick-and-morty.json
--environment or -e
Use the --environment or -e option to provide the environment file to use for the requests. If the provided environment file does not exist, Testfully CLI will throw an error. When you provide an environment file that contains more than one environment, Testfully CLI will use the first environment and ignore the rest. When both --environment-id and --environment are provided, --environment takes precedence.
testfully run --environment pre_prod_env.json -- payment_tests.json
Did you notice the -- before the requests file? It is a separator that tells the CLI that the options have ended and the positional arguments have started. You can use it to separate options from positional arguments.
--globals or -g
Use the --globals or -g option to provide the globals file to use for the requests. If the provided globals file does not exist, Testfully CLI will throw an error.
testfully run --globals workspace_globals.json -- payment_tests.json
To provide multiple globals files, use the --globals option multiple times or pass multiple files as value for a --globals or -g option.
testfully run --globals shared_globals.json ci_globals.json -- payment_tests.json
Did you notice the -- before the requests file? It is a separator that tells the CLI that the options have ended and the positional arguments have started. You can use it to separate options from positional arguments.
--delay-request
You can use the delay-request option to specify the delay between requests in milliseconds. Alternatively, you can use the Delay setting for each folder for a more granular control. In the absence of both, requests are executed without any delay. In the presence of both, the delay-request option takes precedence. Set the delay to 0 to disable it.
testfully run --delay-request 1000 rick-and-morty.json
--reporters
The reporters option allows you to specify the type of test report format to generate. Currently, Testfully CLI only supports the junit format.
testfully run rick-and-morty.json --reporters junit
--reporter-junit-export
The --reporter-junit-export option allows you to specify the directory/file path to export the JUnit report. If the file path does not exist, Testfully CLI tries to create it. If the file path exists, Testfully CLI overwrites the file. If the file path is a directory, Testfully CLI creates a file with the name junit.xml in that directory.
testfully run rick-and-morty.json --reporters junit --reporter-junit-export ~/Downlods/junit-report.xml
positionals
You can specify one or more files to run requests from by passing them as the last arguments.
testfully run --folder-id 1 rick-and-mort-api.json pre-prod.json
Common Errors
This section lists common errors that you may encounter while using Testfully CLI.
Error: No input file specified
You receive this error when you do not provide any input file to the testfully run command. Sometimes, you may have provided an input file but you are missing the -- separator. Make sure to provide the input file(s) after the -- separator. The -- separator tells the CLI that the options have ended and the positional arguments have started. You would need this separator when you have --globals, -g, --environment or -e in your command. Here is an example that throws this error:
testfully run --globals workspace_globals.json payment_tests.json
To fix this error, add the -- separator before the input file(s). Here is the corrected command:
testfully run --globals workspace_globals.json -- payment_tests.json
The -- separator tells the CLI that the options have ended and the positional arguments have started.
FAQ
How does Testfully CLI execute requests?
Testfully CLI uses the host computer to execute requests. It does not use any cloud resources. It uses the same underlying engine as the Testfully App to execute requests so that you can get the same results.
What features are supported in Testfully CLI?
When it comes to executing requests, Testfully CLI supports all features that are supported in the Testfully App except for the following:
- OAuth2 Authorization using the
Authorization CodeorPKCEgrant types.
Does Testfully CLI store any data on my computer?
Testfully CLI does not store any data. Any computed data is stored in memory and is discarded once the request is completed.
Does Testfully CLI require an internet connection?
Testfully CLI does not require an internet connection to execute requests. However, it may require an internet connection to send requests to API endpoints if they are not available locally.
Can I use Testfully CLI in my CI/CD pipeline?
Yes, Testfully CLI is shipped as a standard Node.js package. You can use it in your CI/CD pipeline to run your requests. We have tested it against Windows 10, Windows 11, Ubutnu 20.04, and macOS 13 and 14.
How can I update Testfully CLI?
You can update Testfully CLI using npm or yarn.
# Update using npm
npm update -g @testfully/cli
# or update using yarn
yarn global upgrade @testfully/cli
How can I uninstall Testfully CLI?
You can uninstall Testfully CLI using npm or yarn.
# Uninstall using npm
npm uninstall -g @testfully/cli
# or uninstall using yarn
yarn global remove @testfully/cli