This article was originally published on Botium’s blog on March 24, 2021, prior to Cyara’s acquisition of Botium. Learn more about Cyara + Botium
Botium offers two methods to test an SMS chatbot. It can act as a user sending SMS messages or can interact with the API behind the chatbot. Sending SMS messages looks like an optimal choice because it covers all the stack, but it has a major drawback, the increased costs.
For smaller systems sending SMS messages is usually a good decision. It is an all-purpose solution, and it is easier to set up.
More complex chatbots can have more sophisticated configuration, API-testing all conversation branches, and SMS-based testing for checking performance and availability of the full stack for example.
Tip: If your API delivers NLP information like recognized intent and entity, then in case of API testing you can even use NLP Analytics in Botium (depending on license).
API Testing
There is no step-by-step solution for API testing. We need a Botium-connector which fits to your API. You can use the suitable generic connector of Botium Box (in most cases Generic HTTP(s)/JSON interface). Or you can write your own.
SMS testing with Botium Twilio SMS connector
Botium Box uses Twilio to communicate with any SMS chatbot.
You’ll need a Twilio account, register a telephone number in Twilio, and configure Botium Box for Botium-to-Twilio messages, and Twilio for Twilio-to-Botium messages.
You can use a trial account for Twilio, but there are limitations, and the budget is limited. We use an upgraded account here.
Please keep in mind that SMS communication has no session. If you have for example development, and production tests, then it is a good practice to register telephone numbers and configure connection in Botium for each. Otherwise you can get some random test fails if they are running in parallel.
Twilio account
Follow the steps to start a trial and upgrade.
Register a telephone number
Navigate to Active Numbers, and click on Buy a Number. Consider choosing the same country as the one, where your chatbot’s number is located. And choose ANY or SMS in capabilities, and buy a number.
Check SMS capability (optional).
Notice the telephone number, and the credentials (settings, general)
Configure Twilio in Botium
Now we have to set up Botium.
Under the Chatbots list click on the Register new chatbot button.
Fill the form with the values from the Twilio Dashboard. (Sending SMS to the field is the telephone number of your SMS chatbot.)
Once the form is complete, Botium Box will be able to receive messages. But we can’t test the connection yet. We have to configure Twilio to send messages to Botium Box.
After you choose an API key, you can see the endpoint for receiving messages on the registration form. Copy the endpoint URL.
Configure the Twilio webhook
Open the settings for the registered number in Twilio, and paste the endpoint as a webhook in the Messaging section (not in the Voice & Fax section!), and save it.
Test the connection and save it in Botium Box
Everything is set up. Now we can test it by pushing SAY HELLO (CHECK CONNECTIVITY) button. (We expect that the chatbot responds if we send hello. If it’s not the case, then skip the test. You can test it manually using the Live chat feature of Botium)
Everything is fine, we can save the form.
If it is not working, then check the error message displayed on the connectivity test, or check Twilio logs on the phone number.
Disable parallel execution in Botium
If you have more Agents (workers) in Botium, and more test cases in a test set, then Botium executes the test set parallel. It is good for most of the connectors, but not for Twilio SMS connector as mentioned before.
We have to set the Parallel Jobs Count to 1 (or choose a dedicated Agent) in project settings to disable parallel execution.
Use Botium Box
The connection is configured, and you can use all features of Botium!
See this article in Spanish here! 
