In Cucumber/Ruby when testing a Restful API, you can test it either In Process or Out of Process.
The main difference here is the stack used. In a "In Process" scenario, you are talking to the API within the process itself. Meaning you don't need a webserver, http calls, or Web Framework at all. This is faster since there's less overhead.
In process works by using a gem called Rack-Test. This is a library allowing the user to test the application. It's like mocking out the webserver.
"In Process" to me, seems the same as Unit Tests.
The downside here is that while the code is being directly tested - it is not going through a real world scenario. This might be fine for Unit Testing, but it doesn't handle a real browser.
Out Of Process handles a real world scenario. In this case, you need to have the webserver up and running. Then make calls to the REST service as you would in the production environment. You would pass a HTTP request through the HTTP client, to the HTTP Server, etc.
While this is slower, this is typically how QA Automation works. QA would use a tool like WATIR or WATIR-Webdriver to drive multiple browsers against a site, restful service, api, etc. and get a response.
But you don't always need to do that. Sometimes you just need to make a call, pass some parameters and get a result fast, so you can move on with a test.
An example would be, needing to create a user quickly to run a subscription scenario on your website. You aren't testing user creation, so why go through the front end to create a user? If you can create the user "in process" by simply passing some parameters to set up a user in a specific state... it's much better.
The trick is knowing when to use In Process vs. Out of Process. For QA, I think any point you are testing a flow the user would hit for the BDD's specified, you should be using Out of Process testing. For any set up to get to a BDD's When state (i.e. the Given state), then you would use In Process.
The main difference here is the stack used. In a "In Process" scenario, you are talking to the API within the process itself. Meaning you don't need a webserver, http calls, or Web Framework at all. This is faster since there's less overhead.
In process works by using a gem called Rack-Test. This is a library allowing the user to test the application. It's like mocking out the webserver.
"In Process" to me, seems the same as Unit Tests.
The downside here is that while the code is being directly tested - it is not going through a real world scenario. This might be fine for Unit Testing, but it doesn't handle a real browser.
Out Of Process handles a real world scenario. In this case, you need to have the webserver up and running. Then make calls to the REST service as you would in the production environment. You would pass a HTTP request through the HTTP client, to the HTTP Server, etc.
While this is slower, this is typically how QA Automation works. QA would use a tool like WATIR or WATIR-Webdriver to drive multiple browsers against a site, restful service, api, etc. and get a response.
But you don't always need to do that. Sometimes you just need to make a call, pass some parameters and get a result fast, so you can move on with a test.
An example would be, needing to create a user quickly to run a subscription scenario on your website. You aren't testing user creation, so why go through the front end to create a user? If you can create the user "in process" by simply passing some parameters to set up a user in a specific state... it's much better.
The trick is knowing when to use In Process vs. Out of Process. For QA, I think any point you are testing a flow the user would hit for the BDD's specified, you should be using Out of Process testing. For any set up to get to a BDD's When state (i.e. the Given state), then you would use In Process.
No comments:
Post a Comment