Here are the slides from my talk/demo on ql.io at NodePDX.
This is a beta release of ql.io on node.js 0.6.x.
- Use native cluster module to start the app
- Upgrade all dependencies to the latest
- Limit response size to 10000000 bytes from upstream sources. You can change this with
maxResponseLengthin the config.
- Limit outgoing requests per statement to 50. You can change this with
maxRequestsin the config.
- Chain events for logging done with log-emitter.
- Add a new JSON based interface to browse tables and routes. Try
/routesto start browsing.
If you are interested in running ql.io on node.js 0.4.x, use the 0.3 branch.
To create an app using ql.io 0.4 modules, follow the usual steps:
mkdir myapp cd myapp curl https://raw.github.com/ql-io/ql.io/master/modules/template/init.sh | bash bin/start.sh
One of the key visible benefits of ql.io is that it eliminates the code noise that is common in writing HTTP client apps. As a DSL for writing HTTP client code, it focuses on automating the task of making multiple HTTP requests and processing responses in the best order possible taking care of paralleization, orchestration, projections and normalizations behind the scenes.
In this post, I would like to present the baseline performance benchmarks of ql.io running on node.js 0.4.12. Though I have done some ad hoc tests in the last 2-3 months for hardware sizing purposes, this is my first systematic attempt.
If you’re interested in setting a ql.io app instance on a Joyent’s no.de, here is how.
- Clone the template app
git clone email@example.com:ql-io/ql-io.no.de.git
This template includes a server.js, some sample ql.io scripts.
Create a SmartMachine instance on no.de.
Push the app to no.de
# Assuming your smart machine name is "foo" cd ql-io.no.de git remote add foo.no.de foo.no.de:repo git push foo.no.de master
This will push the app to your SmartMachine, and bring it up.
- Try a sample route.
http://foo.no.de/console to view the console.
See ql.no.de/myapi to see this example in action.
Clients can occasionally get socket hangup errors when origin servers close connections without sending a
Connection: closeheader. See https://github.com/joyent/node/issues/1135 for some background. To avoid such errors, http.request.js now automatically retries the request once provided the statement that caused the HTTP request is a
The engine can now consume CSV response in addition to XML and JSON.
Fixed request body processing for routes (see issue 161).
Use npm installed modules for ql.io-site (see issue 116).
Handle empty response bodies gracefully (see issue 98).
Recover from partial failures in case of scatter-gather calls (see issue 90) - some statements can result in multiple HTTP requests. When this happens, the engine used to fail the entire statement if any of those requests fail. The engine now looks for success responses and aggregates them.
Update CodeMirror to support line-wrapping (See issue 11) - no need to split lines manually anymore.
See Announcing ql.io.