I’m very pleased to say I’ve just made public a repository for a tool we’ve built to make it easier to ingest data automatically into Amazon Redshift from Amazon S3: https://github.com/uswitch/blueshift.
Amazon Redshift is a wonderfully powerful product, if you’ve not tried it yet you should definitely take a look; I’ve written before about the value of the analytical flow it enables.
However, as nice as it is to consume data from, ingesting data is a little less fun:
- Forget about writing raw
INSERTstatements: we saw individual inserts take on the order of 5 or 6 seconds…