Web storage archive pattern
The next pattern will aim at a smaller subset of archival as compared to the previous monitoring integration pattern. While the previous pattern was a more theoretical approach to the larger aggregation and archival schema, sometimes it is relevant to only archive certain files. One choice might be to dump the database to the filesystem. This may not be the best choice for all use cases but it is still a common approach.
A much more common practice is to rotate files, such as logs, out of the server and into a permanent store as they grow over time. There are plenty of ways to do this, such as remote syslog, or some log services such as Loggly or Logstash. It is such a common practice that there are literally hundreds of third-party solutions to choose from.
To demonstrate this, we will use logrotate which is included in nearly every major Linux distribution. Logrotate will handle the file rotation, compression, and synchronization with Amazon S3. The configuration...
