Re: Using the Last Month of Data

I hope that you are using Postgres partitions for this. Depending on your data size, you could ease sorting by date for the 'last month' by day, week or month. This will have an added benefit of requiring less ram for most of your selects and otherwise increasing the speed of your queries.

Neat, it's like an updated version of Tandems 5-minute rule for the cloud era.

What about i/o costs of traffic (outgress).

What other costs do we need to factor in on aws?

For us, I/O isn't as big a cost factor, since our data processing mostly resides in ephemeral storage where I/O charges don't apply. (Our permanent data backups live in S3.)

I/O pricing does apply when using EBS.

How come you haven't predicted the cost of Data Redundancy at first? This seems like something pretty basic to me.

Very true. This is basic stuff, and we simply overlooked it. In the future, we might even need to add increased redundancy to our data, which we need to be aware of.