The Science of Blogging

If you haven't been living in a cave for the past 2 years, you're already quite familiar with the notion of MapReduce. In short, MapReduce is a way of handling computation. It differs from traditional compute models in that the work load is essentially designed to be distributed, which is ideal for large work loads. Google made this technique popular and has published several papers on the topic.

Amazon today sent out an announcement about a new beta product from AWS called Amazon Elastic MapReduce. Utilizing Hadoop, Elastic MapReduce allows you to easily setup distributed compute jobs. You can do things such as:

Specify the number of EC2 instances to use
Monitor the progress of running jobs
Develop the MapReduce functionality in the language of your choice

Given that the Hadoop/MapReduce use case was so prevalent on EC2, it's no surprise that Amazon has rolled this out as a service. I think in general, Jeff Bar and the rest of the AWS team has done a good job of listening to customer feedback and observing real world use cases, and responding as appropriate with new service offerings or service modifications. However, I'll be honest, I would have preferred it if the AWS team had continued to focus on stabilizing its existing offerings such as CloudFront and SimpleDB.

I can say though that the SimpleDB team is amazing. One of the team members for SimpleDB had noticed that here at Compendium our SimpleDB usage had dropped dramatically the previous month and actually reached out to us to see if there was a problem. There actually was a problem and we were able to actually interact with someone at Amazon to discuss our issues.

Amazing. So, despite the fact that I would have loved to see some CloudFront/SimpleDB enhancements, I can't help but be amazed and continuously impressed with the AWS team. I'm looking forward to seeing what changes come up next.

Comments for Amazon Elastic MapReduce

Friday, April 3, 2009 by cloudberrylab.com:: I am excited to see the pace of Amazon team innovations when it comes to their web services platform. There is a growing ecosystem around AWS and my company with our CloudBerry Explorer freeware for Amazon S3 and CloudFront is a proud part of it.