Here at Compendium we use a lot of open source software and other open platforms. The list is innumerable (at least in a blog post) but a few pieces of software that we rely on include:
One of the issues that we run into is how we end up pushing static assets to our CDN (CloudFront) during releases. We have a number of static assets that change somewhat frequently (Javascript & CSS mostly) and because of the nature of CloudFront, we needed to introduce a naming scheme to deal with this. Essentially each static asset gets a different file path every time we publish it. This scheme has a few problems, including having to publish all of our static assets every time one of them changes.
After a particularly change intensive testing period required a number of publications to the CDN, we decided that enough was enough and came up with a more reasonable scheme for publishing static assets. In this new scheme, assets will be named based on their MD5 hash. This means that an asset is only added to the CDN if it has actually changed. It also meant that we needed a new way of handling renaming, compression, etc.
This required a number of phing enhancements including:
- Linux (CentOS)
- PHP
- MySQL
- Agavi
- Phing
- AWS (SimpleDB, EC2, S3, SQS, CloudFront)
- YUI
One of the issues that we run into is how we end up pushing static assets to our CDN (CloudFront) during releases. We have a number of static assets that change somewhat frequently (Javascript & CSS mostly) and because of the nature of CloudFront, we needed to introduce a naming scheme to deal with this. Essentially each static asset gets a different file path every time we publish it. This scheme has a few problems, including having to publish all of our static assets every time one of them changes.
After a particularly change intensive testing period required a number of publications to the CDN, we decided that enough was enough and came up with a more reasonable scheme for publishing static assets. In this new scheme, assets will be named based on their MD5 hash. This means that an asset is only added to the CDN if it has actually changed. It also meant that we needed a new way of handling renaming, compression, etc.
This required a number of phing enhancements including:
- YuiCompression Filter to compress CSS and Javascript
- Md5Mapper to handle renaming of a file based on the hash of the source file
- Rollup scheme for defining Javascript and CSS rollups
- S3 Publisher Task
- Asynchronous Logging for PHP
- Enhanced Agavi logging subsystem
- Yahoo BOSS Access
- Memcache Session Storage for Agavi
- Flexible, policy based, cache subsystem
- Memcache
- Includes chunking for segments greater than 1MB
- Files
- S3
- Memcache
- Selenium Integration for Phing/Agavi/PHPUnit
Comments for Open Source Code from Compendium
Leave a comment