Planet Drupal

Integrating Views Infinite Scroll with Masonry

In this blog post, I am going to extend upon my last module overview of Masonry, which is a Views plugin that utilizes the jQuery Masonry plugin. (Short recap, Masonry sets your elements into a responsive grid that aligns the elements in a brick fashion based on height, compared to the standard float technique.)  

If you read the original jQuery Masonry site, you can see that there's a portion that talks about integrating infinite scrolling. For those of you who are not familiar with infinite scrolling, it is the method where if you are reading a river of posts or content and you scroll to the bottom, more content will automatically load in without using the pager or navigating to a new page. This is great for sites that publish a lot of content or act as news aggregators so the user doesn't have to page through to read the newest content. There are some drawbacks to using infinite scroll, but we'll get to that in a minute.

I'm crazy. I haven't even said anything about Drupal yet, so let's do that. How do we implement infinite scrolling on our site? I'm glad you asked. There is a nifty little module called, go figure, Views Infinite Scroll. Nicely worded, module maintainers. Here are some things you'll need for this overview:

  • Views
  • Masonry
  • Views Infinite Scrolling
  • Devel (optional - used to create dummy content)
  • Responsive theme, such as ZURB Foundations 
  • Patches from the issue queue, I'll talk more about this - Support Masonry module

Install the module (Drupal module installer, FTP, drush, or drop it in your local environment) and enable it. Create your basic view of whatever content type you would like or use Devel to create some content), and set the format to Masonry with the default settings. To enable infinite scrolling, go down to the pager options in the middle column of the Views UI. Select "Infinite Scroll" and configure the options however you would like and save the view.

If you aren't using Masonry and you scroll down to the bottom, you will see more content load into the view. If you are using Masonry, you might scroll to the bottom and notice a big dud as no more content loads. This is where the patches from the issue queue come in! BWPanda went ahead and compiled separate patches together (including his own) to be applied to both Views Infinite Scroll and Masonry. Download both of these and apply them using Git, git apply -v [patchname.patch], or manhandle them. After those are applied, feel free to clear cache just in case. Reload the view and scroll down, now more masonry blocks load into the view via Infinite Scroll. Below is a short overview of what it looks like before and after the patches when trying to integrate Views Infinite Scroll with Masonry.



Infinite Scroll is awesome to be used on blogs, loading in additional content without having to use a pager, and basically pleasing the average, lazy news reader. But is it always necessary? There are instances when you wouldn't want to use infinite scroll. What if the view contains a lot of content, and you have information or navigation options in the footer you want the reader to see? Well, they'll never reach it, like a pug running after a hotdog on a treadmill. So close, yet so far. Patricio Robles made a list of pros and cons of Infinite Scroll in his recent blog post

What are your thoughts on using Infinite Scroll, or its integration with Masonry? Let me know in the comments below!

Using Views Responsive Grid in Drupal 7

In response to Mark's post, Converting Views List Into Responsive Stacked Columns, we've teamed up to create a module for just this sort of thing. Views Responsive Grid is a views display format plugin designed to give you the proper HTML structure for creating CSS grids. Although still a sandbox project, we're working to get it through the application queue quickly.

Views Responsive Grid gives you all of the functionality of the default grid style, but without all of the ugly tables. It also gives the user the ability to utilize custom classes already in their theme to properly layout the columns. Responsive themes will allow grid items to stack(like a list) according to the proper sort order set inside the view.

How to Use

After downloading and enabling the module, create a new view with the responsive grid display format. Specify the number of columns, and the alignment of the grid.

  • Horizontal grids denote how we read, left to right, top to bottom.
  • Vertical grids are more closely related to how you see a newspaper laid out; top to bottom, left to right.

Lastly, you'll need to understand that the the module won't provide any default styling to the grid. You may think it's not working - this is by design. In order for the columns to work, you'll need to specify the class name of your columns. For example, if your theme utilizes a grid, like Twitter Bootstrap does, you would specify "span3" as the column class (making sure to use the correct span size). This will make sure your column adheres to the grid in your Bootstrap based theme.

Try it out, and tell us what you think in the comments below!

UPDATE: This is no longer a sandbox project and can be found here

Recommendations for Submitting a Project Application

So you want to submit a module for full project status? This is a follow-up to another post I wrote, What's Wrong with the Project Application Queue?, which happened to stir up a lot of mixed feelings and people had quite a bit to say. While I don't necessarily agree with some of the processes currently in place, here are some recommendations for other developers struggling to get their projects through the application queue.

Do Your Homework

Probably the single most important thing to remember is to do your homework before submitting your application. Find out what modules are already out there that are similar to your idea.

  • If there isn't anything related to your idea then you're probably ok.
  • If there is, then ask yourself whether or not you can submit a patch/feature request.
  • If you're not sure then it's always ok to use the Drupal IRC channels to ask, or you can still ask in the form of a feature request.
  • If you're just flat out duplicating someone else's module then you're almost certain to be rejected.

Tidy Up and Document Your Code

Remember, submitting an application means that you won't be the only one using your module and everyone has their own "style" of programming. Drupal coding standards and documentation standards are an agreed upon template for writing modules so they can be easily read by other Drupal developers.

While I don't really agree with PAReview as a basis for reviewing project applications, people do use it, and you will catch a lot of flak for not meeting the coding standards. To avoid getting sent into an endless cycle of reviews, make sure you run your module through the PAReview app, and Coder each time you make a commit. PAReview will catch some issues, like security bugs, and code errors, but at the very least, it's good practice, and will save you a lot of headaches later on.

Be Patient and Keep Your Cool

It might take a while for your project to get approval, but try to understand that project reviewers are busy just like you. You're not the only one wanting attention, and quite frankly, they probably have more important life matters to deal with. They are also volunteering their time for free to do the reviews, so be polite. Take their advice and learn from it rather than just getting frustrated.

The review bonus may be a drag, and it is optional, but it may also help get your project through the queue faster, plus it helps out the other developers; however, if you're going to review other applications, don't be lazy and just run it through the PAReview app. You need to legitimately look through the code for improvements, otherwise you're really only adding to the problem. Refer to the How to review Full Project applications page on for more clarity on what to do and what not to do.

Maintain Your Project

Congratulations, you're approved! Getting final approval on a project application doesn't, by any means, mean that you are "finished". An important part of being a module maintainer is just that, maintaining it. People are going to rely on you to fix bugs, answer questions, and keep your module up to date. If you can't commit the time to do all of these things then it's probably better if you don't submit the module at all, or hand off the responsibility to someone who is willing to take it.

Don't Give Up!

I think we can all agree, rejection sucks. It's frustrating to put your blood, sweat, and tears into something only to have it rejected by the community, but it's not the end of the world. If your submission is rejected, don't give up! There are other ways to contribute, and you can always come up with new ideas.

Thanks to all of the various commenters on my previous post for some of the suggestions here, your feedback does get heard! 

Responsive Drupal Theming, Done the Right Way... At Least For Now Anyway

A little over a year of following my own advice, I decided to re-visit the long overdue topic of responsive theming and share several valuable techniques and processes I've learned along the way. 

[php]if ($attention_span === 'tl;dr') { $summary = $awesome_theme / ($bootstrap + $less + $display_suite); }[/php]


Responsive Framework

The most important part of any toolkit is probably going to be the theme framework you'll use. Omega, hands down, claimed this crown for me in 2012. Packed with an amazing amount of flexibility and features, there wasn’t much I couldn't do with it. Towards the end of the year, however, I found myself feeling like I was fighting it more than working with it. Granted, this issue probably stemmed from my “need-something-new” itch. Alas, I gave in and decided to really dig and see what other frameworks existed... and not just in Drupal either.

To Google, I did go. Searching for "responsive frameworks", I came across this kick-ass article by Ben Gremillion from Treehouse Blog: A Look at Responsive CSS Frameworks. He gives a pretty darn good breakdown of the pros and cons on many of them.

My decision? Drum roll...

Tickle me spring green and call me mobile, [Twitter] Bootstrap claimed the 2013 framework crown! So it wasn’t really that dramatic of a decision, but here are a few of the many reasons I decided to go this route:  

  • Co-created by Mark Otto, Designer at GitHub and previously the Web Platform Designer at Twitter. This, like Ben said, gives me a little more ease knowing that this a rather solid framework that is used by millions of people already.
  • Extensive documentation. No need to drudge around some new framework that “looks” awesome but really hasn’t had the time to be properly documented. Someone has to have time to play the laser pointer chasing game with my cats.
  • Based in the LESS preprocessor, which, for me, has a far simpler syntax than one like Sass. Many people can argue this until their hairline recedes, but ultimately I would consider using Sass for more powerful things like actually creating the framework rather than just a simple sub-theme. For sub-theming, it’s nice that I don’t have the overhead of @includes and @extends everywhere.
  • Built-in 12-Grid system, Base CSS, JavaScript Plugins, Components...... and the kitchen sink, what more do you want?
  • Wasn't really necessary, but glad to know that there is already a project on Drupal: Bootstrap. Albeit, this probably needs some fine tuning, but still a nice surprise. Ehh, who am I kiddin? "There's a module for that". 

CSS Preprocessor

This is probably the next important tool in your responsive arsenal. No, it’s not actually necessary to get the job done, but by golly it certainly speeds up the process! Now any preprocessor will do and you can generally find a GUI or CLI program for any operating system. 

For those of you who use Macs though, have I got an app for you! CodeKit — THE Mac App For Web Developers. It is, by far, the best program I have bought in a LONG time! Not only does it preprocess LESS automatically, but also Sass/Scss (with even Compass and Bourbon built-in), Stylus, JavaScript, CoffeeScript, Haml, Jade, and Slim. Oh and the frequent updates are nice too! 

LESS Media Query Mixins

I found this useful technique that guided me down the road to creating my own that I use quite frequently through out my projects. This allows for a much easier implementation for responsive design. It also allows me to keep styling together and keeps the code pretty clean. No more additional files, no more separate definitions, it just works! 

@mobile: ~"all and (max-width: 767px)";
@tablet: ~"all and (min-width: 768px) and (max-width: 979px)";
@normal: ~"all and (min-width: 980px) and (max-width: 1199px)";
@wide: ~"all and (min-width: 1200px)";
.style-article-featured {
  width: 100%; // Mobile first
  @media @tablet {
    width: 80%; // Tablet
  @media @normal, @wide {
    width: 50%; // Normal & Wide


Display Suite: Layouts & Classes in Harmony 

Many of you are probably already familiar with Display Suite. Until this module came along though, implementing consistent styling through out a Drupal site presented its challenges. Often times, more than not, areas of the site (like node templates or views) had to be constructued to "match" the theme. Because of this, I think it's relatively easy for us to be stuck in the mindset that we should be letting these other modules handle our structuring (and even sometimes styling) of entities or other data.

So I tried a little experiment. No node templates, no fields in views, just render the entity with a desired layout. To my complete and delightful assumption, Display Suite worked like a charm! Let's take it a step further, create custom layouts for the content types. These, of course, should probably be based off the theme and should be predetermined before you actually start theming or building your site. 

Now, you could use the existing classes that come standard with rendering an entity (which include the layout name), but I prefer to create my own region classes in Display Suite. It allows for a cleaner approach in the CSS so I don't have to target multiple classes. Either way, matching an entity's layout classes with your themes CSS is where the power and magic really starts to happen.


Display Suite Concept Example

Take our site here, this blog you're reading. We have six different layouts for this content type. We use these different layouts through the site and each have their own unique theme stylings. For each style, I created a custom layout with a matching CSS selector. In this example, I will use our Article Teaser layout: 

In the Display Suite classes admin area (left image), I added [code]style-article-teaser|Artical Teaser[/code] where the text on the left of the pipe is the class name and the text on the right is the human readable label. Once my classes were setup, I headed over to the content's "Manage display" configuration form (top right image). From there, I chose my "Article Teaser" layout and scrolled to the bottom where I selected the "Custom classes" tab (bottom right image) and chose the appropriate class region for my entire layout. 

You may wonder, "What does this have to do with responsive?" Actually proably more than you might think. Whenever I start a new project, I like to create an internal style guide that shows the rendered element and the required markup (complete with internal classes). I do this for three reasons:
  1. Helps the developer build things the right way, markup wise and inserting classes where necessary.
  2. It provides a single place to view both generic/site-wide elements and layout specific elements.
  3. In addition to above, I can then resize the window and view all these elements and how they react to responsive media queries.


Ta-ta-ta-da! Fewer one-off styles or pages that aren't responsive. It just works! Granted, you'll still have a few pages that you might need to tweak, but you're going to be far better off with only a few and not all!

Overall, responsive theming can be a daunting task, to say the least. But if you're willing to get a groove goin', things can speed up pretty nicely. Hopefully some of these techniques and processes I've discovered will help aid you in your troubles. Also, feel free to let me in on some of yours down below in the comments!


Photo Credit: jiraisurfer

What's Wrong with the Project Application Queue?

In April 2012, I submitted a module called twitter_signin_callback to the project application queue. The concept was simple, my module was meant to add functionality to the twitter_signin module that allowed a developer to specify a path for where to send the user after logging in via Twitter OAuth. My thought was to get approval on a simple module to meet my goals for my job, and get commit access to be able to contribute some bigger ideas later.

Disclaimer: I understand that the project reviewers are extremely overloaded, this post is not to complain, but rather to point out some problems and open it up for discussion.

First, I'd like to start by sharing my experience with the project application queue:

In April 2012 I submitted a module called twitter_signin_callback to the project application queue. The concept was simple: my module was meant to add functionality to the twitter_signin module that allowed a developer to specify a path to send the user after logging in via Twitter OAuth. My thought was to get approval on a simple module to meet my goals for my job, and get commit access to be able to contribute some bigger ideas later.

After about 6 months of review by other users, and running my project through the PAReview app, it finally passed inspection and moved onto the next phase - review by a project reviewer. At this point I was informed that in order to have my project reviewed, I needed to complete the review bonus. So, I spent some free-time painstakingly reviewing other projects to get the bonus. Shortly thereafter, my project was rejected. I was told to submit my module to the Twitter module issue queue as a feature request.

Currently, I still do not have commit access, wasted 6 months, and I have to start all over again. I'm fairly certain that I'm not the only one this has happened to, and honestly, my experience has almost made me not want to try again.

Rejection is fine, but...

Why did it take 6 months to get an answer?

Why didn't anyone tell me to commit it as a Twitter feature in the beginning?

My code was flawless, so why can't I still get commit access so I can meet my goals?

These issues are a big detractor to new Drupalers who might be willing to put in the time and effort to contribute back. Being that Drupal's open-source community is such a big selling point, these problems are not conducive to community growth.

The review bonus is a start, but I think some kind of incentive needs to be in place for senior contributors to do these reviews, or mentor other developers. After all, how can you expect people who haven't even gone through the project application process to be reviewing other projects?


Yet another simple Amazon S3 backup script for Drupal

As part of setting up servers, we need to set up a backup system. While there are many out there, often times they boil down to a script. In that regard, I recently wrote a backup script to back up a server with multiple sites to Amazon S3.

Here were my requirements:
1. Backup off the server (i.e. Amazon S3)
2. Backup all sites and databases on the server individually.
3. Only backup the contents of tables that are not cache tables.
4. Have auto expire of backup files.

I decided on S3 because it was easy, fast and cheap. It also had the ability to set up lifecycle rules so that as files got older, I could move them to glacier or delete them. This removed the need for the backup script to handle the deletion logic. All I needed was a script to send the backup files to Amazon S3 every night.

This is how I did it.

First, sign up with Amazon S3 and generate an Access and Secret keys. This can be done from your account Security Credentials page.

Next, create a bucket. Follow their instructions for how to do this if you don't know how. For the bucket, go to the properties and set the lifecycle rules however you want. For now mine are to send to glacier after 7 days and then delete after 3 months. Glacier is far cheaper than S3 but takes 4-5 hours to recover.

Next, install s3cmd which is a command line tool for sending files to S3. If you are on Ubuntu, it is in the apt repo so just type apt-get install s3cmd. You can get more information along with other install methods at

Then run s3cmd --configure from the command line of your server and enter your Access and Secret keys. This will store the keys in a config file so that your server can access your S3 account. You can also optionally enter a GPG encryption key. This is separate from the access keys and will be used to encrypt the data before it is sent to S3. Use this if you are very worried about security. By default, your data should be safe unless you make it public. Encrypting sensitive stuff, however, is never a bad idea.

At this point, your command line should be set up to send files to S3. Now add in the script to do the backups. Create a file that will be run daily with cron that contains:


DATE=$(date +%Y-%m-%d)

# Backup all databases to S3
for DB in $(mysql --user=$DB_USER --password=$DB_PASSWD -e 'show databases' -s --skip-column-names|grep -Ev "^(information_schema|performance_schema|mysql)$");
#First dump the structures
TABLES=`mysql --skip-column-names -e 'show tables' --user=${DB_USER} --password=${DB_PASSWD} ${DB}`
mysqldump --complete-insert --disable-keys --single-transaction --no-data --user=$DB_USER --password=$DB_PASSWD --opt $DB $TABLES > $TMP/$DB-$DATE
#Then dump the data, except for cache and temporary tables.
TABLES2=`echo "$TABLES" | grep -Ev "^(accesslog|cache_.*|flood|search_.*|semaphore|sessions|watchdog)$"`
mysqldump --complete-insert --disable-keys --single-transaction --no-create-info --user=$DB_USER --password=$DB_PASSWD $DB $TABLES2 >> $TMP/$DB-$DATE
#Gzip everything
gzip -v $TMP/$DB-$DATE;
#Upload to Amazon S3
s3cmd put $TMP/$DB-$DATE.gz $S3_BUCKET/databases/$DB-$DATE.gz;
rm $TMP/$DB-$DATE.gz;

# Backup all sites to S3
for DIR in $(find "$SITES_DIR" -mindepth 1 -maxdepth 1 -type d);
#Tar and Gzip each directory
BASE=$(basename "$DIR");
tar -czf $TMP/$BASE.tar.gz $BASE;
#Upload to Amazon S3
s3cmd put $TMP/$BASE.tar.gz $S3_BUCKET/sites/$BASE-$DATE.tar.gz;
rm $TMP/$BASE.tar.gz;

That's it! The bash script should be fairly straightforward and easy to modify if you want to. One thing to note is that I first backup the table structures and then the contents so that I don't get the contents of cache tables (Couldn't find the article where I got that code so if you add a comment below I'll add in a link to it).

Let me know if you have any improvements.

Why I didn't use drush or backup_migrate?

I wanted a solution that ran outside of Drupal and would work for any type of site. While backup_migrate can do many of the same things, having it run from within Drupal is a major issue in case it breaks.

Photo courtesy of

Lessons Learned from 3 Drupal Multilingual Projects

Over the past year I've led a team that launched 3 different Drupal multilingual projects and during that time I have come to the conclusion that doing a multilingual site is a difficult undertaking. In fact, I would say that converting your site to a new language is probably more challenging that building your initial site, especially if you can't read the additional languages. Looking back, I was just naive enough during the sales process to say "oh yea, Drupal handles multilingual sites no problem;" and while that statement is 100% true, today I probably would be a little more cautious in my approach. Anyway, this isn't going to be some massive tutorial on how to setup and configure a multilingual site in Drupal, there are numerous videos and tutorials out there that do just exactly that; but I will hit some of the pain points I experienced and give you a feel for what you're getting into.

First, let me start by saying there is tons and tons of information online to help you through the process of setting up your multilingual Drupal site; that's both a blessing and a curse. The blessing is that odds are someone has experienced your problem and most likely there is a posted solution; on the other hand it's often like finding a needle in a haystack because the information is not well organized. When issues arose during my projects, I would often find myself going down the Google search rabbit hole and steadily hitting the back button. The point here is simply to give yourself time to research things when you run into issues and on average figure that it's going to take 2-3 times as long the first go around. The information is probably out there, but it might not be that easy to find.

Second, find a translation workflow that works best for your team to minimize the overhead during the translation process. I recommend, establishing a gatekeeper for all content translation to ensure all content is input properly. To support this, make sure you're using several of the translation overview modules that will help keep track of what has and has not been translated. This does not need to be a single person, but could be multiple individuals assigned to their respective translation; in the end this will provide a higher level of consistency for your project. Remember that it's not just content you'll have to translate, but views configurations, blocks and sidebar items, navigation and other miscellaneous items you might have forgotten about.

Third, QA, QA, QA because no matter what, you're going to find little strings here and there that didn't get translated properly; areas where a view isn't filtering by language or a block title is still listed in the default language. Spotting those issues is rather easy, fixing them is time consuming; so if the content has to be flawless on launch, then it will be vital that your team has the necessary time to review. Now, if you're a little more agile and 90-95% completion will suffice, let your community or team help QA after you go live; in my opinion, that's the best approach since your team could move on to other tasks if need be.

Finally, if you really want to take your experience to the next level, consider using localized imagery as well. Obviously this can be a little trickier but putting in images of Hong Kong instead of New York for your Mandarin translation or even photos of Asian models when appropriate can help the site to better connect with your audience. Another factor to consider for internationalization is the difference colors mean in other cultures, it might be a good idea to modify the design slightly based upon the targeted region.

Here are 3 simple takeaways about building a multilingual site in Drupal:

  1. Give yourself plenty of time to complete, test and double check the work
  2. Utilize the numerous multilingual module add-ons to maximize your efforts and reduce headaches
  3. Put a plan together on how you'll roll out each language and establish which team mates are responsible

Photo Attribution: ZapPowBang

Using jQuery Masonry for a creative Pinterest effect in Drupal

I know we're all guilty of doing this. We get on Pinterest, start pinning our favorite holiday cookie recipes and those "laugh awkwardly by myself" e-cards, then start thinking, “Geez, what a cool layout for all these pins! How can I make my Drupal 7 site do that?” Well by golly, now you can with the jQuery Masonry plugin.

Straight from the documentation, here’s a brief introduction to what Masonry is: “Masonry is a dynamic grid layout plugin for jQuery. Think of it as the flip-side of CSS floats. Whereas floating arranges elements horizontally then vertically, Masonry arranges elements vertically, positioning each element in the next open spot in the grid. The result minimizes vertical gaps between elements of varying height, just like a mason fitting stones in a wall.”

So as they explain, it’s like stacking bricks. Make the window smaller, and Masonry will find a new spot for the elements. Now you’re probably wondering, is there a module for that? You betcha. It’s actually a Views plugin called Masonry, and it’s pretty easy to use. Below you’ll find a screencast I made using the Masonry plugin. It’s a quick overview of the installation and use of the module, but I’ll write a quick summary.

  1. Added jQuery Masonry plugin to the libraries directory - check status report to confirm.
  2. Installed/enabled necessary modules.
  3. Chose responsive theme - Bootstrap as my weapon of choice
  4. Made a demo content type, generated demo content with devel.
  5. Created a View page based on the demo content type.
  6. Set format to Masonry grid, left settings default - for transition effect, check Enable Animations and Make Grid Resizeable.
  7. Shrink the window size and shazam! Watch the awesome masonry effect unfold... or re-stack.

Optional: Add some pretty little CSS to make those boxes pop!

 .masonry-item {
    background: rgb(122,188,255);
    border-radius: 5px;
    box-shadow: 0 1px 6px -1px #242424;
    box-sizing: border-box;
    text-align: center;
    padding: 10px;
 .masonry-item:nth-child(2n) {
    background: rgb(205,235,142);
 .masonry-item:nth-child(3n) {
    background: rgb(255,175,75);


      Modules used during the demonstration:


  • Views
  • Field Formatter
  • Libraries
  • Devel
  • Masonry


Last week I wrote a blog post on ways to find popular content on your site through Drupal modules, and I think using Masonry with Radioactivity would add an extra level of awesome to your site.

What other kinds of content could you use Masonry on to add a neat effect? Leave your comments below.

Create a high performance Drupal server for just $30 a month

I was recently in need of a personal server to run some Drupal sites and being a cheap person, set out to get the best bang for my buck in a server. To start out with, forget any shared host servers. They just aren't going to cut it. I looked at several cloud hosting providers and finally decided on amazon based on price and future expansion. After getting everything set up my server is extremely fast and I'm only paying about $30 per month! Here's how I did it.

Calculating the cost

First lets look at the cost.

A small instance of Linux on Amazon EC2 currently costs $0.065 per hour which works out to:

0.065 * 730 = $47.45 per month

This is fine for setting the server up but once the server is configured, purchasing a reserved instance can seriously save some money. I'm going to be running this instance 24x7 since it is a web server and I don't ever want to take it down so I purchased a Heavy Utilization Reserved Instance. Reserved instances are purely an accounting function. You don't actually have to change the instance in any way, just pay the up front fee and get the lower rate. Just be sure to double check the region and get the right region when you purchase the reserved instance.

The heavy utilization reserve instance price works out to:

(0.016 * 8766 + $195)/12 = $28 per month

Add in a couple bucks for ebs storage and transfer costs and you are at $30. Sweet!

Setting up the server on Amazon

I recommend Ubuntu 12.04 (LTS) as it is supposed to be more stable and supported long term. You can find the Amazon AMI instance here:
Find the precise 12.04 (LTS) amd64 ebs storage in the region the right region and copy the AMI-ID. That will be needed for launching the EC2 instance.

In the EC2 interface, launch a small instance of the AMI-ID from the previous step. You'll want to set up a security keypair, add an elastic IP address and Add ports 22 and 80 to the default security group.

A small instance has 1.7GB of RAM and 1 EC2 Compute Unit. This should be enough for moderate website traffic but it is easy to ramp the instance up for an additional cost with only a few minutes of downtime.

I'm not going to go into more detail here about starting an instance. Amazon has some excellent documentation at

General Housekeeping

Once logged in to the server lets start with a little house cleaning. This will make sure all the packages are up to date and install some packages we'll need later.

apt-get update
apt-get upgrade
apt-get install emacs vim git-core git-doc rsync unzip patch curl make drush

Install nginx

I've used apache since I started developing for the web a long time ago. I finally decided to try out nginx and will be sticking with it for the near future. So here is what is needed to install it on this server.

apt-get install nginx php5-cli php5-mysql php5-fpm php5-cgi php5-gd php-pear libpcre3-dev
service nginx start
update-rc.d nginx defaults

Install the site settings file. This one is customized for Drupal and is set to listen on port 8080 which is important for Varnish below. If you don't want varnish, either leave off the listen line or set it to 80.

Place this file in /etc/nginx/sites-available/

server {
  listen 8080;
  root /var/www/; ## <-- Your only path reference.

  location = /favicon.ico {
    log_not_found off;
    access_log off;

  location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;

  # This matters if you use drush
  location = /backup {
    deny all;

  # Very rarely should these ever be accessed outside of your lan
  location ~* \.(txt|log)$ {
    deny all;

  location ~ \..*/.*\.php$ {
    return 403;

  location / {
    # This is cool because no php is touched for static content
    try_files $uri @rewrite;

  location @rewrite {
    # Some modules enforce no slash (/) at the end of the URL
    # Else this rewrite block wouldn't be needed (GlobalRedirect)
    rewrite ^/(.*)$ /index.php?q=$1;

  location ~ \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    #NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_intercept_errors on;
    fastcgi_pass unix:/tmp/php-fpm.sock;
    #fastcgi_pass; # Use this if you didn't change the php-fpm listen.

  # Fighting with ImageCache? This little gem is amazing.
  location ~ ^/sites/.*/files/imagecache/ {
    try_files $uri @rewrite;
  # Catch image styles for D7 too.
  location ~ ^/sites/.*/files/styles/ {
    try_files $uri @rewrite;

  location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
    expires max;
    log_not_found off;

Symlink the file to sites-enabled.

cd /etc/nginx/sites-enabled
ln -s ../sites-available/

You may need to bump the memory limit up a bit in php. This depends on how many modules are installed on your site.
Next we need to set up some php settings. edit the file /etc/php5/fpm/php.ini

memory_limit: 256M

With the nginx config file we need to set another setting in php.ini


Next we want to switch where php-fpm listens from the IP address to sockets. This is slightly faster and matches the site configuration below.
In /etc/php5/fpm/pool.d/www.conf change

listen = /tmp/php-fpm.sock

Next install pac caching for php

pecl install apc

Add apc settings by creating the file /etc/php5/conf.d/apc.ini and put the following in it:
apc.shm_size = 256M
apc.apc.stat = 0

Install upload progress and drupal isn't happy without it.

pecl install uploadprogress
echo '' > /etc/php5/conf.d/uploadprogress.ini

Install MariaDB

This is another big change from me as I've always used Mysql. MariaDB is a drop in replacement but made by the non-Oracle founders of Mysql and with lots of improvements for speed and freedom.
Create the file /etc/apt/sources.list.d/MariaDB.list and put in it:

# MariaDB repository list - created 2012-07-11 21:37 UTC
deb precise main
deb-src precise main

Then install the key

apt-key adv --recv-keys --keyserver 0xcbcb082a1bb943db

Finally, use Apt to install mariadb

apt-get update
apt-get install libmariadbclient-dev libmariadbclient18 libmariadbd-dev libmysqlclient18 mariadb-client mariadb-client-5.5 mariadb-client-core-5.5 mariadb-common mariadb-server mariadb-server-5.5 mariadb-server-core-5.5 mariadb-test mariadb-test-5.5 mysql-common

Be sure to set and save your root password.

Install Varnish

The next step is we want a reverse proxy cache for anonymous visitors so nginx and php can get a break for repeated tasks.

Create the file /etc/apt/sources.list.d/varnish-3.list and put the following in it:

deb precise varnish-3.0

Then install the key

curl | apt-key add -

or this if you are running everything through sudo

sudo curl | sudo apt-key add -

Finally, us Apt to install varnish

apt-get update
apt-get install varnish

Now we need to set varnish up to run on port 80 and nginx to run on port 8080.

Edit /etc/default/varnish and change

DAEMON_OPTS="-a :6081 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"


DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,256m"

Next create a file at /etc/varnish/default.vcl and put in the following. This one is a generic drupal one that I've created from several sources and have had good luck with.

# A drupal varnish config file for varnish 3.x
# Will work with Drupal 7 and Pressflow 6.
# Default backend definition.  Set this to point to your content
# server. We are assuming you have a web server running on port 8080.
backend default {
    .host = "";
    .port = "8080";
sub vcl_recv {
  if (req.restarts == 0) {
    if (req.http.x-forwarded-for) {
        set req.http.X-Forwarded-For =
	req.http.X-Forwarded-For + ", " + client.ip;
    } else {
    set req.http.X-Forwarded-For = client.ip;

  if (req.request != "GET" &&
    req.request != "HEAD" &&
    req.request != "PUT" &&
    req.request != "POST" &&
    req.request != "TRACE" &&
    req.request != "OPTIONS" &&
    req.request != "DELETE") {
      /* Non-RFC2616 or CONNECT which is weird. */
      return (pipe);
  if (req.request != "GET" && req.request != "HEAD") {
      /* We only deal with GET and HEAD by default */
      return (pass);

  # Always cache things with these extensions
  if (req.url ~ "\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
    return (lookup);

  # Do not cache these paths.
  if (req.url ~ "^/status\.php$" ||
      req.url ~ "^/update\.php$" ||
      req.url ~ "^/cron\.php$" ||
      req.url ~ "^/install\.php$" ||
      req.url ~ "^/ooyala/ping$" ||
      req.url ~ "^/admin" ||
      req.url ~ "^/admin/.*$" ||
      req.url ~ "^/flag/.*$" ||
      req.url ~ "^.*/server-status$" ||
      req.url ~ "^.*/ajax/.*$" ||
      req.url ~ "^.*/ahah/.*$") {
    return (pass);

  ## Remove has_js, toolbar collapsed and Google Analytics cookies.
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js|Drupal.toolbar.collapsed|Drupal.tableDrag.showWeight)=[^;]*", "");

  ## Remove a ";" prefix, if present.
  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");

  ## Remove empty cookies.
  if (req.http.Cookie ~ "^\s*$") {
    unset req.http.Cookie;

  ## fix compression per
  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      # No point in compressing these
      remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      # unkown algorithm
      remove req.http.Accept-Encoding;

  # If they still have any cookies, do not cache.
  if (req.http.Authorization || req.http.Cookie) {
    /* Not cacheable by default */
    return (pass);

  # Don't cache Drupal logged-in user sessions
  # LOGGED_IN is the cookie that earlier version of Pressflow sets
  # VARNISH is the cookie which the varnish.module sets
  if (req.http.Cookie ~ "(VARNISH|DRUPAL_UID|LOGGED_IN)") {
    return (pass);

  return (lookup);

sub vcl_hash {
  if ( {
  } else {
  return (hash);

sub vcl_deliver {
  # From
  if (resp.http.magicmarker) {
    /* Remove the magic marker */
    unset resp.http.magicmarker;

    /* By definition we have a fresh object */
    set resp.http.age = "0";

  #add cache hit data
  if (obj.hits > 0) {
    #if hit add hit count
    set resp.http.X-Varnish-Cache = "HIT";
    set resp.http.X-Varnish-Cache-Hits = obj.hits;
  else {
    set resp.http.X-Varnish-Cache = "MISS";

  return (deliver);

The next step is to tell nginx to run on port 8080.
Edit all the files in /etc/nginx/sites-enabled and make sure that they are listening on port 8080. It should look something like this:

server {
	listen 8080;

Then restart nginx and then varnish

service nginx restart
service varnish restart

The last step is to make sure that the varnish module is downloaded to sites/all/modules and in your drupal settings.php files you have the following:

$conf['cache_backends'][] = 'sites/all/modules/varnish/';
$conf['cache_class_cache_page'] = 'VarnishCache';
$conf['page_cache_invoke_hooks'] = false;
$conf['reverse_proxy'] = true;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['reverse_proxy_addresses'] = array('');
$conf['omit_vary_cookie'] = true;

For more information on these options, see

Install memcache

Memcache will improve the performance of php by moving all the caching from sql tables to memory. If you have a lot of logged in users it will help a lot.

Start with some installs

apt-get install memcached libmemcached-tools memstat php5-memcached


Make sure that the memcache module has been downloaded to sites/all/modules and then in your drupal site's settings.php put the following:

Drupal 6

$conf['cache_inc'] ='sites/all/modules/memcache/';
$conf['memcache_key_prefix'] = 'something_unique';

Drupal 7

$conf['cache_backends'][] = 'sites/all/modules/memcache/';
$conf['cache_default_class'] = 'MemCacheDrupal';
$conf['memcache_key_prefix'] = 'something_unique';

For more information on configuring memcache for drupal see

Then make sure everything is loaded properly by restarting everything:

service nginx restart
service varnish restart
service mysql restart
service php5-fpm restart
service memcached restart

At this point you should be good to go. Put websites in /var/www/ and set up a site configuration in /etc/nginx/sites-available and symlink to /etc/nginx/sites-enabled.

I'd love to hear any other suggestions for improvements to this server. I haven't done any mysql or nginx tuning yet so I'm sure there are some improvements there. Let me know in the comments below.

Photo courtesy of

Fulfilling the Drupal Destiny: The missing marketing piece?

Last Monday 200 plus of the brightest minds in search marketing from Texas and across the country got together for the annual State of Search Conference. A lot of topics were covered. But one was missing that should concern every Drupaler. Drupal was missing, totally and completely.

The day kicked off with the always entertaining and insightful Chris Brogan. The best-selling author and top 5 AdAge blogger was hitting the highlights of his latest book, The Impact Equation; currently the #1 book on Amazon for web marketing. He was espousing how you need to build your home base, e.g. your own website. His recommendation: "WordPress is the best tool for this." Note he was not talking about just a blog, but a CMS for your whole website.

The second session I attended was titled Link Building, but really turned into a discussion about content strategies - then using that content to build links. Several tools were mentioned to facilitate the process - all WordPress plug-ins.

This theme continued all day long. In five of the six session I attended, WordPress got promoted as the weapon of choice for content strategy and online marketing. (The sixth session was about YouTube) Even in the closing keynote from Bing Webmaster Tools evangelist Duane Forester, WordPress got shout outs. From a Microsoft guy? Really?

How many times was Drupal mentioned? Not once all day.

Pish Posh; Marketers

Now it might be easy to blow off this demographic. After all, us Drupaler's; we are developers. We are engineers doing real work and heavy lifting. We are building big, complex sites for some very important people.

But the marketers at the State of Search are not your usual collection of campers. Many are seasoned PR, marketing and tech vets. Their clients are not a bunch of local small businesses and mommy bloggers. They are a list of the who's who of big brands. And while they might not always be building the websites, they have the ears of the decision makers who are – and they are demanding the tools they are familiar with.

After all the marketers are the people making money for these brands. Us developers are just seen as cost centers. I think is wrong, but it's true.

Tough love

I love the Drupal project and the community. I am a developer first and a marketer second - with a little bit of a competitive streak. Hands down Drupal is a superior technical platform. But, it was very concerning to hear such an influential group of marketers in lock step about their weapon of choice, WordPress.

I have seen too many times where enterprise class leaders have been crept up on and beaten by seemingly lesser, lighter solutions. It happened with voice response and databases in the early 2000. Drupal has done it by capturing much of the commercial eCMS market. Will WordPress creep up and take the enterprise space from Drupal?

As a community, we are not focused on Drupal solutions for marketers. For example, at DrupalCon Denver, there were zero sessions about marketing or for marketers. Just like we have evolved our theming and design in the last few years, we need to work on making Drupal a better marketing platform.

Josh Koenig in his keynote, Drupal's Destiny, at Dallas DrupalCamp shared his vision for Drupal to run 20%+ of the web. I love that vision.

We have the technology. We have the developers. We have worked hard to get the designers. Now we need to work on getting the marketers. I think it's critical for Drupal to reach its destiny.

Doing the Drupal thing? What do you think? How do we get to running 20% of the web?

Photo by: Junnuine Capturnes