I have a confession to make. When I start a new work or freelance project, I often become obsessed with the search to find the perfect tool to help me get the job done. For some reason, I love the process of researching, testing, and comparing software and web applications and I jump at the chance to learn, explore, and experiment. I know it benefits me in the long run, but it can be quite time consuming and wasteful at the time. And to be completely honest, it’s one of my favorite and most justifiable ways to procrastinate.

An Example

So I’ve got a business idea I’ve been kicking around for awhile that I finally want to start working on. I’ll need to have a website up and running eventually, but there are plenty of more important things that need to get done in the mean time. Still, I find myself drawn to the search for what might be the best content management system (CMS) for my perceived needs. Here’s a rundown of some of the packages and tools I’ve considered, and why.

• WordPress: I considered WordPress briefly, as it is a really great blogging platform. It’s got basic CMS capabilities for managing a site, but not enough flexibility for what I’m thinking of doing.
• Movable Type: I’ve got years of experience with Movable Type, since I use it to manage the back end of this blog, as well as a couple others. Movable Type 4 introduced some really good CMS features, so this one is a real contender.
• MODx: I use MODx to manage a couple of websites at work, and while I was initially very excited about it, I grew tired of installing plug-ins and hacking PHP code to get things to work the way I wanted. While I know this one is capable, it’s not high up on my list.
• ExpressionEngine: Years ago I used an older version of ExpressionEngine to manage my church’s website. The site was a mess behind the scenes, something I partially attributed to EE’s design, but I know realize I simply didn’t know what I was doing. Based on PHP and MySQL, it’s both a strong blogging engine and content management system. It’s also the only commercial software on my list that charges for a license.
• TYPOlight: This one came up during my search, but I haven’t really invested much time looking at its details. It looks promising though.
• Drupal: Drupal seems to be the heavyweight PHP-based CMS. It has tons of features and is very extensible, but seems to have a big learning curve.
• Django: I’ve even considered building my site’s back end myself using a popular web development framework called Django. I’ve been wanting to learn the Python programming language for some time now, and this would give me the perfect excuse. And since Django was initially built to manage the Lawrence Journal-World website, I’m sure I could get it to do pretty much whatever I want — at least after I teach myself Python and how the framework itself works…
• Ruby on Rails: I’ve built a couple of web applications using the Ruby on Rails development framework, so the though of using it for my site has also briefly crossed my mind. Deployment is still painful compared to PHP or Python applications though, so I haven’t given it serious thought.

The Point?

My point is that it’s often easier to us (me) to feel like we’re (I’m) making progress on a project by diving into a fun but trivial part of it instead of doing the real and hard work. In my case, while my business website’s content will be extremely important in influencing potential customers to go with me instead of someone else, the content management system will have little bearing on how many customers I’ll get and how much money I’ll make.

Sure, I can make an argument that a framework like Django or Rails will give me maximum flexibility in workflow and integration with 3rd party tools and services, and that it’ll give me a competitive advantage. It’s not the core of my business idea though, so the more time I spend on peripheral activities means less time I have for real business development.

Still, I think it’s a very grey area. Researching and trying content management systems, billing and invoice apps, and wikis isn’t the same as writing copy for the website or doing marketing, but it’s still a hell of a lot more productive than reading digg or RSS feeds on Google Reader. That’s how I justify it, anyway.

Possibly related posts:

1. Movable Type 4 vs WordPress 2.2
2. Kick’in the Tires on Drupal
3. MODx: A Different Kind of PHP CMS
4. Website Redesign: Phase II
5. Playing With Movable Type 3.2 Beta 4

What the Google App Engine Is (In Plain English)

At its core, App Engine a limited, public interface to the technologies Google uses to power most of its web applications, including search, Gmail, and Google Maps, and many others. These technologies include the Google File System and Bigtable, the company’s home-grown software that allows their products to run reliably across thousands of distributed computers for millions of users. In other words, App Engine lets you use Google’s hardware and software to create web applications that can seamlessly handle a handful of users one day and millions the next.

It you’re not familiar with web application development, you might not understand the difficulty in getting a program to scale — to perform as well under high loads as it does with just a few users. Because web apps are usually developed on a single machine, deploying and running a production version to one server is extremely easy. It’s when you need to start adding more hardware resources — web servers, database servers, load balancers, etc. — that things start getting complicated and costly. It’s doable, for sure — just look at popular sites like Facebook, Youtube, Myspace, and Google itself. But, it requires a lot of expertise and a lot of money to do things right. For these reasons, web start-ups wisely hold-off on optimizing their applications until the demand is actually there. Twitter, facing constant scaling problems over the last year, is a perfect case study. So, to developers, App Engine means they don’t need to worry about these scaling problems — at all.

It’s All About the Constraints

The way Google’s distributed infrastructure and App Engine are designed creates some pretty strict constraints for applications running on them. This automatically rules out a bunch of programs that are simply too complex to run on it — mainly ones that require access to the operating system and those that need processes to run independent of users’ interaction with the web app (think scheduled tasks, long running scripts, etc.). The High Scalability blog has an excellent overview of the technical side of App Engine:

Everything that could have tied you to a machine is tossed. No disk access, no threads, no sockets, no root, no system calls, no nothing but service based access. Services are king because they are easily made scalable by load balancing and other tricks of the trade that are easily turned behind the scenes, without any application awareness or involvement. … What isn’t scalable about AppEngine is the scalability of the complexity of the applications you can build. It’s a simple request response system. I didn’t notice a cron service, for example. Since you can’t write your own services a cron service would give you an opportunity to get a little CPU time of your own to do work. To extend this notion a bit what I would like to see as an event driven state machine service that could drive web services. If email needs to be sent every hour, for example, who will invoke your service every hour so you can get the CPU to send the email? If you have a long running seven step asynchronous event driven algorithm to follow, how will you get the CPU to implement the steps? This may be Google’s intent. Or somewhere in the development cycle we may get more features of this sort. But for now it’s a serious weakness.

On the other hand, these exact same constraints make you develop in such a way that your code can instantly go from running on a single development machine to being served from thousands of servers from Google’s farm of computers. Some developers can’t, or won’t, live with Google’s forced limitations, either out of principle, or because their application requires a more complex environment. But for the wide range of potential web apps that do meet the App Engine requirements, it provides a unique opportunity to launch a product without worrying about how to grow the back-end as demand increases. Google does it for you — automatically.

What You Get as a Developer

1. A Python Application Environment: Currently, you can only develop and deploy apps to the App Engine that are written in the Python programming language. Python is heavily used by Google for its products and internal applications, so this limitation only makes sense. There are going to be a lot of PHP developers turned away by this, but Python is a wonderful and powerful scripting language that can be picked up easily by anyone familiar with Ruby or Perl. It also means you can develop using the excellent Django Web framework — a huge plus. Google also provides a free App Engine software development kit that, when downloaded and installed on your computer, locally simulates your app running on Google’s systems. Then, when you’re ready to deploy your application to App Engine, a single command does all of the work for you. Very slick.
2. A Secure Application Sandbox: This piece securely isolates your program from all others running on Google’s servers, and enforces the App Engine constraints by providing only limited access to the operating system. This does prevent you from calling command line scripts and other programs, but means that your app is independent of hardware, operating systems, and other typical dependencies. That means it can run as easily on 1,000 web servers as it can on one.
3. A Reliable Datastore: App Engine provides a simple method for storing structured data, which is based on Google’s BigTable distributed storage system (video link). It sacrifices many of the features of traditional relational database systems like MySQL, Microsoft SQL Server, and Oracle for speed, simplicity, and reliability. It is similar in design to Amazon’s SimpleDB product, and shares many of its advantages and disadvantages.
4. Google Account Integration: App Engine makes it extremely easy to use Google Account logins to authenticate users to your application. You can roll your own method if you like, but this offers a quick and secure way of doing it for both the developer and end users (they won’t need to create yet another account).
5. URL Fetching and Mail Services: Much as you’d expect, the mail API gives your application a way to send out email messages. The URL Fetch API is simple, but key. It’s how applications running on App Engine can consume Atom/RSS and web services from 3rd parties to extend their functionality.

Why App Engine Isn’t Amazon Web Services

As I hope I’ve made clear by now, there is a certain sweet-spot of web applications that can run within the current incarnation of App Engine. The environment isn’t a general-purpose utility computing platform where you “rent” computing resources based on usage. On the other hand, the Amazon Web Services are exactly this. Each of Amazon’s services work well together, but are separate so that you can pick and choose which ones to integrate into your program. If you want a cheap way to store files, Simple Storage Service is for you. If you just need a simple way of storing structured data, you can pick-up SimpleDB.

Amazon’s Elastic Computing Cloud service is the one that people are wanting to compare App Engine to, but that is, in fact, completely different. The EC2 provides a virtualized infrastructure that lets you build your environment exactly how you want. You upload your own Linux image, configured with the settings and applications you want. This could be a web app, but it could also be anything from transcoding audio or video files, performing scientific data analysis, or batch processing tax returns. There are absolutely no limitations to what you can or can’t do, with the one caveat that running databases on EC2 is not recommended since the server instances are transient by nature.

The upside to Amazon’s web services is flexibility — you can pick and choose the ones you want to use. The downside is complexity, particularly with the EC2 service. Sure, you have the ability to add or remove server instances on demand, but it’s your responsibility to monitor this situation and create the mechanism for automating the process. Software such as Scalr can ease this burden dramatically, but I imagine there are a fair number of really small companies that would prefer to never have to even think about the scaling problem.

Summing it Up

App Engine isn’t going to be the solution every web developer is looking for, but I believe that’s actually a Good Thing. By focusing on a narrow use case and keeping things simple, Google will end up creating a better experience for developers. For developers requiring something with greater flexibility and control, the Amazon web services and other competitors remain extremely good options.

Also, remember that the App Engine is a preview release. That means Google wanted to get it out the door and get feedback as soon as possible. It also means that they’re willing to add or change features to meet the needs of developers. So, some of the current pain-points might be addressed in the near future.

Further Reading

• Developers, start your engines
• App Engine: Host Your Apps with Google
• Campfire One: Introducing Google App Engine (pt. 1) (YouTube video)
• Developing and deploying an application on Google App Engine (YouTube video)
• Google Jumps Head First Into Web Services With Google App Engine
• Google App Engine: Cloud Control to Major Tom
• App Engine: Competition Is Good for Everyone
• Google App Engine: History’s Next Step or Monopolistic Boondoggle?
• Early notes on GoogleApps
• Analysis: Google App Engine alluring, will be hard to escape
• Thoughts on Google App Engine

Possibly related posts:

1. Tip of the Day: Customize Your Google Homepage
2. Book Review — The Big Switch: Rewiring the World, from Edison to Google
3. A9 — The New Google Rival
4. Google Public DNS and FUD
5. My Thoughts on Google Web History

I don’t think I’ve officially mentioned this on my blog before, but one of the two businesses I helped startup is a company called Greeneye e-Manifest. What we do, in a nutshell, is make very easy for Canadian trucking companies to submit an electronic manifest of what’s in their trailers, and where it’s coming from, and where it’s going to. We have a website where they can create and send these documents themselves, or we’ll do it for them from a fax or a phone call.

So, anyway, we’ve had a pretty good year with Greeneye, and now have upwards of 750 trucking companies as customers. We wanted to send out a little thank you to each of them, but wanted to do something a little different than the traditional Christmas card. So, two weeks ago, we came up with the idea of doing a little holiday cartoon. One of our guys found an amazing Flash animator named Vlad Kolarov who said he’d be able to get us a finished product in less than a week. Once we got the thumbs up to go ahead from our owner, three of us had a complete storyboard for the animation drawn up on a whiteboard in less than an hour. Then, I sent a quick design spec for some promotional graphics to Casey Jelinski, a freelance designer we’ve had the pleasure working with on a number of our marketing documents. After a few revisions from both Vlad and Casey, I had the final pieces in my inbox this past Monday.

During all of this, I worked on the landing page where this animation would live, as well as the design of the email we’d be sending out. I signed up for a great web service called MailChimp that would let us easily send out our greeting to customers while minimizing the chances of it getting treated as spam. It also produces some really nice reports that give all sorts of stats on the number of people clicking through the email to our website. I highly recommend it.

After lots of testing and tweaking, I had everything finished and ready to go about 3:30am Thursday morning. MailChimp started sending out our greeting at 9:45am Thursday, and since then, it’s been fun watching the reaction. You can check it out for yourself by visiting this page: Happy Holidays from Greeneye e-Manifest.

This whole process highlights one of the many things I love about working at a small business: We had an idea, and we executed it in less time than it would take someone at a corporation to even get approval for the project. It sure helps to be working with great people and flexible freelancers!

Possibly related posts:

1. Here’s an Idea: Customer Service Callbacks
2. About

The primary goal of most web development frameworks is to save development time. Since most programming projects share a good deal of redundant code (database connectivity, error handling, form validation and processing, etc.), that type of functionality can quite easily be extracted and put into a framework for reuse later.

This drastic time savings was one of the initial draws for the Ruby on Rails framework when it started gaining popularity a few years ago. Because Rails’ was created to be “opinionated” software, the framework makes a lot of choices for you and makes a lot of other decisions really obvious. In other words, Rails makes it easy to do the “right” thing and forces you to do some extra work if you want to go against that. Developers can (and do) debate the merits of injecting opinion into software, but to me it means that I can quickly get exposed to a ton of best practices that will make me a better programmer as a result.

One area of web design that has only recent seen movement toward standardized frameworks is in CSS for web design. It turns out that if you create enough multi-column page layouts using CSS, you end up creating similar foundations for each of the design projects. That’s where new CSS frameworks like Blueprint can save lots of time and help to create a very stable baseline to build upon.

Blueprint’s sweet spot is for designers who need to create grid-based multi-column website layouts. Setting this up on your own correctly takes some time and a lot of knowledge and experience, but Blueprint makes it about as easy as you can hope for it to be. You just download the framework, unzip it into a subdirectory where you keep your site’s CSS, and add an @import statement to your page to start using it.

Using Blueprint as a foundation for your site’s styles mean you’ll not only get it up and running in less time, but you’ll automatically have access to CSS best practices created by designers such as Jeff Croft, Eric Meyer, Nathan Borror, and Christian Metts. This makes the prospect of designing good looking, bulletproof CSS layouts a reality for a lot of web designers.

Another web development framework that I’ve been keeping my eyes on is the Zend Framework for use with PHP5. Created and hosted by Zend Technologies, the major contributor to the PHP language and creator of many popular PHP related software products, I consider the Zend Framework to be the closest thing there is to an “official” PHP framework.

There has been a flood of new competing PHP development frameworks since Ruby on Rails made its debut: I count 17 major ones listed on a Wikipedia compilation. I think I know why there are so many, though I think most of them are unnecessary and even harmful to the future growth of PHP and the community. I believe the reason is that PHP has a very low barrier to entry: It’s available on pretty much every web host and the basics are very easy to learn. Because of that second one, there’s a very pervasive “build it yourself” mentality in the PHP crowd that causes many developers to write software from scratch instead of adopting and extending an existing project.

Back to problem of having too many PHP frameworks. Some will tell you that having more to choose from is better — these are the same people that build software with page after page of features. Think about it from the perspective of a new developer though. With 17+ web frameworks to pick from, where do you start, and how do you compare? Unfortunately, chances are that you probably won’t. It’s the classic Paradox of Choice: Faced with too many choices, you most likely won’t pick any of them. In this case, you might even go off to create your own web development framework.

So, my hope with the Zend Framework is that it’s able to channel much of the energy currently going into most of the other PHP frameworks. It’s well designed, well tested, has a business friendly open source license, and offers tons of components available for use. I think it’s the obvious framework choice for PHP programmers.

Possibly related posts:

1. Web Frameworks or: How I learned to stop worrying about PHP and love Ruby
2. Swat — Another PHP Web Framework
3. Ruby on Rails 1.0 Goes Live
4. Ruby on Rails is a Go!
5. Getting Stuck on Tools

Just the other day I was considering moving this blog into WordPress instead of the various versions of Movable Type I’ve been using to run this thing for almost four years now. There were a few reasons I was thinking about doing this:

1. I’m a tweaker, and like to try new things
2. I’ve used WordPress to manage several other personal blogs, and have been very impressed
3. I know PHP much better than I know Perl, so digging around and making changes to WP would be a lot easier
4. A lot of other web developers made the move to WordPress a couple of years ago, so there are tons of new and interesting plug-ins available
5. It’s released under the GPL open source license, which ensures it’ll always be free, in both senses of the word.

So although I like WordPress and will continue to use it for the other sites I run, I’m glad I didn’t make the switch on berbs.us quite yet. Just yesterday, Six Apart announced Movable Type 4, their first big release of the blogging software in about three years. A of the 50 new features got my attention and are making me excited to download and try out the beta:

1. A New User Interface: Movable Type has had the same basic administration interface for as long as I’ve been using it. There have been improvements and tweaks, sure, but no big leaps in the experience have been made for a long time. Version 4 sports a completely redesigned UI, and looks like it’ll be very easy to use.
2. OpenID: MT4 includes built-in support for OpenID, the distributed authentication system that is gaining traction and may someday be the method you use to login to websites.
3. Content Management: The new version of Movable Type will include ways to create and manage non-blog content for your website. This is a really big deal for me (more below).
4. Open Source: Movable Type 4 will be made available under the GPL license sometime later this summer. That means it’s not only free to use, you’re free to make modifications and improvements to give back to the community. There will still be a paid commercial version for businesses, schools, and others who need official support.

One of my biggest accomplishments while working at EduTech was a complete website redesign that used Movable Type as its content management system. The initial setup was not easy — I think it involved something like 10 different weblogs for each of the different sections of the site. It also meant using a lot of complicated PHP template files that still causes my head to hurt just thinking about it. I’ll just say this: If I were doing that project again today, I would not use Movable Type 3 to manage a complex website.

Movable Type 4 seems to be a completely different animal though. With the built-in ability to create and organize regular web pages, it could turn out to be a very capable light-weight CMS.

I’ve been using MODx CMS for several projects at my current job (which I’m still not at liberty to officially announce on my blog yet) with some success. There’s absolutely no integration with our Movable Type blogs though, which means two completely different interfaces for managing different parts of the site. It’d be great to bring everything in under the same system, and MT4 just might give me that opportunity.

I’m thinking of using our church website as a testing case for Movable Type 4. I still haven’t made any changes to it after I considered moving it to Drupal and MODx last year, so this would be the perfect opportunity to see what MT4 is capable of.

Possibly related posts:

1. Tempting Fate With Movable Type 3.0
2. Movable Type Upgrade
3. Required Reading for Movable Type Users
4. Playing With Movable Type 3.2 Beta 4
5. How to Generate a Yahoo Sitemap with Movable Type

My Facebook lovefest continues. Late yesterday they announced the latest version of the Facebook Platform, which opens up the site (and its millions and millions of users) for 3rd parties to build on. If that doesn’t mean anything to you, just know this: It’s a very big deal.

TechCrunch calls Facebook the “Anti-MySpace”, and for good reason. MySpace gives users the appearance of openness by letting them make their pages look as crappy as they want, but does almost everything they can to prevent users from interacting with outside sites and services. For example, when it temporarily blocked Youtube videos back in 2005, and more recently, when it blocked Photobucket content before buying them a month later.

While Facebook doesn’t let users change the way pages look (one of its best features, in my opinion), it instead opens itself up to 3rd party developers with a really impressive API that’s got great documentation. Existing applications can now be integrated right into the platform so they look and feel like they’re part of the site, and entirely new apps and business ideas can (and will) be created to reach Facebook users.

I love how Zuckerberg is thinking long-term with Facebook. He could have easily sold the company and cashed out, or littered the site with tons of ads, but he didn’t. Instead, he’s building a platform that will make Facebook one of the most powerful (and money-making) sites for years to come. He’s doing everything right, as best I can tell. Pure genius.

Since Facebook/Twitter integration has already been done on the new platform (it’s one of the first 3rd party apps), next I’m hoping to see a Flickr application that will act as a drop-in replacement for the Facebook photo app. I’m sure someone is already working on it.

I’ve already got a few good Facebook app ideas in my head that could make a ton of cash if correctly executed. Of course, my lack of time will prevent me from starting anything, but that’s a different story…

Possibly related posts:

1. Thrift: Facebook Open Source Tool
2. Fast Company on Mark Zuckerberg and Facebook
3. Facebook, MySpace, and Class Divisions
4. Google App Engine: Embrace the Constraints
5. The Salesforce Blitzkrieg

Browsing through Thrift’s whitepaper (PDF), I’m still trying to completely understand it. It appears to provide a consistent way for different programming languages (PHP, Python, Ruby, Java, C++) to communicate with each other. In other words, it lets you build a web application by using the best programming language for a particular task, then tie the pieces together in a clean way.

I think Facebook’s site is the best testimonial that Thrift framework could ever get. I’ve been using the social networking app for awhile now, and it’s always fast, and always online (compare that with MySpace). Hopefully some other entrepreneurs will be able to make good use of this tool too.

Possibly related posts:

1. Facebook Loves Developers, and I Love Facebook
2. Fast Company on Mark Zuckerberg and Facebook
3. Open Source Software and the Future of Business
4. The Open Handset Alliance, or Why Open Standards are for Losers
5. Kindersay: Free Online Videos that Help Your Preschooler Learn New Words

Improving the Firefox Interface

Even though Firefox is amazing out of the box (or, more accurately, after the download), I use a few extensions to fill in some gaps.

Adblock Plus

Adblock Plus blocks so many web ads, it’s really tough browsing on a computer where it’s not installed. It comes with a set of filter subscriptions that will automatically stay up-to-date so you only have to see the ads you really want to. Highly recommended.

Download Statusbar

Firefox’s default download manager annoys the heck out of me, but Download Statusbar fixes this. Just as its title says, it displays a download status bar at the bottom of the browser window to let you know when your files are completely retrieved.

Tab Mix Plus

I’m a huge fan of tabbed browsing, so I appreciate the extras offered by the Tab Mix Plus extension. Among other things, it color codes unread tabs, gives you the download status for each one, and lets you lock and protect tabs so you won’t close them accidentally.

Fission

One of the nice little touches I appreciate in Apple’s Safari browser is the address bar that doubles as a progress indicator. The Fission extension’s only job is to add this feature to Firefox, and it does it well.

ErrorZilla

ErrorZilla replaces the default Firefox error page with a set of options, including going to a Coral Cache version of the page (if it’s available), visiting the Wayback Machine, or doing a trace route or ping to see if the web server is down. It comes in handy every so often.

Power Tools

Here are some other extensions that don’t fit into the above category:

Greasemonkey

Greasemonkey is a little hard to describe. It lets you create little scripts that can modify pretty much any webpage right before it’s rendered in Firefox. You don’t have to actually write scripts yourself (there are tons available for download), though there’s a good guide available for free in case you get adventurous.

One of my favorite user scripts is one that creates a “smart” subscribe subscribe button for every page that has an RSS feed. If I’m already subscribed to that feed in Google Reader, it displays a little check mark letting me know I’ve already got it. A nice feature, especially when you have a couple of hundred subscriptions.

iMacros for Firefox

To be honest, I have the iMacros extension installed, but haven’t done much with it yet. I like the idea a lot though. It lets you write macros for a web page that automate repetitive tasks.

PwdHash

PwdHash makes it easy to create site-specific passwords, giving you a higher level of security than if you just use a single password everyone.

Operator

Operator exposes Microformats present on pages and makes it easy to work with them. A little handy now, but Firefox 3 will probably have this sort of functionality built right in.

Tails

Tails is another Microformat reader, though this one is lighter-weight than Operator. Basically does the same thing though.

Attention

Attention Recorder

Right up front, I admit this one is pretty geeky. There’s a small movement going on right now advocating the ownership of your personal attention data made as you use the web. I’ll write more about this at some point in the future.

For now though, Attention Recorder captures your clickstream and browsing history in a file on your computer which you can do whatever you want with. You can also choose to upload this same data to a trusted 3rd party such as Root Vault for storage and analysis.

del.icio.us extension

I’ve been storing my bookmarks and links of interest to my del.icio.us account for almost two years now instead of saving them in Firefox. Not only are they now easier to search and available from any computer, there’s also the side benefit of sharing them with everyone else. I’m just that nice of a guy!

The del.icio.us extension makes it really easy to tag and save pages as you browse, and gives you quick access to your account when you need it.

Web Development

As you probably know, I’ve been big into web design and development for a while now. There are some awesome tools available for Firefox that makes the process easier. Here are some of my favorites.

Firebug

Firebug is unbelievable. This tool has become invaluable in debugging CSS and JavaScript. You can easily see how styles cascade (or don’t) across your HTML elements and edit them on the fly to see how things look in the browser. It can also debug AJAX requests, something that has personally helped me in my Ruby on Rails projects.

If you do any sort of web design or programming, do yourself a favor and download Firebug. It’s free, but it’s one extension I’d actually pay money for.

Web Developer

I’ve been using the Web Developer extension since 2004, and it has become an indispensable part of my web design toolkit. Some of its functionality is duplicated by Firebug, but there’s still a ton of other features there, including a “view generated source” option, which lets you view the HTML as it’s currently displayed in the browser window. This is one of those things that come in really handy when debugging AJAX and JavaScript that modify the Document Object Model after the page is actually downloaded from the server.

Html Validator

I’ve become used to regularly validating my HTML documents during development, and the Html Validator extension makes this process automatic. Instead of manually going out to the W3C validator service, this extension will do the validation locally on your computer right after the page is rendered in Firefox. This can save a lot of time over the course of your project.

I’ll make one small note on this extension: I have run into some cases where this extension says a page has valid markup, but where the official validator picks out some problems. Even so, it’s still a good first line defense.

Live HTTP Headers

Useful mainly when doing web development/programming, Live HTTP Headers lets you take a peek at the conversation going on between your computer and the web servers it gets content from. You probably won’t use it on a daily basis, but it comes in handy in special cases.

Professor X

Professor X lets you take a look at the content of a page’s “head” section without viewing the source. Nothing earth shattering here, but still a nice option to have.

SEO For Firefox

The SEO For Firefox extension adds a bunch of search engine optimization links and resources that’ll give you better insight into a page’s search rankings, along with a bunch of other useful info.

Well, there you go. I hope you found one or two tools that can help you out. If you’ve got a favorite extension, please let me know about it in the comments.

Possibly related posts:

1. My Favorite Tech Things for 2008
2. Extensions vs Bookmarklets
3. Will Ubiquity for Firefox Change the Way We Use the Web?
4. Great Software
5. Three Great Uses For Dropbox

I recently deployed a Ruby on Rails application to Media Temple’s Dedicated Virtual Server 3.0 setup, and after running into a few problems along the way, eventually got everything working nicely.

I decided to publish my steps and documentation for anyone else who might possibly benefit from what I learned. If you’re not in that very small group of people, there’s probably not much for you to see here. If you’re still curious though, continue reading after the jump.

Why Another Rails Deployment Tutorial?

You’re right, there are already a bunch out there, some of them really good. Media Temple even has one for their Dedicated Virtual Server 2.0, though it’s a litle out-of-date and somewhat incomplete toward the end. I wanted to fill in the gaps until MT releases an updated article in their knowledge base.

Why Not Use Mongrel Clusters Instead of FastCGI?

The current preferred method for deploying Ruby on Rails apps is to use Apache, Pound, or some other type of reverse-proxy to pass HTTP requests to a cluster of Mongrel web servers.

The problem? Media Temple lets you manage your dedicated virtual server using Plesk 8.1, which is overall a really good way to do things. The core web server services (Apache, MySQL, PHP, among others) are managed and updated by Plesk, which is where the issue lies. Apache 2.0.52 comes installed, which is not compatible with the mod_proxy_balancer module used by the most common Mongrel setup. Apache 2.2 is needed to use that.

You could compile Apache 2.2 on your dedicated virtual to take advantage of mod_proxy_balancer, but this will drop you out of MT’s Update Option Program, which basically means you’re left to do every security update and version upgrade on your own. You’re free to go that route, but I’m not ready to opt-out of the UOP just yet.

If you want to stay eligible for Media Temple’s UOP, you can also try using Apache 2.0 as a Mongrel frontend with the older mod_proxy module or hacking together a homemade proxy with a randomizer and a text file. If you want to keep it relatively simple for now though, I’d stick with the officially supported FastCGI method until Apache 2.2 is officially supported on the Dedicated Virtual Servers.

Assumptions

I’m going to assume a few things in this how-to. Call it an “opinionated tutorial” if you’d like:

1. You know your way around the command-line shell, particularly bash.
2. You already know how to use a text editor like vim or emacs and will use it to modify your text files on the web server.
3. You’re using Subversion, CVS, or some other source management system to keep track of your code. This is a good practice on its own, but it’s a requirement for the next assumption.
4. You’re going to use Capistrano to automate the deployment of your Rails application and that it’s already installed on your local development machine. If you’re not using it already, you should be. Trust me.

Before You Start

Before you do anything else, make sure you have the following ready on your DV server:

1. Developer Tools — Media Temple staff will need to install the compliers and other tools necessary to build programs from source. Submit a support ticket to request this.
2. Root access — You’ll need root access enabled before compiling any programs. Submit a support ticket to Media Temple to request this.
3. A domain account created in Plesk with mod_fcgid support enabled — You can do this easily in Plesk using the control panel. Make sure your domain user has shell access and that a MySQL database has been created.

Optional, but recommended: At this point, it’s also a good idea to give your domain account’s user sudo rights, especially if Capistrano will be using it to automate your application deployments. To do this, issue the visudo command when you’re logged in as root then add a new line so it looks something like this:

root ALL = (ALL) ALL <username> ALL = (ALL) ALL

1. Login to the server via SSH as root

You’ll need root access in order to continue, so login with that account now.

2. Download compile and install the latest version of Ruby

At the time of writing this, the current version of Ruby is 1.8.5. You can always find the latest supported version by visiting the Ruby on Rails download page.

cd /usr/local/src wget http://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.5-p2.tar.gz tar xvzf ruby-1.8.5-p2.tar.gz cd ruby-1.8.5-p2 ./configure make make install cd ..

Note: If you receive error messages at the ./configure step, it’s probably because Developer Tools aren’t installed.

3. Download and install the latest version of Ruby Gems

You’ll need RubyGems in order to install Rails and some other required pieces of software, so we’ll do that now.

The current version as the time of writing is 0.9.1. You should already back in the /usr/local/src directory before continuing.

wget http://rubyforge.org/frs/download.php/16452/rubygems-0.9.1.tgz tar xvzf rubygems-0.9.1.tgz cd rubygems-0.9.1 ruby setup.rb

4. Install the latest version of Ruby on Rails

The gem command will automatically download the latest stable version of Rails, which is 1.2 at the time of writing.

gem install rails --include-dependencies

5. Install the required gems and dependencies

Next, install other required Gem libraries. The first line assumes you’ll be using MySQL as your database server.

gem install mysql \-- -|-with-mysql-include=/usr/include/mysql --with-mysql-lib=/usr/lib/mysql gem install rake --include-dependencies gem install capistrano --include-dependencies gem install termios --include-dependencies

6. Download and Install FastCGI

Plesk already has mod_fcgid installed and enabled with Apache if you checked the mod_fcgid support option in your domain setup. You’ll still need FastCGI installed in order for things to work correctly, so we’ll do that now.

The current version of FastCGI as of writing is 2.4.0.

Make sure you’re in /usr/local/src/ before continuing. wget http://fastcgi.com/dist/fcgi-2.4.0.tar.gz tar xvzf fcgi-2.4.0.tar.gz cd fcgi-2.4.0 ./configure && make && make install

7. Install the FCGI gem

With FastCGI installed, you’ll need to install the FCGI RubyGem that will allow Ruby on Rails to use it. gem install fcgi

8. Enable the vhost.conf file for the domain:

There will already be a couple of Apache configuration files created on your server, the master one located at /etc/httpd/conf/httpd.conf, and the domain specific one at /var/www/vhosts//conf/httpd.include. Do not modify either of these! Both of these can be overwritten by Plesk, so your configuration changes aren’t guaranteed to be safe.

Instead, you create a vhost.conf and/or a vhost_ssl.conf inside the same directory as httpd.include to contain your changes. You might as well do both at the same time, just in case you decide to use SSL at some point.

touch /var/www/vhosts//conf/vhost.conf touch /var/www/vhosts//conf/vhost_ssl.conf

Creating these files is not enough. Now you need to tell Plesk about them so they’ll be automatically included in your domain’s httpd.include file:

/usr/local/psa/admin/sbin/websrvmng -u --vhost-name=

9. Prepare for Deployment with Capistrano

At this point, I’m assuming your application is tested and ready to go on your local development machine. It should also be “capistranized” and your deploy.rb modified to suit your situation.

I suggest creating an app directory to hold your Rails application and telling Capistrano to use the domain’s username to do the deploy. You could do it as root, but you’re far less likely to run into permission problems of you use a regular user instead.

set :deploy_to, "/var/www/vhosts//app/#{application}" # defaults to "/u/apps/#{application}" set :user, ""

On your development machine, run the following to setup the correct directory structure for your Rails app on your dedicated server:

cap setup

If Capistrano was able to do its job, you should now see some subdirectories with app on your server.

10. Edit the vhost.conf file to point to the public directory of the rails application

My first choice was to use a Symbolic Link to point httpdocs to the public directory within my Rails application, but I kept getting 403 Forbidden errors that I couldn’t work around. Instead, you can use the vhost.conf file you just created to tell Apache to use the Rails public directory as the web root instead.

In this case, current is the symlink created by Capistrano that points to the latest version of your deployed application.

DocumentRoot /var/www/vhosts//app//current/public /app//current/public> AllowOverride All Options +FollowSymLinks Options -Includes -ExecCGI  AddHandler fcgid-script .fcgi Options +FollowSymLinks +ExecCGI  

Restart the Apache webserver by issuing the command httpd –k graceful. This will make your configuration changes take effect.

11. Edit the htaccess file in the rails application /public directory

Before deploying your application to your dedicated server, on your development machine edit the .htaccess file located in the public directory of your Rails app:

• Around line 2 change AddHandler fastcgi-script .fcgi to AddHandler fcgid-script .fcgi
• Around line 32 change RewriteRule ^(.*)$ dispatch.cgi [QSA,L] to RewriteRule ^(.*)$ dispatch.fcgi [QSA,L\]

This will tell Apache to use the built-in fcgid module (line 2) and tell the Rails app to use FastCGI instead of the much slower regular CGI.

12. Deploy Your Rails App

On your development machine, issue the following command when you’re at the main directory of your Rails app:

cap deploy

This will tell Capistrano to get the latest version of your application from your source management system, create a new directory for it, point current to its public directory, then restart the FastCGI processes.

You should now be setup to reliably deploy new versions of your application to your Media Temple Dedicated Virtual Server. That wasn’t so hard now, was it?

Possibly related posts:

1. Media Temple — New Web Host of Choice
2. Ruby on Rails is a Go!
3. Ruby on Rails 1.0 Goes Live
4. This blog is moving to Media Temple
5. Refactoring Rails

David has got some really good examples of what’s wrong with software today. His book blog (Suckbusters) covers some of the same ones he talks about in the podcast. His basic thesis is that people don’t care about the inner workings of an application — they only care about completing the task they’re doing. Like he says, people don’t go to Home Depot and buy a drill because they just want a drill — they want the holes the drill lets them make. That may seem like a really simple and obvious observation, but developers forget it all the time.

One of my recent pet peeves is programmers who assume you’re using their web application all the time. One example is MyCheckFree.com — an online bill pay service that we’re required to use by several companies. It works pretty well except for one thing — its internal messaging system used to inform you of any problems that occur while drawing from your account or paying a bill. It looks like this:

This information is great, but there’s one problem: In order to see it, I need to log into their website. I usually only go to this site once a month when I pay our American Express bill (our others are on auto-pay). That’s probably going to be a little to late to fix any problems that have shown up in my inbox.

The better thing to do would be to just email me if my attention is needed. They’re already doing this when a new bill arrives and needs to be paid, so it wouldn’t be a big leap to do it here too. Sure, keep the message inbox on the site in case I want to see a history of messages, but don’t assume I’m checking in daily. The world doesn’t revolve around you!

Possibly related posts:

1. PeopleSoft Still Sucks
2. Symantec MySupport Follow-up
3. Prerequisites if You’re Creating a Desktop or Web Application
4. Thinking About Software and Lock-in
5. Great Software