A Comments Engine for Webby

The most common complaint about using Webby for a blogging platform is probably “with static HTML files how can you handle comments?”. The standard answers are:

  1. Don’t bother with comments.
  2. Use Disqus or some other 3rd party commenting plugin.
  3. Build some sort of commenting engine yourself.

Today I am going to describe my contribution to this third option. I have extracted the comments engine I use for this blog. For now, you need to install it from source since it’s not ready to be pushed out as a gem.

bzr branch http://ananelson.com/code/webby_email_comments

A gem and a nicer name will be forthcoming when I think they are deserved. For now, the code works but it’s very hard-coded to the way I do things, so if you aren’t comfortable modifying this to the way you work, you’re better off waiting until this is a little more polished and configurable.

Here’s basically how this works:

  1. Someone fills in a comment form on one of my blog posts.
  2. The form is posted to a PHP script which generates an email which goes to a dedicated email account.
  3. Every time I run a “webby build”, a script checks for new emails and pulls the contents into a local DataMapper database. The emails are marked as read once they have been successfully downloaded.
  4. When a blog post page is rendered, it checks in this DataMapper database and pulls in any comments for that page.

This does mean that you have to render every page freshly in order to include a new comment. There are other possible approaches, such as using JavaScript calls to incorporate comments dynamically. I took this one first since it was the easiest to implement, and it means that comments are available for indexing to search engines.

If I want to refresh my local cache of comments, I can simply delete the sqlite3 database, mark all the emails as unread, and the next time I build they will all be fetched again. If I receive a spam comment or an unwanted comment, I simply delete the email (or just mark it as read) before running “webby build”. I can also interact with comments, delete unwanted ones etc., via DataMapper. If you have legacy comments you want to include, you can just import them into the database.

Installing WebbyEmailComments

WebbyEmailComments depends on DataMapper and JSON. To get the source, you’ll need Bazaar and to build the gem, you’ll need bones.

sudo easy_install bzr # Assuming you have Python's SetupTools installed, or see bazaar-vcs.org for install directions
sudo gem install bones
sudo gem install json
sudo gem install dm-core dm-more

bzr branch http://ananelson.com/code/webby_email_comments
cd webby_email_comments
sudo rake gem:install

In your webby project Sitefile, you need to require "webby_email_comments".

Configuration

Webby uses the Loquacious config engine. This makes it very easy to add additional config settings to Webby, which is done in lib/config.rb in webby_email_comments:

Loquacious.configuration_for(:webby) {
  email nil, :desc => "Webby author's email address (your own personal email address)"
  
  comments {
    email nil, :desc => "Your dedicated email address for comments"
    login nil, :desc => "The login id to log in to the email server, probably the same as your email address"
    password nil, :desc => "The password to log in to the email server"
    reply_template nil, :desc => "A message automatically sent to everyone who posts a comment to let them know their comment was received. No message sent if this is nil."
    host 'imap.gmail.com', :desc => "IMAP server, defaults to gmail's IMAP"
    port 993, :desc => "IMAP port"
    ssl true, :desc => "Whether to use SSL for connecting to IMAP"
    db "email-comments-db.sqlite3", :desc => "Filename for database to store comments."
    php_bin "emailbin", :desc => "Location to put PHP handler files."
    inbox "Inbox", :desc => "The name of the Inbox IMAP folder. This folder is checked for new comment messages."
  }
}

You can set all of these new configuration options in your Sitefile, just as you do your other Webby options, as shown below.

You can view all the Webby configuration options and their current values by calling:

<%= Loquacious.help_for(:webby).show :values => true %>

It gives you quite a lot of detail, here is a short excerpt:


    Location to put PHP handler files.
     - comments.php_bin          => "emailbin"

    IMAP port
     - comments.port             => 993

That’s a very convenient way of seeing what settings are available, and checking that values are being set correctly.

Using WebbyEmailComments

Create a Comment Form

There is a comment form helper built in. In your layout template, just call <%= comment_form %> whereever you want the comment form to appear. I actually call the comment form like this:


    <%= comments_for_page %>
    <br />
    <% if @page.dirty -%>
    <%= comment_form %>
    <% end -%>

comments_for_page is the helper which displays the actual comments.

This means that in order to enable comments on a blog post, I must add dirty: true to the Webby metadata for that page. This is a good idea anyway since it will mean that new comments will get added to that page every time you build your site.

There are a few spam prevention attempts built in to WebbyEmailComments (I have no idea how effective they are as I haven’t studied my logs yet). One of them is to have a secret code built in to the form and the receiving PHP script. In order for this to work, the PHP script and the comment forms must be generated at the same time so they share the same key.

Hence, I recommend you follow the same procedure:

  • To enable comments on a blog post, set dirty=true.
  • In your blog layout, show the comment form only if dirty=true.
  • Ensure that dirty=true is kept in the PHP form (it’s there by default).
  • When you are ready to close comments for a post, simply remove dirty=true from the metadata.

Also, in your CSS you should have something like this:

.emphasise {
    visibility: hidden;
    display: none;
}

to hide the math question which is just there to trick bots.

PHP Processing

There is a template/ directory in webby_email_comments which contains the PHP script to process completed forms, along with a copy of XPertMailer, an open source PHP library to create emails. I have minimal PHP experience so if anyone wants to suggest a better library, I’m happy to add that in, although XPertMailer seems to work very well and is very easy to use.

This PHP script is dynamic, it gets processed via ERb each time to incorporate the secret key and to fill in user preferences like what your email address for comments is.

If you run the webby_email_comments executable (which should be available after you install the gem), from the root directory of your webby installation, it will create a directory called “emailbin” in your webby content directory and install your PHP handling and the XPertMailer library. You can change the name of this directory in the config.

You can then edit the PHP code yourself if you want to make any changes, and you should just upload this directory along with the rest of your site, taking care to check that the permissions on the PHP files are correct (e.g. not world-writable).

Moderating Comments

The easiest way to get started is to create a gmail account like “blog-comments-ananelson@gmail.com” and specify this in your Sitefile as follows:

require "webby_email_comments" # must require this before setting configs

SITE.email = 'ana@ananelson.com'
SITE.comments.email = 'my.comments.email.address@gmail.com'
SITE.comments.reply_template = "Hey, thanks for leaving a comment. :-) ..."
SITE.comments.login = 'my.comments.email.address@gmail.com'
SITE.comments.password = 'passw0rd' # yeah, right


The SITE.comments.email is your dedicated email for comments, SITE.email is your real email address and the reply_template is an acknowledgement message which gets sent back to people who leave a comment. (No acknowledgement is sent if reply_template is nil.) A copy of each comment gets sent to your personal email as well as the comments email so you know when you have received a comment. Just delete emails of unwanted comments in your comments.email, or mark them as spam if you want to try to train gmail (or whatever spam filter you use) to recognize spam.

I have a rake task to check for new comments.

task :comments do
  puts "initializing comments database..."
  WebbyEmailComments.db_init

  puts "fetch latest comments..."
  if WebbyEmailComments.process_inbox
    puts "ok"
  else
    puts "unable to connect to comments email, skipping..."
  end
end


process_inbox does all the work. It returns false if there has been a SocketError, such as when you don’t have internet access, and true if all is well.

By specifying task :build => :comments, you can ensure that the latest comments are fetched every time you rebuild. If you want to check comments more methodically, then don’t add this dependency and run webby comments manually.

Currently only IMAP is supported. The default host, port and ssl settings will work if you use gmail. Otherwise set the variables in your Sitefile to the appropriate values.

The database defaults to email-commends-db.sqlite3 in your webby root directory, you can change this (such as to put it in a comments/ directory or somewhere else out of the way) by specifying SITE.comments.db in your Sitefile.

Displaying Comments

As discussed above, place <%= comments_for_page %> in your template wherever you wish comments to be displayed.

Getting Help, Asking Questions

If you have questions, feel free to post a comment here or on the Webby mailing list. Once again, this is early days and for now this is intended for people who are pretty comfortable with using Webby already.




Ana Nelson 06 Jul 2009

Edited original post to clarify install instructions and to mention CSS for emphasise class.

JohnPlum 25 Sep 2009

Thank you. Very nicely delivered.