WebMake
WebMake Documentation (version 2.4)

Contents

Contents

Contents for the 'Introduction' section

The Blurb

WebMake is a simple content management system, based around a templating system for HTML documents, with lots of built-in smarts about what a "typical" informational website needs in the way of functionality; metadata, sitemapping, navigational aids, and (of course) embedded perl code. ;)

  • Creates portable sites: It requires no dynamic scripting capabilities on the server; WebMake sites can be deployed to a plain old FTP site without any problems.
  • No need to edit lots of files: A multi-level website can be generated entirely from 1 WebMake file containing content, links to content files, perl code (if needed), and output instructions.
  • Useful for team work: Since the file-to-page mapping is no longer required, WebMake allows the separation of responsibilities between the content editors, the HTML page designers, and the site architect. Only the site architect needs to edit the WebMake file itself, or know perl or WebMake code. Standard file access permissions can be used to restrict editing by role.
  • Efficient: WebMake supports dependency checking, so a one-line change to one source file will not regenerate your entire site -- unless it's supposed to. Only the files that refer to that chunk of content, however indirectly, will be modified.
  • Supports content conversion, on the fly: Text can be edited as standard HTML, converted from plain text (see below), or converted from any other format by adding a conversion method to the WebMake::FormatConvert module.

  • Edit text as text, not as HTML: One of the built-in content conversion modules is Text::EtText, which provides an easy-to-edit, easy-to-read and intuitive way to write HTML, based on the plain-text markup conventions we've been using for years.

  • Rearrange your site in 30 seconds: Since URLs can be referred to symbolically, pages can be moved around and URLs changed by changing just one line. All references to that URL will then change automatically. This is vaguely Xanalogical.

  • Scriptable: Content items and output URLs can be generated, altered, or read in dynamically using perl code. Perl code can even be used to generate other perl code to generate content/output URLs/etc., recursively. New tags can be defined and interpreted in perl.
  • Extensible: New tags (for use in content items or in the WebMake file itself) can be added from perl code, providing what amounts to a dynamically-loaded plugin API.
  • Inclusion of text: Content can incorporate other content items, simply by referring to it's name. This is a form of Xanadu-style transclusion.
  • Edit content in your web browser: WebMake now includes webmake.cgi, which provides a CGI front-end to editing and managing a WebMake site.
  • Site replication: with webmake.cgi's CVS integration, multiple copies of the same site can be replicated, and changes made on any of the sites will be automatically replicated to all the others.
  • Version control: changes made to sites using webmake.cgi will be kept under CVS version control, so older versions of the site can be "rolled back" if necessary.

But enough of the bulleted lists. Here's where you should start:

  • First of all, read WebMake Concepts for a quick intro to the assumptions and concepts that are used in WebMake.
  • Next, read WebMake Operation for an overview of how WebMake operates.
  • Then, read How To Migrate to WebMake for a guide to bringing an existing, simple web site into WebMake.
  • After that, you just need to read the rest of the manual, which is mostly reference text. Good luck!

Concepts

Here's a list of the main concepts behind WebMake's design and implementation. Before using WebMake, it'll probably help to have a read of this, so you can understand where the functionality is coming from.

1. Templating

When you start working with the web, it's easy enough to write a few pages and put them on your site. However, you quickly realise that they all look different; there's nothing binding them together as one "site".

The next step is to add some common elements to tie the pages together, so you add some header text or graphics, and maybe a table on one side listing the other pages in the site, allowing your users to quickly find the other pages. Maybe you add some information at the bottom of the page, describing who you are, that kind of thing.

After a while, you'll have quite a few pages, each with a different piece of main content, but a lot of them sharing some, or all, of the shared elements -- the templates.

One day, you need to change the templates -- but there's no easy way to do this, without manually editing each of the files and changing them by hand. Wouldn't it be easier to just change this once, and be done with it?

That's one of the main features of WebMake: templating. It allows you to define the templates in one place, then generate pages containing the content wrapped in those templates.

There's quite a few products that do this; WebMake differs in that it's very flexible in how you can include your content text in the templates. Often, other products are limited to just setting a header and a footer to be added to each page; WebMake takes its cues from traditional UNIX tools by allowing very deep recursion in its templating, so your templates can include other templates, etc. etc.

2. Edit Text As Text, Not HTML

In some situations, you'll want to write HTML; but in others, text is best, for ease of editing, and reading while you're editing. WebMake supports Text::EtText and POD formats, converting them to HTML on-the-fly.

Text::EtText aims to support most of the de-facto conventions we've been using in mail and in USENET for years, converting them into HTML in a sensible way.

3. Breaking Down the File-Per-Page Mapping

Another annoyance comes from the default way a web servers serves web pages; normally, each web page is loaded from a separate file.

This is fine for some sites, but in other circumstances you might want to produce lots of small pages, or include identical text in several pages; or you may just prefer editing your entire site in one editor, rather than having to switch from one window to another.

WebMake allows you to specify several content items inside a single WebMake .wmk file (the .wmk file uses WebMake's XML-based file format), and/or load content from a data source, such as a comma-separated values file, a directory tree, or (possibly in future) an SQL database.

You can then include these content items into the generated web pages, whichever way you wish, based on the outputs and templates you specify in the WebMake file.

4. Support URL Changes

At some stage, you may feel like rearranging your site, changing one URL that's always bothered you, so that it becomes more aesthetically pleasing or descriptive. Or maybe some directive might suddenly appear, ordering you to do so for policy reasons (ugh). Whatever!

WebMake allows you to track output pages or media, such as images, or non-WebMake generated pages, using URL references; references to the name will be converted to the correct URL for that page or image.

5. Site Mapping and Indices

The obvious next step is to allow site maps, indexes, and navigational information to be generated automatically.

WebMake accomplishes this using metadata; in other words, if you tag your content items with information like its title, its "parent" content item, and its precedence compared to its neighbours (to specify the order of items), WebMake can automatically use this information to generate the following maps:

6. HTML Cleaning

Often, the HTML you'll have to work with may be crufty, with img tags that have no size information, or other inefficiencies.

WebMake includes a HTML cleaner which will rewrite your HTML until it sparkles. It can also be turned off for a "HTML verite" effect, if you feel so inclined. (Alright, it's also a little faster with the cleaner off. Not much though ;)

7. Plugins, User-Defined Tags And Perl Code

You can define your own tags, similar to how JSPs support taglibs; this provides a way to add scripted components to your pages, without making things too messy or confusing, or arbitrarily peppering code into the text.

Or, if you like peppering code into your text, WebMake provides support for Perl code embedded directly into the text or WebMake file, similar to PHP, ePerl, HTML::Mason, or ASPs. It also provides an API for that code to examine or alter WebMake's behaviour.

There's a plugin architecture as well, providing an easy way to load code on demand from self-contained components.

8. The Web Is "Read-Mostly": Bake, Don't Fry

Several other similar web site management systems revolve around dynamic code running on the web server, which assembles the pages as they're requested by the client. In the terminology used by Ian Kallen when building Salon.Com, they "fry" the pages on-demand.

For most sites, the pages do not change based on which client is accessing them, or if they do, they don't change entirely; perhaps an extra set of links becomes available in the page footer allowing a logged-in user to make modifications using CGI, or PHP or Perl code, but that would be it. The page just isn't volatile enough to require continual re-generation for each request.

As a result, all this churning about, generating pages on the fly from its raw components each time, is wasted; it just eats the server's CPU and memory for no real gain, and introduces yet another breakage point (databases, memory usage, the /. effect...) where things can go wrong, just when you're not looking at it.

WebMake takes the "baking" approach, generating virtually all its output before the web server gets involved. The web site admin runs the webmake command, and this generates the pages.

Note that WebMake doesn't preclude dynamic content in the pages, however. PHP, CGI, ASP or embedded Perl code can be used, and WebMake will not interfere. In fact, a future version of WebMake will probably provide some "fried" features of its own...

9. Site Replication

You can replicate web sites quickly, easily, and securely over the internet. WebMake does this using CVS and SSH, two standard UNIX utilities that have been used for years to do exactly the same thing for other types of data; why not web sites?

A bonus of using CVS is that you also get seamless version control and conflict management, so users can edit a WebMake site at any replicated point, check in the changes, and it won't overwrite everyone else's modifications.

10. Edit-In-Browser

The WebMake distribution includes a CGI script which provides a simple interface allowing a WebMake site to be edited over the web, and the changes to be checked in to CVS. At the moment, it's not too user-friendly, so it's not quite suitable for a newbie to use without some instruction -- but it's getting there, and it'll improve.

It's certainly handy for an experienced user who wishes to correct a typo or add a new page to their site, without requiring command-line access to the server; so if you check out your site in an internet cafe and spot a typo, you can immediately fix it without downloading an SSH client! ;)

WebMake Operation

First of all, WebMake relies on a WebMake file. This is an XML file, with a filename ending in .wmk, containing most of the important data on the structure, inputs and files that make up your site.

Finding The WebMake File

If you run WebMake without a -f or -R switch on its command-line, it'll first search for a file ending with .wmk in the current directory, then in the parent directory, and so on 'til it hits the root directory.

You can specify exactly which file to build from by using the -f switch. Alternatively if you use the -R switch, it'll search relative to the filename specified on the command-line; this is very handy if you're calling WebMake from a macro in your editor or IDE, as it means you don't even have to be running the editor in the same working directory as the files you're working on.

The WebMake File Structure

A WebMake file is made up of several conceptual chunks, as follows:

  • the header
  • options and libraries (optional)
  • inputs: searching directories and data sources
  • inputs: content embedded directly inside the WebMake file
  • metadata (optional)
  • catalog generation (optional)
  • outputs
  • the footer

The header: Every WebMake file must start with a <webmake> tag.

Options and libraries: Quite often, you may want to use some of the optional plug-ins provided with WebMake, or occasionally, you might need to set options to control WebMake's behaviour. The top of the WebMake file is a good place to do this.

Inputs: searching directories and data sources: The important bit! WebMake allows you to load content text, HTML templates, or URLs of media files (such as images), from directories in the filesystem.

Inputs embedded in the WebMake file: Another key area. Content text, HTML templates and tables of small items of content or metadata can be embedded directly into the WebMake file, for ease of editing.

Metadata: If you want your site to contain pages which list details about, or links to, other pages, generated on-the-fly, metadata is the way to do it. WebMake supports several ways of tagging your content with metadata to provide this. Metadata can be embedded into the content text, or tagged onto the content after its already been declared.

Catalog generation: once you've tagged your content text with metadata, WebMake can generate catalogs -- indexes, sitemaps, and the like -- from this. Built-in catalog types include a site map, back and forward navigation links, and "breadcrumb trails". You can also write your own Perl code to generate custom indexes using the library functions, if you prefer.

Outputs: Finally, all that data needs to be written somewhere. The out tag takes care of this. Each out block is roughly equivalent to a target in traditional UNIX make(1) terminology; the text inside the tag is expanded (by expanding ${content references}) and written to the named file. Since quite a lot of output is typically almost identical in terms of the templates it uses and they way it converts the output filename to the name of the content text to insert, the for tag is useful here to automate the process.

The footer: Finally, the WebMake file ends with a </webmake> tag.

Which Outputs Are Created?

Normally, all outputs named in the WebMake file are scanned, and possibly re-generated. However, if a target has been specified on the command line, only that file will be "made".

Dependencies And Other Optimisations

"Making" the target is not the end of it -- strictly speaking, the target may or may not be updated. WebMake tracks the dependencies of each file, and if these have not changed, the file will not be rebuilt.

That's the first optimisation. However it doesn't always work; if some of the file's text is generated by, or depends on text that contains dynamic Perl code, WebMake will always have to rebuild the file, as it cannot determine exactly what the Perl code is going to do!

To avoid continually "churning" the file, regenerating it every time WebMake is run, a comparison step takes place. Before the file is written to disk, WebMake compares the file in memory with the file on disk; if there are no changes, the on-disk file will not be modified in any way. This means tools like rsync(1), rdist(1) or even make(1) itself will work fine with a WebMake site.

All of these optimisations can be overridden by using the -F (freshen) command-line switch; this will force output whether or not the files have changed.

Ensuring A Seamless Transition

A very large (or very complicated) WebMake site can take a while to update. To avoid broken links while updating the site, WebMake generates all output into temporary files called filename.new; once all the output has been generated, these are renamed into place. This minimises the time during which there may be inconsistencies in the site.

Caching

Since WebMake uses dependencies to avoid rebuilding the entire site every time, it needs to cache metadata and dependency information somewhere.

Currently this data is stored in a file called filename/cache.db, where filename is a sanitised version of the WebMake file's name, in the .webmake subdirectory of your home directory.

How to Migrate to WebMake

Chances are, you already have a HTML site you wish to migrate to WebMake. This document introduces WebMake's way of doing things, and how to go about a typical migration.

Place The WebMake File

First, pick a top-level directory for the site; that's where you'll place your .wmk file. All the generated files should be beneath this directory. In this example I'll call it index.wmk.

Make Templates

Next, identify the page templates used in the site. To keep it simple, let's imagine you have only one look and feel on the pages, with the usual stuff in it; high-level HTML document tags, such as <html>, <head>, <title>, <body>, that kind of stuff. There may also be some formatting, such as a <table> with a side column containing links, etc., or a top-of-page title. All of these are good candidates for moving into a template. I typically call these templates something obvious like page_template or sitename_template, where sitename is the name of the site.

For this example, let's imagine you have the HTML high-level tags and a page title as your typical template items.

So edit the index.wmk file, and add a template content item, by cutting and pasting it from one of your pages. Instead of cutting and pasting the real title, use a metadata reference: $[this.title]. Also, replace the text of the page with ${page_text}; the plan is that, before this content item will be referenced, this content item will have been set to the text you wish to use. 

     <webmake>
     <content name=page_template>
       <html><head><title>$[this.title]</title></head>
       <body bgcolor=#ffffff><h1>$[this.title]</h1>
       <hr />
         ${page_text}
       <hr />
       </body></html>
     </content>

Grab The Pages' Text

Next, run through the pages you wish to WebMake-ify, and either:

  1. move them into a "raw" subdirectory, from where WebMake can read them with a <contents> tag, or;
  2. include them into the index.wmk file directly.

It's a matter of taste; I initially preferred to do 1, but nowadays 2 seems more convenient for editing, as it provides a very easy way to break up long pages, and it makes search-and-replace easy. Anyway, it's up to you. I'll illustrate using 2 in this example.

Give each content item a name. I generally use the name of the HTML file, but with a .txt extension instead of .html. This lets me mentally differentiate the input from the output, but still lets me quickly see the relationship between input file and output file.

Strip the template elements (head tag, surrounding eye-candy tables, etc.) from each page, leaving just the main text body behind. Keep the titles around for later, though. 

     <content name="document1.txt">
       ....your html here...
     </content>
     <content name="document2.txt">
       ....your html here...
     </content>
     <content name="document3.txt">
       ....your html here...
     </content>

Convert To EtText (OPTIONAL!)

Now, one of the best bits of WebMake (in my opinion) is EtText, the built-in simple text markup language; to use this, run the command-line tool ethtml2text on each of your HTML files to convert them to EtText, then include that text, instead of the HTML, as the content items. Don't forget to add format="text/et" to the content tag's attributes, though: 

     <content name="document1.txt" format="text/et">
       ....your ettext here...
     </content>
     ...

To keep things simple, I'll assume you haven't used EtText in the examples from now on.

Add Titles

Next, you need to set the titles in the content items, so that they can be used in higher-level templates, such as the page_template content item we defined earlier.

To really get some power from WebMake, use metadata to do this.

What is Metadata?

A metadatum is like a normal content item, except it is exposed to other pages in the index.wmk file. Normally, you cannot reliably read a dynamic content item that was set from another page; if one content item sets a variable like this: 

     <{set foo="Value!"}>

Any content items evaluated after that variable is set can access ${foo}, as long as they occur on the same output page. However if they occur on another output page, they may not be able to access ${foo}.

To get around this, WebMake includes the <wmmeta> tag, which allows you to attach data to a content item. This data will then be accessible, both to other pages in the site (as
$[contentname.metaname], and to other content items within the same page (as $[this.metaname]).

Think of them as like size, modification time, owner etc. on files. A good concept is that it's data used to generate catalogs or lists.

Anyway, titles of pages are a perfect fit for metadata. So convert your page titles into <wmmeta> tags like so: 

     <content name="document1.txt">
       <wmmeta name="title">Your Title Here</wmmeta>
       ....your ettext here...
     </content>
     ...

(BTW it's not required that metadata be stored in the content text; it can also be loaded en masse from another location, such as the WebMake file, or another file altogether, using the <metatable> directive. Again, it's a matter of taste.)

Sometimes, for example if you plan to generate index pages or a sitemap, you may wish to add a one-line summary of the content item as a metadatum called abstract. I'll leave it out of the examples, just to keep them simple.

Metadata may seem like a lot of bother, but it's a perfect fit when you need to generate pages that list links to, or details about, the pages in your site.

It should always be referred to in $[square brackets]. I'll explain why later on.

Naming The Output URLs

Finally, you've assembled all the content items; now to tell WebMake where they should go. This is accomplished using the <out> tag.

Each output URL, in this example, requires the following content items:

  • ${page_template}, which refers to:
    • $[this.title]
    • ${page_text}

As you can see, both this.title and page_text rely on which output URL is being written, otherwise you'll wind up with lots of finished pages containing the same text. ;)

There are several ways to deal with this.

  1. Set a variable in the <out> text, using <{set}>, to the name of the content item that should be used for the page_text.
  2. Derive the correct value for page_text using the name of the <out> section itself.

The simplest way is the latter. WebMake defines a built-in "magic" variable, ${WebMake.OutName}, which contains the name of the output URL. (Note that output URLs have both a name and a filename; you'll see why in the next section.)

To do this, define another content item: 

     <content name=out_helper>
        <{set page_text="${${WebMake.OutName}.txt}" }>
        ${page_template}
     </content>

What Does That Do?

Line 2, in the example above, needs an explanation.

This takes the name of the output URL (as discussed above), using a content reference: ${WebMake.OutName}. For example, let's say the
page was named pageurl. 

     <{set page_text="${${WebMake.OutName}.txt}" }>

${WebMake.OutName} expands to pageurl: 

     <{set page_text="${pageurl.txt}" }>

It then appends .txt to the end: 

     <{set page_text="${pageurl.txt}" }>

and expands that as a content reference. 

     <{set page_text="...entire text of page..." }>

Finally, it stores that in a content item called page_text.

This looks pretty complicated -- and it is. But the important thing is that, as in traditional UNIX style, it's also a very powerful way to do templating and variable interpolation; once you get the hang of it, there's plenty more stuff it can do.

BTW: you could simply skip defining this "helper" content item altogether,
and just go to the top of the file and change the template to refer directly to ${${WebMake.OutName}.txt} instead of ${page_text} . That's what I usually do.

What's With the Square Brackets?

But what about the title? Handily, since we defined the titles as metadata, and referred to them as $[this.title] in page_template, this is taken care of; once the ${page_text} reference is expanded, $[this.title] will be set.

Remember I mentioned that metadata should always be referred to in $[square brackets]? Here's why. Square bracket references, or deferred references, are evaluated only after normal, "squiggly bracket" content references.

The example page contains the following content references:

  • ${page_template}, which refers to:
    • $[this.title]
    • ${page_text}

Since ${page_text} is a normal content reference, it will be expanded first; and when it's expanded, the <wmmeta> tag setting title will be encountered. This will cause this.title to be set.

Once all the normal content references are expanded, WebMake runs through the deferred references, causing $[this.title] to be expanded.

If page_template had used a normal content reference to refer to ${this.title}, WebMake would have tried to expand it before ${page_text}, since it appeared in the file earlier.

Anyway, I digress.

Writing The <out> Tags

Each output URL needs an <out> tag, with a name and a file. The name provides a symbolic name which one can use to refer to the URL; the file names the file that the output should be written to.

Typically the name should be similar to the page's main content item's name, to keep things simple and allow the shortcut detailed in the previous section to work.

Also, sites typically use a pretty similar filename to the name, for obvious reasons. At least, they do, to start with; further down the line, you may need to move one (or more) pages around in the URL or directory hierarchy; since you've been referring to them by name, instead of by URL or by filename, this means changing only one attribute in the <out> tag, instead of trying to do a global search and replace throughout hundreds of HTML files.

Anyway, here's a sample <out> tag: 

     <out name="document1" file="document1.html"> ${out_helper} </out>

But what about multiple outputs? Two choices:

  1. Simply list all the output HTML files, one after the other. Works fine for small sites, and it's simple.
  2. Use a <for> tag.

I don't think you need to see how 1. works, it's pretty obvious. Let's see how 2. does it: 

     <for name="page" values="document1 document2 document3">
       <out name="${page}" file="${page}.html"> ${out_helper} </out>
     </for>

The important thing here, is that any references to ${page} inside the <for> block, will be replaced with the name of the current item in the values list.

Putting <out> Names To Work

So you've named the output URLs. However all your content items contain static URLs in the HREFs! Let's fix that.

This really is up to you; it's a global search-and-replace. Let's say you want to fix all links to "document1.html". Replace this: 

     <a href="document1.html">foo</a>

with an URL reference, like this: 

     <a href="$(document1)">foo</a>

Now, even if "document1.html" is renamed to "blah/whatever/doc1.cgi", you won't have to do a search-and-replace again.

Getting Advanced - Adding Navigation and a Sitemap

This hasn't been written yet. Sorry!

Invoking Webmake

WebMake can be run using the command-line tool webmake, using webmake.cgi, or by using the Perl module HTML::WebMake::Main.

In addition the EtText format can be used using its command line tools, or by using the Perl modules directly.

The command-line tools' POD documentation:

And the POD documentation for the Perl module:

WebMake as a CMS

WebMake is, arguably, a Content Management System, or CMS.

To be more specific, it's oriented entirely towards generating a relatively static site, such as a weblog, a news site (without comments or personalisation) or a typical informational site.

It does not have any dynamic, database-driven, features suitable for "live" sites that update frequently with dynamic data; nor does it have support for "personalisation" features, where the site displays different data based on what the user presents in their HTTP request. (Of course, using WebMake does not preclude using PHP, mod_perl, Mason etc. to provide these, however.)

Here's the relevant details of what it can do.

WebMake's CMS Features

  • Separation between content and layout

    Since, logically, content and layout are entirely separate tasks, they should be easy to keep separate in the CMS.

    WebMake uses content references to include content into pages, and implement templating. This allows you to separate the content text from the template layout HTML; the template designers just need to include a content reference, such as ${body}, instead of the text.

  • No requirement for text editors to know HTML

    Only the layout staff should really need to know HTML, so the staff who provide text content can do this without HTML knowledge.

    WebMake provides Text::EtText, which provides an easy-to-edit, easy-to-read and intuitive way to write HTML, based on the plain-text markup conventions we've been using for years.

  • Generation of pages automatically, using metadata from content items

    It should be possible to generate index pages, sitemaps, navigation links, and other text automatically, based on properties and metadata of the pieces of content loaded.

    WebMake supports this by allowing any content item to carry arbitrary textual metadata. Perl code can then be used to dynamically request a list of content items that have a particular set of metadata, and any page can refer to another content item's title, description, abstract etc. without itself needing to parse the content text.

  • Flexible URL support

    It should be trivial to rearrange a site, if required, totally changing the URLs used in the site's pages.

    WebMake supports this by using symbolic URL references, which can be modified by changing one line, causing references to that URL throughout the site to change.

  • Edit-In-Page Functionality

    Most CMSes boast a nice, browser-based user interface to creating, naming, uploading and filling out content items and media.

    WebMake now provides a CGI script, which allows a certain degree of web-based maintainance and content editing. It's not quite as foolproof as some of the bigger CMS systems, but it's a start!

What WebMake Is Missing

  • Database Support

    It would be nice if WebMake could load content from a database. It currently cannot, although there's nothing in the architecture that would preclude this; there just has not been a need, just yet.

    Unfortunately, this may not be possible -- this IBM software patent details a mechanism whereby a server can dynamically rebuild its pages, based on changes to objects in a database. WebMake could run afoul of this if database support is added (although there are a few points where this could be avoided).

  • XSLT Support

    This will definitely arrive -- as soon as a good XSLT engine becomes part of Perl, or at least becomes easy to install from CPAN. It's on my list ;)

  • Workflow

    There's currently no logic to support workflow. This would not be difficult to add, though.

Tips On Using WebMake

Editor/IDE Support

The root directory of the WebMake distribution includes a Vim rc file to support syntax-highlighting for WebMake. To use it, make a directory called .vim in your home directory, copy it there, and add the following lines to your .vimrc: 

au BufNewFile,BufReadPost *.wmk so $HOME/.vim/webmake.vim
map ,wm :w!<CR>:! /usr/local/bin/webmake -R %<CR>

Change /usr/local/bin/webmake to whatever the real path to the webmake command is.

Once you do this, the macro sequence ,wm will cause a rebuild of the site which contains the file you're currently editing. In addition, opening a file called something.wmk will automatically use WebMake syntax highlighting (if you have syntax highlighting enabled in VIM).

The Button

WebMake now includes a WebMake button:

Feel free to include it on your pages; but please, if possible, add it with a href to http://webmake.taint.org/, so people who are curious can find out more about WebMake.

It's 88 pixels wide and 31 high, by the way. If you look in the "images" directory of the distribution, there's also an 130x45 one and a 173x60 one.

To make things really easy, here's some cut-and-paste HTML for the image: 

 <a href="http://webmake.taint.org/"><img
 src="http://webmake.taint.org/BuiltWithWebMake.png"
 width="88" height="31" border="0" /></a>

Contributors to WebMake

Here's a list of people who've contributed to WebMake:

  • Justin Mason <jm /at/ jmason.org>: original author and maintainer
  • Mark McLoughlin <mark /at/ skynet.ie>: added perlout directive, fixes to HTML cleaner

  • Caolan McNamara <caolan /at/ csn.ul.ie>: EtText contributions; lists, pre-formatted text, lots of suggestions; he's written a nice testimonial here.

  • Jan Hudec <bulb /at/ ucw.cz>: navtree plugin, patches to remove metadata from site mapping and control mapping of media items
  • Matthew Clarke <clamat /at/ van.maves.ca>: doco fix for datasource documentation
  • rudif /at/ bluemail.ch: lots of help with supporting Windows

Thanks all! Patches and suggestions are welcomed -- send them in! (By the way, patch contributors get listed at the top, 'cos patches save me writing the code ;)

Contents for the 'Tags and Their Attributes' section

The <webmake> Tag

The <webmake> section is required in a WebMake file. Any text before or after this section will be ignored.

In the current implementation, you can leave these tags out, but it isn't advised; their requirement may be enforced later.

Example

  <webmake>
    [...WebMake file omitted...]
  </webmake>

The <include> Tag

Arbitrary files can be included into the current WebMake file using this tag. It has one attribute, file, which names the file to include.

A set of libraries are available to include, distributed with WebMake. See the Included Library Code section of the index page for their documentation. However, these should be loaded using the <use> tag instead of this one.

Example

  <include file="inc/footer.wmk" />

The <use> Tag

WebMake supports "plugin" libraries, which are generally other .wmk files or Perl modules which can be loaded to extend WebMake's functionality.

For example, there are standard plugins to provide support for "download" links, which allows links to files including their size, ownership information, etc.; there's also a plugin which allows HTML tables to be defined using a comma-separated value list.

It has one attribute, plugin, which names the plugin to load.

Plugins can be loaded from the WebMake perl library directory, or from the user's home directory. The search path for a plugin is as follows:

  • ~/.webmake/plugins/plugin.wmk
  • ${WebMake.PerlLib}/plugin.wmk

The set of standard plugins are listed in the Included Library Code section of the index page.

Example

  <use plugin="safe_tag" />

The <content> Tag

The <content> tag, along with the other similar content-defining tags like <contents>, <template> etc., is used as one of the basic building blocks of a WebMake file.

Essentially, you use it to wrap input, and give them a name, so that you can refer to them later in <out> blocks or other content items.

This tag has one required attribute: its name, which is used to substitute in that section's text, by inserting it in other sections or out tags in a curly-bracket reference, like so:

${foo}

If you wish to define a number of content sections at once, they can be searched for and loaded en masse using the <contents> tag.

Every content item can have metadata associated with it. See the metadata documentation for details.

The following attributes are supported. These can also be set using the <attrdefault> tag.

format
This allows the user to define what format the content is in. This allows markup languages other than HTML to be used; webmake will convert to HTML format, or other output formats, as required using the HTML::WebMake::FormatConvert module. The default value is "text/html".
asis
This will block any interpretation of content or URL references in the content item, until after it has been converted into HTML format. This is useful for POD documentation, which may be embedded inside a file containing other text; without "asis", the text would be scanned for content references before the POD converter stripped out the extraneous bits. The default value is "false".
map
Whether the content item should be mapped in a site map, or not. The default value is "true".
up
The name of the content item which is this content item's parent, in the site map.
preproc
Pre-process content items using a Perl function.
isroot
Whether or not this content item is the root of the site map. The default value is "false". (This cannot be used as a parameter to a tag that loads multiple content items, like the <contents> tag.)
src
Allows the text of the content item to be loaded from a given URL (remote content) or file in the filesystem. (Again, this is not usable from a tag that loads multiple items.)
updatefreq
How long a remote content item should be cached. (Again, this is not usable from a tag that loads multiple items.)

Using Remote Content

Content items can be loaded remotely, ie. via HTTP or FTP, by using a URL in the src attribute. These will be cached for as long as the update frequency updatefreq dictates, by default 1 hour. The update frequency is a string in this format:

[n days] [n hours] [n mins] [n secs]

So, for, example, 1 hour 20 seconds converts to 3620 seconds.

Pre-Processing

Using the preproc attribute, you can specify a block of perl code to execute over each content item's text. The content item's text is provided in the $_ variable. (Since the XML attribute format doesn't provide much room for perl code, your best bet is to call a function to do the work.)

This can be very handy. Here's some suggested uses:

  • multiple templates can be loaded from one HTML file; for example, if your designer has created a template for a "list page", with HTML for the page layout, a table, odd list lines, and even list lines, you can use just one template file as a src, and define multiple content items from it using different preproc functions and the scrape_xml() Perl code library function. The Scraped Templates page goes into more detail on how to use this.
  • If you combine this with an agreed format for "filler" text or variable references, then you can replace filler with valid content references on-the-fly, and avoid having to persuade the designer to understand how content refs work. For example, your designer could use the lorem ipsum text to indicate "main body text"; using a sub like this 

		s/lorem\s+ipsum[^<]+/\${main_body}/gs;

you can convert that text into a reference to a content item called main_body.
  • you can convert raw formats to more friendly-looking presentation on the fly; for example, my blog at taint.org (view source) is updated through email, and those mails are stored as raw mails to the filesystem. WebMake converts them to HTML using EtText and a short preproc function which strips out email addresses for spam protection. (See example below)
  • sections of text can be loaded from third-party websites or files, regardless of the markup surrounding it. By using a perl sub like 

		s/^.*?<!-- start of text table -->//gs;
		s/<!-- end of text table -->.*?$//gs;

you can strip off the unwanted parts of the file; in other words, HTML screen scraping. Again, the scrape_xml() Perl code library function is handy here.

Defining Content Items On-The-Fly

The <{set}> processing instruction can be used to define small pieces of content on the fly, from within other content or <out> sections.

In addition, Perl code can create content items using the set_content() function.

Using Content From Perl Code

Perl code can obtain the text of content items using the get_content() function, and can treat content items as whitespace-separated lists using get_list().

In addition, each content item has a range of properties and associated metadata; the get_content_object() method allows Perl code to retrieve an object of type HTML::WebMake::Content representing the content item.

Example

  <content name="foo" format="text/html">
  <em>This is a test.</em>
  </content>

  <content name="bar" format="text/et">
  Still Testing
  -------------

  So is this!
  </content>

  <content name="remote" format="image/png"
	  src="http://webmake.taint.org/BuiltWithWebMakeBigger.png">
  </content>

  <{perl sub mail_fmt {
      local ($_) = shift;
      s/\S+\@\S+/\(spam-protected\)/gs;		# remove email addrs
      $_;
    }
  ''; }>
  <contents src="raw" format="text/et"
	   name=".../*.mail" preproc="mail_fmt($_)" />

The <template> Tag

The <template> tag is identical in most respects to the <content> tag.

Typically, one will want to differentiate textual content, such as news articles, from template content, such as page templates. This tag allows those semantic differences to be expressed at a high level; use <content> blocks for textual content, and <template> blocks for template content.

Note that <template> blocks are never mapped in site maps, and cannot hold metadata.

It is implemented as a content item with the map attribute set to false.

Example

  <template name="page_header" format="text/html">
  <html><head><title> $[this.title] </title></head>
  </template>

The <contenttable> Tag

Quite often, it's handy to define small (one-line) content items quickly, in bulk, directly inside the WMK file itself. The <contenttable> tag provides a good way to do this.

Firstly, pick a delimiter character, such as |. Set the delimiter attribute to this character.

Next, list a table of content names and their values, separated by a delimiter character, one name-value-pair per line.

Note: if you would prefer to load the content items from a separate file, the <contents> tag is better suited.

Another note: this is not the way to define data about other content items (in other words, metadata), such as titles, authorship, or brief descriptions, as WebMake's built-in metadata support will not be available in that case. Embedding the metadata into the content item using <wmmeta> tags, or loading them in bulk using <metatable> tags, should be used instead.

Example

  <contenttable delimiter="|">
  person1|Justin
  person2|Catherine
  person3|The cat
  </contenttable>

The <contents> Tag

Content can be searched for using the <contents> tag, which allows you to search a data source (directory, delimiter-separated-values file, database etc.) for a pattern.

Apart from the fact that it loads many contents instead of one, it's otherwise identical to the content tag; see that tag's documentation for details on what attributes are supported.