I've just added basic authentication to the Kamaelia Web Client, since it seems the useful thing to do, and appear to have created a need for exogenous connectors as an emergent property, which is incredibly cool. This will potentially allow:

Pipeline(
    ConsoleReader(eol=""),
    SequentialTransformer(
           HTTPRequest,
           AddHeader(AuthorizationHeader("Username", "password"))
           AddHeader(CacheControlHeader("no-cache"))
    ),
    SimpleHTTPClient(),
    SequentialTransformer(feedparser.parse,
                          pprint.pformat),
    ConsoleEchoer(),
).run()

This would take a URL from the console, turn it into an HTTP request, add a header which happens to be an Authorization header, also add a cache control header, and then pass this onto the HTTP Client. That makes a request, and then passes it on to the next component to be parsed and then put through a pretty printer. It's still transformational, but rather than requiring you to write new components for everything you can write simple transformations. The next step is to figure out how to invert this - how to make it such that Sequential components can be the primary component framework. (this would allow Kamaelia components to be used as a library in "normal" code). (Consider for example taking an RSS feed from a site that publishes links to .torrent files, extracting the ones you like, auto dumping the results to disk for local playback when you get home... )

I've just checked in an initial pass at a SequentialTransformer into Kamaelia.Util.SequentialTransformer.SequentialTransformer. Thinking about it though, I've perhaps written too much code - the Sequential Transformer is just a specialised version of the PureTransformer...

So how did we get to that stage? Well, locally for testing, I've created an authenticated RSS feed that looks like this:

Pipeline(
    ConsoleReader(eol=""),
    PureTransformer(AuthenticatedRequestStream("Username", "password")),
    SimpleHTTPClient(),
    PureTransformer(feedparser.parse),
    PureTransformer(pprint.pformat),
    ConsoleEchoer(),
).run()

And it just struck me - this could be rewritten as:

Pipeline(
    ConsoleReader(eol=""),
    PureTransformer(AuthenticatedRequestStream("Username", "password")),
    SimpleHTTPClient(),
    SequentialTransformer(feedparser.parse,
                          pprint.pformat),
    ConsoleEchoer(),
).run()

However you would then get the ability to do this sort of sequential transform elsewhere. This combined with the ideas that Kamaelia's website engine runs on generated the example at the top. Essentially sequential components operating on a shared expected state, which they update and passback.