RSS Syndicationplex

Yesterday I worked on an RSS aggregator site for our group and those we intersect with. Rob and I discussed the benefits and he has seen a preliminary product. http://dhtc-group.uchicago.edu/ is now working URL for it. I've added syndication for a number of different information sets, largely as illustrations, but perhaps permanently:

  • twiki (selects updates from the "Main" site)
  • confluence (selects updates from the uc3 space)
  • github's ubolt repo (selects any commit message)
  • OSG JIRA (assignee = marco)
  • MWT2 JIRA (labels = public and labels = (uc3 or mwt2))
  • individual contributor blogs (selects all new content)

Each of the blog feeds imports tags and categories from the source. The JIRA, twiki, github, and confluence additionally attach a suitable "ubolt", "uc3", or "mwt2" tag onto the syndicated posts. JIRA feeds attach "ticket", TWiki and Confluence feeds attach "wiki". This gives us a tool for selecting content topically or by type of post. Some of these were suggested by Rob, some are my own idea. Feel free to suggest alternative or additional ways of tagging feeds. Once we're done with Indianapolis, I'll go over how this all works in more detail. Issues:

  1. Some of the RSS generators (e.g. github and jira) will tend to bleed CSS and HTML markup into their post descriptions, which can lead to cluttered and unreadable post excerpts in some cases — e.g. a confluence page update post contains some random-looking CSS text from Confluence's presentation of the same information. This is really flow in the RSS generators, not in the WordPress syndication, but it's a problem we may need to deal with in the syndication. Titles and excerpts also get a little markup (e.g. [twiki] prefix) to help distinguish whether the post is worth extra effort to read.
  2. Some RSS feeds require authentication. I've added mainly public feeds so far. It's possible to make syndicated posts private, so access restrictions are not necessarily a concern. However in at least one class (TWiki) I don't know of a way to pass authentication information when collecting the RSS at all, so there's currently no way to syndicate private "Webs" from TWiki, like the Admins web, to a private group in WordPress. This derives from the fact that TWiki only appears to do form-based login, and doesn't accept HTTP Basic or Digest. Someone who knows TWiki better might know how to address this.
  3. Regarding the "ouroborogator" logo — we probably do want to change this, or perhaps even remove it and leave only words. That's fine — I didn't want to spend a lot of time on logo customizations yesterday, but I did want a placeholder.

Because each of these syndicated feeds produces actual posts in the aggregator blog that are tagged and/or categorized, you can actually republish them as a new feed. For example:

This means that one feed or web page can be a single public URL for aggregation of many different origins — and more than that, we can add new information sources (or amend current ones) without needing to communicate those changes to users, because the aggregator will handle it transparently. I suggest making the most of this, and including these topical URLs in your presentations or posters. Later I will have a demo of repeating the UC3 feed(s) on the UC3 welcome site. I'll blog that here as well so that it gets into the update feed, and self-referentially dislays itself on the uc3 page.