I've used Yahoo Pipes for some prototyping, but I'm unwilling reluctant to use it in full production as we'll have no control over it uptime etc.
Is there an (ideally open source) alternative that I can run on my own server? Or any application that can convert a pipe into PHP or Ruby?
What about this? http://pipes.deri.org/
"DERI Pipes: Open Source, Extendable, Embeddable Web Data Mashups
Inspired by Yahoo's Pipes, DERI Pipes is an engine and graphical environment for general Web Data transformations and Mashup. Supports RDF, XML, Microformats, JSON and binary streams. Use it as a "Web Pipe" or embedded in your applications Works as a mashup command Line tool Supports SPARQL, XQUERY, Several scripting languages. Extend it as needed DERI Pipes, in general, produce as an output streams of data (e.g. XML, RDF,JSON) that can be used by applications. However, when invoked by a normal browser, they provide a end user GUI for the user to enter parameter values and browse the results (see the examples below) ."
In our company, we've been using Yahoo Pipes for quite a long time and we're still using it for some specific and more complex feeds.
Otherwise, we are developing a simple Sinatra-based application to work like Yahoo Pipes and return the result as a standardized XML/JSON response. It's quite simple to create such application, however I agree with Pascal. Between Yahoo's servers and yours, relying on Yahoo! should be more effective than relying on a custom server.
The only Pipes downside we've been experiencing is the API throttle. We've partially solved the issue caching the responses on our side and implementing a custom throttling mechanism to dynamically slow down requests when Pipes returns an HTTP error.
Regarding uptime, you could cache the output of Yahoo Pipes on your server in case the feed goes AWOL for a short time (though as PascalMartin said, their uptime is very likely better than yours).
It's unlikely that Yahoo would suddenly close Yahoo Pipes but if they did I'm sure clones would start springing up. Worst case scenario it wouldn't be too difficult to write a custom program/script to fetch a bunch of feeds and filter them on your criteria.
