How to Create Your Own RSS Mashup With Yahoo Pipes

How to Create Your Own RSS Mashup With Yahoo Pipes:
The internet is a great resource for news and updates, and no matter what you’re looking to keep track of, you’re sure to be able to find countless sites that will be able to keep you up to date with the latest information. To help make it easier to keep track of new developments, you might make use of an RSS feed to save you having to look things up manually. You might already be used to using RSS in apps like Google Reader, but there’s so much more you can do with RSS feeds.
Pipes is a tool from Yahoo that enables you to take things a step further so you can, amongst other things, create your own custom RSS feeds that pull in content from a variety of sources and filter it so that you only see the most relevant news stories. It’s a venerable web app, starting off life in a rather Google-ish way of being in a lengthy period of beta but then living on for years, long enough that many of us have likely forgotten about it. But it’s still a great tool, even in 2012, so let’s dig in and see what you can do with it.

Getting Started

Ready to get started? Make sure you’ve got a Yahoo! account (something you likely already have if you’ve ever used Flickr). Then, fire up your preferred web browser and pay a visit to the Yahoo Pipes. Pipes is a tool that lets you take RSS feeds and mix them together, like pipes mixing two fluids together. Just sign in, and you’ll be ready to get started.
You’ll be presented with a blank workspace and this is where you will be creating your pipes in a visual, drag and drop based environment. To the left you’ll see a list of Sources and these are what you will use to pull in data from other web sites. The pipe we’re going to create is going to filter news from existing RSS feeds, so start by dragging a Fetch Feed module from the left on the workspace.

Yahoo Pipes provides you with a blank canvas on which to work on your creation

In the text field, enter the URL of an RSS feed you would like to work with and then repeat the process of adding a Fetch Feed module and a feed for as many feeds as you need. You can check that the feeds are working correctly by dragging the debugger pane up from the bottom of the screen; click on of the Fetch Feed modules and the output of the feed will be displayed here.

Use the debugger to check that your feeds are working as expected

Filtering Feeds

Now we’re going to add a filter to each feed to control which news stories are displayed. Click the Operators link to the left to expand the group and then drag and drop three Filter modules to the workspace. You will now need to join each of the Fetch Feed modules to a Filter operator – just click on the white dot at the bottom of a Fetch Feed module and drag to a white dot at the top of a Filter box to establish a link.

Modules need to be linked together in order for filters and other operators to take effect

Once links have been set up you can use the drop down menu in each of the Filter boxes to choose to block or permit different content and you can then specify keywords that will be looked for in titles, authors and other parts of feed items. As well as permitting and block content based on individual keywords, you can also configure multiple rules that must be matched before content of displayed or blocked.

Filter modules can be used to tightly control which news items appear in your pipe feeds

Unifying Filtered Feeds

Additional operations can be applied to feeds, but for the purposes of this guide, we’ll start to tie thing up. Back in the Operators section to the left, drag a Union module onto the workspace. You can then join each of the Filter modules to the nodes on the top of the Union module, before joining this to the Pipe Output module at the bottom of the workspace.
You should use the debugger to check that your filters are working correctly and if you’re happy with the output, click the untitled tab at the top of page and enter a suitable new name before clicking OK.

A Union module can be used to bring all of your filtered content together

Click the Save button to the upper right of the page and then click the Properties button. In the pop up window that appears you can enter a description for your pipe as well as a number of keywords to enable other people to search for and make use of it.

A tags and a description so that other people can track down the pipes you make.

To check how your pipe looks, click the Run Pipe link – you can also visit the My Pipes section of the web site and then click the pipe you are interested in. If you find that you need to make any changes or additions, just click the Edit Source link, but otherwise you can make use of the pipe as a feed by clicking the Get as RSS link

The My Pipes section of Yahoo Pipes provides access to your creations

Browsing Premade Pipes

Creating your own pipes can be fun and rewarding but, depending on what you are looking for them to do, it can also be a complex and frustrating experience. Before you spend an age creating a complicated pipe, it is worth taking look through what other users have put together – even if you do not find something that precisely meets your needs, you may well stumble across something that could be adapted, or at least gives you an idea of how to achieve what you’re looking to create.
Once you’re logged into your account you can either use the search box to the upper right of the page to look for something matching certain criteria, or you can click the Browse button in the upper navigation bar to take a look through other people’s creations.

There are a wealth of readymade pipes that you can use as-is or adapt to suit your needs

And that’s all, folks!

Yahoo Pipes is an extremely versatile service that can be twisted and tweaked to work in a huge variety of ways. If you are a programmer, there is great scope for getting your hands dirty with interactive pipes that enable you to get more form the web, but even the most simple creations are very useful. But even if you’ve never coded in your life, Yahoo Pipes makes it easy to get the data you want from the web, mixed up and sorted just the way you want it.
Have you ever used Yahoo Pipes, or is this your first time to try it? It’d be fun to hear some of the ways our readers are using Yahoo Pipes to make feeds that work best for them.