Real-Time News Curation - The Complete Guide Part 2: Aggregation Is Not Curation
We are no longer just consumers of content; we have become curators of it, too. In Part 1 of this Guide, I have introduced why we really need real-time news curation and what is the basic idea behind it (Part 1 — Real-Time News Curation, Newsmastering, And Newsradars — The Complete Guide Part 1: Why We Need It).
In Part 2, I want to continue illustrating what “real-time news curation” is all about, and more specifically, why it differs from automatic aggregation and why you really need a human being to do it. As I see it: “Aggregation is automated, curation is manual.”
Back in 2004, I wrote an article entitled: The Birth of The Newsmaster.
It was my first public realization that a real-time news curator, which I labeled at the time a “newsmaster,” was soon due.
The explosion of RSS-based information channels and my early experiments with it led me to think that whoever, like Robert Scoble, was claiming to be able to follow hundreds of thousands of news feed sources would have had to soon question his assessments.
You cannot follow and keep yourself updated in an effective way by simply subscribing to as many news sources as possible.
De facto RSS did (and does) put a large quantity of content in a sort of tacit public domain as it offers the easiest way to reuse, syndicate, and aggregate formatting-free content in an unlimited number of ways.
It is in the DNA of RSS to be wanting to be free, to be further reused, personalized, and syndicated.
And it was this very technical innovation, the democratization of most web content into a standardized format that anyone could subscribe and work from (RSS), that made me realize how big an opportunity there was for those with sufficient expertise, passion, analysis, and research skills to become unique aggregators and intelligent news curators on specific topics of the best content available out there.
The ability to aggregate, filter, and curate content coming from the most disparate sources appeared to be the key operation skill required to make this idea become a reality.
I then imagined how valuable could have been the creation of specialized, focused information channels, intercepting the needs and interests of very specific niches:
· Valuable to those who want to learn and stay updated instead of being inundated with tons of content garbage and unverified news to sift through,
· Valuable to those who curate and publish such channels as these provide them with the perfect means to build their own visibility, credibility, trust, and authority,
· Valuable to publishers and brands supporting such news publishing approach as it brings real value to their customers and helps everyone make better sense of all the news available out there.
The problem of information overload is like any other problem, one side of a new, bright opportunity. As the quantity of news and information grows by the hour, we need new means and approaches to organize and make sense of all of it.
Automated aggregation is not the solution. Human-powered, manual news curation is.
Human news curators can add more value and understanding to the news by aggregating, filtering, and curating them than what is available in individual news stories taken by themselves.
By organizing, filtering, selecting, and picking the best and most relevant information for a target community of fans/readers/customers, an online publisher can provide his audience with high-quality, unique value that will be hard to find elsewhere.
So, as I wrote six years ago, it did, and it still does look to me as if a renaissance of sorts is truly taking place.
Understanding what this renaissance is all about and how you or your organization can best leverage it is what this guide on real-time news curation is all about.
Here, in Part 2 of this guide, find out what are the key differences between aggregation and curation and why real-time news curation is so much better than automated aggregation when it comes to providing high-quality news and information on a specific topic.
“Anytime you hear someone talk about Google News, The Huffington Post, Gawker, blogging, aggregating, curation, and indexing as if they are the same phenomenon, ignore them.
And if they attach that discussion to a set of policy recommendations, without acknowledging the full complexity of what it is people actually do when they aggregate, curate, and index information - well, then you should put your fingers in your ears and run in the other direction.”
“Aggregators, curators, and indexers: There’s a difference, and it matters” C.W. Anderson - Nielsen Journalism Lab
Part 2 - Aggregation Is Not Curation - The Human Factor
1. Aggregation is not curation
2. Filtering is not curation
3. Aggregation is Automated, Curation is Manual
4. Automated Aggregation without Curation is mostly…
5. The Solution is in the MIX — Human Curation + Machine Aggregation
6. News Curation: What A Difference A Human Can Make
“We are no longer just consumers of content, we have become curators of it too.
If someone approached me even five years ago and explained that one day in the near future I would be filtering, collecting and sharing content for thousands of perfect strangers to read — and doing it for free — I would have responded with a pretty perplexed look.
Yet today I can’t imagine living in a world where I don’t filter, collect and share.
More important, I couldn’t conceive of a world of news and information without the aid of others helping me find the relevant links.”
- Aggregation Is Not Curation
- Automatic Filtering is not Curation
- Aggregation is Automated, Curation is Manual
- The Solution Is In The MIX — Human Curation + Machine Aggregation
- Automated Aggregation Without Curation, Is Mostly…
- News Curation: What a Difference a Human Can Make
- My Own Curated Mini-Channel on Real-Time News Curation
1) Aggregation Is Not Curation
Aggregating content and news from multiple sources and feeds is one basic form of curation. The selection of news sources is, per se, an editorial choice, an act of curation, and in some cases, it can provide good enough extra value to create value and even build services around it.
This is how news aggregation services like Google News, Techmeme, Memeorandum, and many others (which I will showcase in the next Part of this guide) have been born.
Sources are selected once and updated/vetted on a periodic basis by a human curator or a newsmastering team, while an array of algorithm-based software programs search, aggregate, and filter the incoming news stories while ranking them.
The ranking has been generally based on popularity (the number of sources picking up the specific news story), authority/trustworthiness, and freshness indicators.
Techmeme, one of the most popular technology news aggregators on the web, was originally born simply as an automatic aggregation and filtering web service that required no human intervention.
Techmeme's original editorial focus has been a very wide range of technology and media-related news aggregated and filtered via the news service's own software algorithms.
Over time these have appeared to be based more on popularity (which news has more of a news chamber effect) than on originality, relevance (too broad an audience), or other inherent information quality.
To do those things, you just need a live human being to do that.
But in 2008, Gabe Rivera, Techmeme founder, and CEO, officially hired a human curator, adding a subjective expert eye to the already excellent work done by its software. He later extended the human-curation support also to his other niche news hubs.
Duration: 5':32" - Gabe Rivera of Techmeme on Why a Human Editor is Better - Video from LeWeb ’08 - Paris
“Curation is the sibling of aggregation, a word that the web has know for a while. Aggregation means gathering; finding all videos with the keywords “Easter Supper” in them.
But as more devices like cell phones are used to create content (video of a hotel room, a tweet from a rock concert, an audio post from a political protest) gathering no longer adds value.
In fact, aggregation can equal aggravation.
Strangely enough, curation shifts the balance of power back to brands and publications. While anyone can make content, the decision to gather it, and present it by trusted content curators has more risk, and therefore more value.”
Can Curation Save Media
Steve Rosenbuam — Business Insider
2) Automatic Filtering is not Curation
Once you start aggregating content, it does not take much time to realize that unless you apply even some kind of basic filtering to it, you are going to drown inside it.
Duplicates, re-takes of the same news, artificially created content, spam, unverified stuff…
But thanks in particular to the power of social technologies, there exist indeed many different approaches to the automatic filtering of news content:
· Algorithmic Filtering - exclude-include specific keywords, terms, and phrases
· Filtering Based on Social Graph - what your friends like
· Crowdsourced Filtering - what most people like, prefer, vote
· Shared Sources Filtering (Meta) - what people share online
· Influence Filtering - what is shared by those who influence you the most
· Social Search - what your connections have preferred
· Location Filtering - where you are
See: Different Approaches To News Filtering
All these approaches to filtering can be automated. As they are designed to work on sets of predetermined rules, they can help indeed filter out a lot of irrelevant stuff automatically.
But when it comes to organizing, sorting, making sense, attributing, or checking what is inside those stories and which ones do deserve attention, it is another story.
Some of the newest real-time curation and newsmastering engines have started sporting some “intelligent” internal auto-filtering capabilities, for which I can only, at least for now (just like Kevin Marks of BT in the video above), be very skeptical.
Most of these auto-filtering capabilities are based on some of the above filtering approaches, but unless a news curator could fully edit, tweak, and switch off any of these filters, it may be hard for him to be able to fully realize his mission of explorer-reporter, as some risks must be taken in order to explore new ground.
“It is also true, however, that left to its own devices, social media can lack an important, cohesive and meta narrative that creates clarity and helps the world make sense of events.
While algorithms and basic filters can help, social media needs the ethics, rigor and insight of professional journalists to help tell big stories.”
Chris Saad — Real-Time Storytelling
The greatest value in this editorial and curation process can be provided by human selection capabilities.
Therefore if you bypass them in favor of an algorithmic solution, you may find out that, at least in some cases, algorithms will make choices and exclusions that the news curator would not have made.
This is why the more you leave the filtering process in the hands of the curator (within reasonable limits - in the sense that some basic automatic filtering is definitely a good thing to have), the more quality you will get in the final result.
The real-time news curator should have in his power to decide completely on which filters to use and, in my opinion, should not rely in general on automatic filtering solutions (unless he has been the one to set and define them explicitly).
Since it is difficult to define many of the variables that differentiate a good story from a bad one, as this may greatly vary depending on the niche, audience, and personality doing the curation, it is, in my opinion, not a good idea to leave all of this power to an algorithm, or to substitute the news curator unique editorial skills with a set of software algorithms that can be bought or rented by anyone.
No matter how good your filters are, my experience has taught me that you can make a great selection only if you put a competent, skilled “curator” at it.
The result is simply light-years better.
Noise May Actually Help Curators Do Better Filtering - The RSS Dolby
As I look back at my original vision for a curated news universe, I recall making an analogy that, while hard to grasp at the beginning, makes a lot of sense to me.
The idea is that the more “noise” there is in the news stream, the easier it is to distinguish what is good and relevant from what is not.
Just like for audio and the noise filtering technology known as Dolby, it may actually be that to make the filtering process more accurate and precise, the level of noise needs to be increased rather than decreased.
The Dolby sound system achieves its goal through pre-emphasis, generating a cleaner, hiss-free sound by first adding more noise to the original source and then filtering out with greater ease what appears to be the superfluous parts.
In the same way, human curators “leverage” news noise to help themselves identify and distinguish true valuable stories from generic duplicates or retweets.
You can hear the very same concept explained by Robert Scoble (at min: 19' 30") in this excellent video interview done with Howard Rheingold on the very topic of online content curation:
Duration: 26':49" - Robert Scoble was interviewed by Howard Rheingold on Online Content Curation.
In other words: News curators are the “Dolby” of online news.
3) Aggregation is Automated, Curation is Manual
If someone tells you: “Hey Paulo, why don’t you simply pull together a bunch of good feeds from authoritative feeds in your sector, we mix them up with Yahoo! Pipes or some of the cool new tools out there, and you then slap it on your web site pages.
Imagine how cool to easily and rapidly re-purpose relevant content coming form other sites into your web site, email newsletter or AdSense pages.
You get fresh content, which the search engines love, you provide fresh news for your readers and you become a leader, a trusted authority in your sector without having to add an extra task in your busy workflow or having to hire another person.”
A bell should ring in your head.
After the bell has rung, a sign should appear saying:
“You always get what you pay for.
There are no cheap miracles around here.
Get back to serious work, dude.”
“…In 2005 it was clear to me that an ideal news aggregation site would need to combine automation with direct, hands-on editing.
…Any competent developer who tries to automate the selection of news headlines will inevitably discover that this approach always comes up a bit short.
Automation does indeed bring a lot to the table — humans can’t possibly discover and organize news as fast as computers can. But too often the lack of real intelligence leads to really unintelligent results.”
Guess what? Automated news doesn’t quite work.
Gabe Rivera — Techmeme
“Even giant, technically-accomplished corporations have had trouble breaking news using algorithms.”
Breaking: Google News doesn’t break tech news
Eric Eldon — VentureBeat
“We have many automated means to aggregate news…
Curation is a step above that, human selection.
It’s a way to add value.”
Death of the Curator, Long Live the Curator
Jeff Jarvis — Buzz Machine
“The word curation may seem to be a synonym for aggregation, but in fact it’s a double for “intelligent aggregation”.
Museum curators do not, I hope, assemble as much art as possible for an exhibition; rather they apply judgment in selecting what they deem to be appropriate.”
Steve Rosenbaum
Content Curation vs. Content Aggregation — Abnormal Returns
Let me start analyzing at the basic level.
Is aggregating the stream of news from your most trusted sources equivalent to doing news curation?
Many social media enthusiasts have gone beyond simple serendipitous sharing and into “curating” or “giving coverage” to a specific, focused area of interest by republishing the aggregated streams of news of their most trusted contacts on specific topics.
Is this real-time news curation?
No.
Why?
Quantity doesn’t make up for relevance.
By automatically aggregating and mixing together a group of selected feeds, you cannot really avoid excluding all of that superficial noise made up by personal messages, serendipitous commentary, unverified stories and links, spam, or just generic junk content.
If you are an active social media user of platforms like YouTube, Twitter, and Facebook, I am sure you have already experienced much of this in your daily news consumption activities.
If not, listen here to a few “in the know” who already have. At the last European Internet conference LeWeb in 2009, Cathy Brooks video interviewed a few expert geeks, CEOs, and media journalists, asking them how large a percentage of their daily social media news consumption was truly relevant. Here is what she found out:
Ouriel Ohayon of Appsfire
Kevin Marks of BT
Tony Hsieh of Zappos
Louis Gray
4) The Solution Is In The MIX - Human Curation + Machine Aggregation
“Interacting directly with an automated news engine makes it clear that the human+algorithm combo can curate news far more effectively that the individual human or algorithmic parts.”
A curator is not an expert.
The expert seeks to provide the definitive answer, while the curator seeks to provide the greatest range of high-value info-facts and perspectives to allow you more leeway in constructing your own picture.
The expert is a judge… working toward restricting the view.
The curator is an explorer-reporter, working toward expanding the view.
His job is all about sense-making, organizing, illustrating, and offering the best resources and viewpoints available out there.
The real-time news curator helps to see the big picture.
“We’ve combined sophisticated automated aggregation technologies with direct editorial input from knowledgeable human editors to present the one indispensable narrative of an industry in transition.
We collect relevant takes on an issue and package them together in a comprehensive group of links.
That way, you not only get the lead opinion on an issue, but you can easily find the supporting, opposing, smart, controversial, notable, and previously unseen viewpoints.
You get the big picture.”
Introducing MediaGazer - Megan McCarthy
“…I have long believed in there being a perfect place for humans to act as curators and guides, while there is another perfect place for machines to provide, to the best of their ability, resources to aid your discovery.
So when you are challenged with a mountain of information coming at you from any angle, think of the best way to get it handled. Should you turn to an editor, to the will of the people, to your friends, or to code?
The options are all there, and more tools are coming to help you attack the noise — because there’s little chance it will fade away any time soon — and a very strong chance it could get much worse very quickly.”
“Humans do things software cannot, like grouping subtly related stories, taking into account sarcasm or skepticism, or posting important stories that just broke.”
Techmeme Offers Tech News at Internet Speed - NY Times
The real-time news curator doesn’t just select the good, relevant news from the rest.
The newsmaster orders, edits, re-titles, adds or re-writes introductions and synopsis, adds commentary, tags, adds images or reference links, credits, and re-syndicates through his own networks the most relevant news stories on the topic-theme he covers.
That’s not a small difference.
It’s the same difference between listening to a bunch of songs on the radio and having a personal DJ who spins only the type of music you REALLY like.
The human curator beats automated mechanisms because he can always rely on a broad set of extra resources and information channels, normally not available to a software algorithm.
A human curator can tap into contacts and friends, including topic experts that he personally knows, can call in or email business companies to verify news, can follow up on leads received via an instant message, or utilize valuable information from a print magazine article.
Those information items would never make it into an automated newsradar as I call it, or real-time news curation, newsmastering syndication tool of any kind.
This is why there is no escape from it: you have to…
“…combine sophisticated automated aggregation technologies with direct editorial curation from trusted and authoritative human editors…”
“Philosophically, I believe human editing plus automation have always been and will always be needed for top-notch aggregation. Pure automation sufficed for a few years for us, and got us to the point where we could hire more editors.
But until editors arrived, Techmeme would often make questionable choices - like spotlighting too many redundant stories, keeping obsolete stories on the page, and overemphasizing odd topics only introspective bloggers care about. Also, obviously significant stories would often take much too long to appear.
In the age of Twitter and hyper competitive news bloggers, even a 15-minute delay on big news is inexcusable. Our editing helps on all these fronts: we can block the automation from posting stuff to the site, and instantly post stuff if need be.”
Richard MacManus interviews Gabe Rivera
Techmeme Turns 5: Interview With Founder Gabe Rivera - ReadWriteWeb
5) Automated Aggregation Without Curation, Is Mostly…
“Hey Robin, but what about all these sites that integrate a fully automated news aggregated feed into their pages just for the sake of SEO and more frequent indexing?
It looks as an easy way to produce new fresh content, isn’t that cool?”
The result of uncurated content, aggregated and filtered for the only purpose of creating new fresh content for a site, blog, or brand or to increase one’s own SEO relevance, can, in most cases, only be classified as spam.
That is why early news curators and aggressive marketers who have jumped on this train with the idea of having found a chicken by the golden eggs have remained somewhat disappointed with the results.
Why?
Once again, the answer remains the same: While it is easy to systematically aggregate a lot of content on just about any topic, the value of such automatically aggregated content is generally disappointing.
There is not enough value in the basic aggregation and filtering.
To create a truly valuable news channel, you can rely on all the search, aggregation, and filtering technologies of the world, but at the end of the line, you need to put an expert, passionate curator to pick the best stories in some meaningful way.
And so if you see or hear of a company doing news curation, and all they do is assemble together a bunch of feeds and re-publish them with a cool design, you know there is an abundant margin of improvement available to them.
Automatically aggregating and republishing relevant and value-rich content feeds is not per se a despicable activity, but the type of results it produces are generally useful only in a very limited number of situations.
Say you want to bring together all of your different content-sharing activities on different sites or social networks, then it makes a lot of sense to simply aggregate and republish a bunch of RSS feeds together.
On the other hand, it is also true that on very specific topics like auctions or job offers with specific characteristics, adopting an automatic approach can, in many cases, work wonders.
However, this applies only to those situations where all information items have the same basic information structure and value types.
6) News Curation: What a Difference a Human Can Make
If, at this point, you are still wondering what is the unique value that a human, a news curator, can add to a news stream that has already been created by aggregating and filtering the best content and sources available out there, here is an initial list of what I think are the additional benefits that such a person can bring to the emerging new digital news publishing workflow:
1. Crap detection - Verifies whether a news story is trusted and reliable.
2. Topic competence - Is an expert on the specific topic
3. Synthesis - Can effectively synthesize, summarize, title
4. Pattern detection - Sees story patterns, developments, and related items
5. Commentary - Adds commentary, opinion, and notes
6. Visual enrichment - Illustrates, visualizes, and supports visual data
7. Referencing - Adds links, references, resources
8. Framing - Frames and contextualizes stories according to the specific interest/audience being served
9. Organization - Juxtaposes, sorts, groups, and finds relationships among different stories
10. Classification - Adds editorially relevant metadata, such as tags
11. Trust building - Builds trust with the audience by using a personal voice, and style and by being direct
12. Credit attribution - Gives appropriate credit where due
13. Topic expansion - Expands topic by uncovering related stories and information
14. Time extension - Identifies and links relevant past stories
15. Sense-making - Helps readers see the big picture through diversity in viewpoints and sources
There are probably many more, as these are the ones I have identified so far.
As you can see, there are a lot of things that a human curator can do to significantly extend the value of the basic news aggregation and filtering work done by any software.
This does not mean that real-time news curators do not need specialty, dedicated tools to be able to do this. They do, and badly so.
This is one aspect I will dwell on in detail as I will not only provide you with a full panorama of all the tools you may want to use for real-time news curation, but I will also outline what are the specific features and characteristics that the ideal real-time news curation-publishing tool should have.
My Own Curated Mini-Channel on Real-Time News Curation
End of Part 2 - Automatic Aggregation
Coming up next in this Complete Guide to Real-Time News Content Curation:
In Part 3 - Types of Curation and Real-World Examples
In Part 4 - How To Do It
In Part 5 - The Curator Attributes And Skills
In Part 6 - The Tools Universe
In Part 7 - Business Applications and Trends
Previous parts already published:
Part 1 - Real-Time News Curation, Newsmastering, and Newsradars - Why We Need It
Originally written and “curated” by Robin Good and first published on September 15th, 2010, on MasterNewMedia as “Real-Time News Curation - The Complete Guide Part 2: Aggregation Is Not Curation.”
Robin Good
