Today I'm happy to announce that SweetTweets (both the stand-alone cfc, and the Mango plugin) have been updated and are available for download.
Huge, huge, huge hat tip to Ray Camden, who brought this proejct to my attention. After deconstructing how it worked, I was able to rip out the old search method that SweetTweets did and replace it with the same technology. If you're curious about the tech, I'll get into it a little bit after the download links.
For now, the short version of the story is that link reverse lookups are finally possible -- which means that ANY url shortening service* can be used, and you'll still get your tweetbacks... Yes, even bit.ly!
SweetTweetsCFC users can download version 2.0 from the Riaforge project page. Mango Blog plugin users can get the latest version here:
- Plugin:
- SweetTweets
- Version:
- 2.0
- Requires:
- Mango Blog 1.2+
- Auto-install URL:
- http://fusiongrokker.com/get/SweetTweets
How does it work?
There's a service available called Topsy that has their own API, including reverse-lookup for links -- essentially exactly what I was looking for in my last post about Tweetbacks. And since we don't have to lookup and cache what the short url's are, you should notice that SweetTweets is much faster than previous versions. The only thing I rewrote was the search tech. Local caching and AJAX access are more or less untouched.
If you were using custom code to implement SweetTweetsCFC, you'll notice that the getTweetbacksHTML and init methods have two new optional arguments allowing you to specify the header text for when there are or are-not tweetbacks found. I'm pretty sure that's about the only interface change. The data returned should be roughly the same. Oh, and I updated the links that it adds to hashtags to use search.twitter.com instead of hashtags.org, which is the new standard; and I also fixed a bug where only the first hashtag in the tweet would be converted to a link.
Using Topsy does have its own limitations. They limit results to 50 tweets per page, and I don't see the need to implement paging, so at the moment the most you can get back is the 50 most recent tweets.
*I'm not sure if shorteners that add a "bar" (frame?) to the page will work. This includes services like ow.ly (the shortener used in HootSuite).
in
Mango |
My projects |
2 Responses
Posted 2010-02-11 11:20
A little bit less than a year ago I created an open source CFC and companion Mango Blog plugin called SweetTweets that would search for shortened links to your blog entries on Twitter and display any tweets found in a manner similar to TrackBacks. (Note that this wasn't my idea, Dan Zarella invented and pioneered it, I just made a ColdFusion version.) At the time, I was really excited about the project and it seemed to be fairly popular; it even got included in the BlogCFC core, which made my week.
Unfortunately, I'm at a bit of a crossroads. I personally view TweetBacks as nearly dead, if not completely dead. Functionally, they still work the same as they did when I originally released SweetTweets in January of 2009. What's changed is market share. In May, Twitter switched their integrated URL shortening service from TinyURL to Bitly, and within days Bitly's market share overtook TinyURL. You can get the latest statistics at Tweetmeme but the rankings from the last 24 hours have Bitly in 1st place with over 57% of shortened urls, and TinyURL in a distant 2nd place with a mere 6.85%.
Why Bit.ly is bad for TweetBacks
The biggest problem is that Bitly doesn't repeat a short URL. If I shorten the URL http://www.google.com with Bitly, I get http://bit.ly/14d7yE. I bet that if you do the same thing, you won't get the same short URL. This behavior is bad for tweetbacks because we rely on being able to predict the shortened url and search for it. If everyone gets a different short url, then we can't predict what it will be, and thus we can't search for it. Your tweetback is out there, we just can't find it.
This isn't to say that Bitly is evil or anything. I'm sure they have their reasons -- chief among them is probably being able to track click statistics, and if everyone gets the same short url then there's no easy way to tell whose post of that url was the source of the click. That alone could give Twitter something to monetize, so while I'm not particularly fond of the move from TinyURL to Bitly, I understand it.
Where do we go from here?
One option is to just kill SweetTweets. That probably won't happen. Instead, I will probably release an update that pares down the list of supported url shorteners to just TinyURL and Is.Gd. That means that a majority of potential tweetbacks will still be un-searchable, and therefore NOT found and NOT listed on your blog. Unfortunately, we can't just support every URL shortener under the sun (that keeps 1 unique short url per long url), because the twitter search API has a length limit; so we can only support the top N. Right now that's TinyURL and maybe 1 or 2 others.
I wish it didn't have to come to this, but it has. I thought about asking if anyone is aware of a way to reverse-lookup shortened bitly urls, but even if we could, in a practical sense we would run out of room in just a few links. I'm a little sad, but I guess this is the best we can do with what we've got.
in
Mango |
My projects |
Twitter |
4 Responses
Posted 2010-01-04 10:57
Over the last several days I've been working hard on not only changing SweetTweets to be ColdFusion 6 & 7 compatible, but also adding some serious improvements.
Here's a summary of what's changed:
- Now working on ColdFusion 6 and 7 -- Should also be Railo & OpenBD compatible, would love for someone to test it.
- Bitly and Snurl url shortening services removed
- If there are more tweets than the limit allows, a link is added to view all of them on Twitter Search. (Suggested by Rob Cawte)
- Caching has been improved. Page load times will be much faster after tweetback data has been cached.
- Caching is now optionally done in one of two places: Application scope or locally inside the CFC. (Suggested by Seb Duggan)
- The getTweetbacksHTML function is a very simple web-service. It's always had access="remote", but now I include a couple of example files showing how to load tweetbacks via AJAX with both ColdFusion 8's CFDiv tag as well as jQuery.
Download
Of course, with a new version of SweetTweetsCFC comes a new version of my SweetTweets Mango plugin that makes it even easier for Mango users to add tweetbacks to their blogs, which means SweetTweets (the Mango Plugin) is now CF 6, 7, etc compatible and all of the other great stuff from this post.
Download the latest version of SweetTweetsCFC from RIAForge
Download the latest version of SweetTweets Mango Plugin from RIAForge
So, down to the nitty gritty. This post is going to be long enough as it is so I won't include code samples here. Not too much has changed, but check the example files for implementation guidelines.
ColdFusion 6, 7, OpenBD, and Railo Support -- Please Test!
Unfortunately, I don't have all of these platforms at my fingertips to test with. I have tested against CF 7 and 8, but not CF 6, Railo, or OpenBD. If anyone out there wouldn't mind testing in one of these environments and leaving a comment to let me know how well it worked, that would be swell. The example files are ready to run; you just put them somewhere web-accessible and hit the URL of the template. I can't make it any easier than that.
I'll even buy you a beer at the next conference we both attend if you post a "trip report" for me. :)
Bye Bye Birdie Bitly (and Snurl)
I've come to find out that Bit.ly and Snurl.com (among other services) do not make any apparent attempt to shorten the same url into the same short url every time. In fact, some of them go out of their way to give unique short url's. (I'm still unclear what good that does them, but that's neither here nor there...) Because of that, they are more or less worthless to us, so we're not going to waste page load time, our bandwidth and cpu cycles and theirs accessing their API to get and search for short URLs. The end result would be that you would find your own tweets, and if it's important to you to see your tweets as tweetbacks, it's within your power to use one of the supported services. As of right now, supported services are TinyURL.com, Is.gd, Hex.io, and Cli.gs; with more to be added later. Feel free to suggest additional services.
Tweetback Limit Changes
I did account for limiting from the beginning, but I've improved it a bit in this version. The limit is still optional, and still defaults to 100. To specify unlimited, set a limit of 0. If the number of tweetbacks for the given post exceeds your limit, a link will be included to the Twitter Search results. One caveat here is that twitter search is case-INsensitive, so you'll get some false positives that the plugin is recognizing and stripping out. Does that bother people? If so, would you rather it had a facility to show all tweetbacks locally instead?
Caching Overhaul
First and foremost, I fixed a bug in cache expiration. Then I totally rewrote the cache system. The whole thing is much easier to use now and makes the rest of the code more readable, but more importantly, it's faster.
Depending on your implementation, you may want to store the cache inside the CFC instance itself (for example, in a Mango plugin, storing the cache in the CFC variables scope will enable you to clear its cache by de-activating and re-activating the plugin...). Note that if you choose to store the cache inside the CFC, you are responsible for persisting the CFC instance -- otherwise the cache is worthless (worse yet, we're wasting time creating it, so it has a negative net impact!).
On the other hand, if you're lazy or otherwise inclined to, you can have the cache stored in the application scope. It will use Application.SweetTweetCache, which you can delete any time you like in order to clear the cache. As I mentioned previously, this is the behavior (currently there is no other option) when you access SweetTweets as a web service; since there is no other (practical) way to persist the data.
A Web Service, if you can call it that...
It's the crumbiest web service I've ever seen or written, but it still sort of almost counts. The getTweetbacksHTML method allows remote access, so you can hit it with AJAX to load your tweetbacks onto the page without affecting initial page loading time. The way I look at it is that by the time someone finishes reading your blog post, the tweetbacks will have loaded and they won't notice a difference -- except that the server didn't need to churn longer before rendering the page for them. Seems like a good idea, right?
The zip now includes two additional examples that demonstrate loading tweetbacks via AJAX. One uses CF8's CFDiv tag, and the other uses jQuery; and just like the standard example, both are ready to use out of the box: just put them somewhere web-accessible (even on your localhost) and navigate to the page.
The reason I hesitate to call it a web service is that it still just returns HTML (and I don't want to insult the authors of real web services by putting this in the same category as their work). It's nothing overly fancy. One thing to note is that in the case of the jQuery example, if you're running on CF 6 or 7, (and you look at the request in firebug) you'll notice that the result is returned as a WDDX packet and I'm stripping it out of the packet and un-escaping some characters. I see this as one potential point of failure for Railo and OpenBD (and possibly CF 6) if the WDDX packet format is different. So please do let me know if there are any issues there, so that I can whip up a fix for you. :)
If there is enough interest in the idea, I might write a proper web service into it, and I'm thinking that might be neat because I could include pagination that way.
Fin
I think that's about enough information for one post, don't you?
in
ColdFusion |
Mango |
My projects |
4 Responses
Posted 2009-01-31 12:01
What a whirlwind evening. This afternoon, Rey Bango tweeted about something called "tweetbacks," and had previously written about how you can add them to your blog.
Tweetbacks are like "trackbacks" (which, if you're unfamiliar with those, is when you display a list of links -- usually other blog posts -- that link to the current post), except instead of being blog posts, they are tweets. See it in action just above the comment form on this very post!
I added them here on my blog; and while I was impressed with the idea and simplicity of adding it (a single line of JavaScript), it was un-acceptably slow. For starters, it was a dynamic JS file hosted on someone else's server (ugh), but it also took several seconds -- sometimes as many as 10 -- to do its thinking and let the page finish loading. Unacceptable!
So I set about writing my own version in CFML to run locally. What I came out with, several frantic hours later, is SweetTweets. (I still haven't eaten dinner. This is more fun.)
Well, really, I came out with SweetTweets -- a Mango plugin -- and SweetTweetsCFC, a drop-in CFML replacement for the JavaScript version from Dan Zarella.
It works by looking up the shortened URL for your blog entries from several of the shortening services (is.gd, tinyurl.com, hex.io, bit.ly, and snurl.com) and then using the Twitter Search API to find references to any of those URLs. The shortened versions of the URLs are cached semi-permanently (in Application scope), and the results from twitter searches are cached for 5 minutes, so that you don't overload them if you get lots of traffic or constant refreshing.
Bit.ly and Snurl.com require API keys to do url shortening, so you may want to register with those two sites (both optional) -- I'll cover this in a little bit more depth in the instructions below.
I have to give a pair of hat tips to Nathan Mische and Andy Matthews, for their respective projects JSONUtil and shrinkURL, which saved me a lot of time in putting this together.
Due to some heavy use of implicit structure notation, both projects currently require ColdFusion 8. Not to fret, I've already set to work in converting them to work in CF7!
Implementing SweetTweets Mango Plugin
Download it from RIAForge
Unfortunately, there are no built-in events that I wanted to piggy-back on. I was almost satisfied with the placement when using the end-of-post-body event, but that would trigger it on the index and archives pages as well, several times a page, which I didn't want; and the placement wasn't exactly where I wanted it. So in the end, I decided to use a custom event name.
What this means for you is that installation is a 4-step process:
- Unzip plugin into place (/components/plugins/user/SweetTweets)
- Activate it in the admin console
- Configure its settings (either go to settings -> SweetTweets, or click the "configure it now" link after activating)
- This is where you'll enter your Bit.ly and Snurl.com API credentials, if you want to use those services.
- The other setting here is the tweet limit, which defaults to 100. This is the maximum number of tweets to display for a given post.
- Add a line of code (below) to your skin wherever you want the tweetbacks to be displayed. (Preferrably on the single-post/single-page template, but hey, that's your call.)
I'm happy to help you over IM, Twitter, or Email if you have any trouble with this.
Implementing SweetTweetsCFC on your Blog/etc
Download it from RIAForge
As I said above, you can implement SweetTweetsCFC in a single line of CFML. It does take a little chaining, and it's a little bit of a long line, but it's pretty simple, and technically it's a single line:
#createObject('component','path.to.SweetTweets').init('bitly_email','bitly_api_key','snurl_username','snurl_api_key').getTweetbacksHTML('current_url',limit)#
Note that all 4 arguments of the init function are optional, and if either of the first two is left blank Bit.ly will not be used, and if either of the last two are left blank... you guessed it... Snurl.com will not be used. If you're not using any of them, you can just do init(), you don't have to do init('','','','').
Lastly, you need to pass in the actual current URL of the blog post / page, and you can optionally set the tweet limit as the 2nd argument to the getTweetbacksHTML function. I hope you're capable of figuring out what your current URL is and plugging that in, but again, if you get stuck, I'm happy to help you over IM, Twitter, or Email.
in
Mango |
My projects |
13 Responses
Posted 2009-01-22 10:27
