AltSci Cell

by Javantea
June 10, 2009
Update July 23, 2009

Digg Diversity is a new project by AltSci Concepts. It uses the Digg API to calculate a more fair score for articles on Digg. Why is this algorithm necessary or preferable? Digg has an algorithm that is based entirely on profit, which is acceptable for a company like Digg. The more diggs that occur, the more profit that Digg makes, which means that they will accept, even encourage their users to game the system. The Digg front page algorithm which promotes articles to the front page with as few as 100 diggs means that a small number of people can control the front page of Digg by simply getting 100 like- minded people to digg their articles (and visa-versa). The company Digg benefits when corrupt users promote the same content repeatedly, but the overall community is diminished (especially those users who wish to see important non-repetitive content). This topic is extremely deep and deserves an essay but definitely not tonight on the night of the beta release of Digg Diversity. Many digg comments, blogs, and even a mashup that is currently offline have been written about this issue, but I hope to write the solution.

Digg Diversity is a entirely javascript mashup using the Digg API to retrieve important information about what data is found on Digg. The first set of results may be rather surprising. You will see a list of results quite similar to the front page of Digg. However, the order is by "divvs" which are a new calculated value based on timing and repetitiveness of the digger. The raw data can be found at the bottom of the page (there is a link that displays the data).

The general idea behind the algorithm is that a person's diggs are worth subtantially less to us if we have seen them digg before. Thus the algorithm simply counts the sum of 1/n where n is the number of times we've seen that person digg.

On the first page load, you will notice quite low numbers for all the stories -- some at the bottom of the page may even be at zero. The reason for this is the limitation of the API. We are only able to download a fraction of the total digg data on the first page load. Thus, this is a very brief snapshot of the total digg data. However, if you leave this tab open* and click on the More button, you will retrieve the next set of data making your information more complete.

* In the future, you will be able to close the tab and come back.

When you click on a different category, the stories from previous categories you have visited are stored in memory so you can return without waiting. This makes the UI slightly faster than Digg when a user likes to switch between a few favorite topics.

The end result of Digg Diversity is a perfect proof that a competitor to Digg with the exact same data set but focusing on an algorithm that promotes diversity of thought instead of brute force union democracy will succeed in providing more useful results for its users.

Digg Diversity is a work in progress. I plan a 1.0 release this month (June 2009) with client-side persistence, and experimental server-side persistence (for algorithm development).

Digg Diversity was designed to control the front page in a different manner than Digg to encourage diversity of thought. If you would like to share feedback about Digg Diversity, there are a dozen ways to contact the developers. Twitter, Adele on Twitter, e-mail, Digg, or commenting on this page.