FutureTap - Smart iPhone Apps

The FutureBlog

April 30, 2009 by Ortwin Gentz

Scraping App Store Featured Entries

iTunes Ranking.pngMany iPhone developers are monitoring their app’s ranking and reviews. There are some tools to automatically retrieve the ranking (MajicRank) and the reviews (Erica Sadun’s perl script) for all country-specific App Stores using web scraping.

What is missing so far is a solution to retrieve information about feature entries on the App Store start page such as “New and noteworthy”, “What’s hot” and “Staff favorites”. Mentions on the start page have a considerable influence on sales. To avoid monitoring the different country-specific stores manually in iTunes, I wrote a Perl script that can be scheduled to run daily. The script is launched with the app’s iTunes ID (the numeric ID at the end of the iTunes URL you can copy in iTunes using right click on your app title) as a parameter and puts out the current date, country and the type of the mention:

./iTunesFeatured.pl 284940039 Navigation
29.04.2009	AT	Home page, NEW AND NOTEWORTHY
29.04.2009	DE	Home page, NEW AND NOTEWORTHY
29.04.2009	GB	Home page, STAFF FAVORITES

The script takes a while to complete because it needs to fetch 8 9 pages per country. The output is in tab-delimited format to facilitate post processing of the data. In our case, we’re importing the data into a FileMaker database that allows an easy graphic representation (colors to indicate trends) and the creation of all kinds of reports.

So, let’s get to the meat: The script is available on http://github.com/futuretap/iTunesFeaturedCheck.

Do you find this useful? I’d love to know your feedback in the comments or via Twitter!

Update 23.11.2009: Modified to work with iTunes 9 App Store
Update 30.12.2009: Modified Japanese match term for New and Noteworthy
Update 29.07.2010: Implemented check for category level featuring, posted on GitHub
 

2 Comments

  1. Tweetback!

    @kkonotchick via Twitter

    wrote on April 30, 2009 at 2:11am

    RT @futuretap: How to scrape #appstore feature entries: http://tr.im/k3Tv #iphonedev

  2. Tweetback!

    @alexmarktl via Twitter

    wrote on May 6, 2009 at 3:41pm

    @futuretap thanks that worked! Really nice and helpful script for tracking your iPhone apps. http://tr.im/k3Tv

Comments are closed.