About the author


The hostess of this here party.

Related Articles


  1. 1

    AJ Robins

    This is really, really cool!

    Unfortunately, I don’t have the time to install and play with it, but I wish I did! :-)

  2. 2


    I’m digging all of the features that you’re building into this, I’d love to try it out once it gets a little further along (with an easy way to update).
    Keep up the good work Gina, this is coming along really nicely.

  3. 3

    Thejesh GN

    Awesome. I am going to install it today.

  4. 4

    Laurent Denoue

    I like your code! I am also curating twitter data, but to detect stuff happening in cities.
    I detect city names, dates and event keywords and aggregate tweets. It’s kind of cool to see where people are going (festivals, conferences, tweetups, etc.) in a given city:

  5. 5

    Steven Rossi

    Looks really cool – I like that you’ve created lots of unique ways to display information. I’m looking forward to trying it out.

  6. 6

    Sandaruwan Gunathilake

    I tried this on my local web server. (Manually entered the oauth_token in the GET url).

    There was a problem in the crawler logger. The realpath(‘logs/’) evaluated to null it tried to save logs in my root directory. Why did you commented “global config” parameters? I took the log path from it.

  7. 7


    This is great stuff! I got it up and running in less than 15 minutes with a few minor changes.

    By putting the code up on GitHub, are you releasing the code under some sort of opensource license such as the GPL, MIT, MSD, etc.?

    Also, since you’re getting back into coding PHP, I’d highly recommend taking a look at CodeIgniter. I found CI to be a great framework for implementing MVC in PHP.

  8. 8

    Sandeep Bali

    I have a doubt.. by default.. in twitalytic/common/config.inc.php the path for logs goes to /advanced/logs/ while there is no such directory.. am I supposed to replace it to crawler/log/ along with rest of the path..??

  9. 9

    Ken Yarmosh

    I own the domain (www.twittalytic.com) and Twitter handle (tiwttalytic) and am planning to release something relatively soon. But I wasn’t sold on “Twittalytic” because it’s a little too geeky.

    In any case, I’m approaching some of these problems slightly differently. Good to see some validation here though. Congrats. :-)

  10. 10

    Gina Trapani

    @Sandeep and @Sandaruwan: the sample config file line to the log should be:

    $TWITALYTIC_CFG[‘log_location’] = ‘/your-path-totwitalytic/crawler/logs/’;

    Sorry about that.

  11. 11


    I like this a lot. I would definitely use it, if for no other reason than I’m intrigued by the stats (’cause I’m just a regular gal with a small number of followers).

    @Ken it’s *supposed* to be geeky ๐Ÿ˜›

  12. 12

    Waldron Faulkner

    Hey Gina, long time, no talk!

    Love what you’re doing. I’ve also been working on a Twitter project, which I’m calling GraphEdge (http://graphedge.com). I’m looking at some of the same metrics, but focused more on follows and drops. More a B2B play, but certainly valuable to obsessed Twitter consumers. Will be adding something close to “least likely” and “most likely” soon, plus a lot more.

    Drop me a note and we’ll talk.

    – Waldron

    waldron [at] graphedge.com

  13. 13

    Sandaruwan Gunathilake

    It seems that the current code doesn’t use the path mentioned in $TWITALYTIC_CFG[‘log_location’]. In line 14 of “common/class.Logger.php”, you are using “realpath(‘logs/’);”.

    Small suggestion : use “dirname(__FILE__)”.

    ~ Sandaruwan

  14. 14


    Awesome! I have just installed it and it’s working great!

    One thing I did notice is that if I view a users page (like click on someone else’s username to see mutual friends ect.) it screws up the session. Then every page I go to after that gives me an error like “Error, selection query failed: SELECT * , round(total_replies_in_system/(datediff(curdate(), earliest_reply_in_system)), 2) as avg_replies_per_day FROM instances i INNER JOIN owner_instances oi ON i.id = oi.instance_id WHERE oi.owner_id = ORDER BY crawler_last_run DESC”

  15. 15


    I’ve managed to install this on dreamhost and got the front end working succesfully but every time I try to run the crawler I get the following error.

    $php crawler/crawl.php

    Parse error: syntax error, unexpected T_STRING, expecting T_OLD_FUNCTION or T_FUNCTION or T_VAR or ‘}’ in /home/.amelda/me/my.domain.cc/common/class.User.php on line 54

    Any thoughts?

  16. 16

    Gina Trapani

    @undisclosedp: Are you sure you’re using PHP5 to crawl? Check this out: Installing Twitalytic on Dreamhost

  17. 17


    @Gina Doh… its the simple things. The domain was set to php5 but calling php on the cli uses 4! I’ve moved things on to a permissions error now.

    $usr/local/php5/bin/php /home/me/my.domain.cc/crawler/crawl.php

    Warning: fopen(/crawler.log): failed to open stream: Permission denied in /home/.amelda/me/my.domain.cc/common/class.Logger.php on line 41
    can’t open file /crawler.log

    Which is peculiar. I’ve even done the old chmod 777 on the whole directory structure (bad I know) and its still erroring. Thoughts? And thanks for your help!

  18. 18


    Aha, reading the comments above looks like the logger doesnt log to the path in the config. Having edited class.Logger.php to read the correct value all is well! Brilliant. Crawling my tweets now. Thanks Gina!

  19. 19

    Michael Doornbos

    I installed this today and love it. Thanks Gina!

  20. 20


    I’m a total novice! I hope I can get this set up. Couple ideas though:
    1) When Do you Twitter Most? (Day of Week, Time of Day, etc.)
    2) What sites do you link to the most (NOT Bit.ly)

    Looks awesome!
    Will post back!

  21. 21

    Stephen Zeidner

    Thanks for this wonderful open source contribution Gina! I installed this on my Ubuntu server at home and really like the threading features and the fact that I can keep a local copy of my tweets. I can’t wait to see how it influences the movement towards a federated Twitter.

    P.S. Keep up the good work on TWiG. You, Jeff and Leo always have great insights that I enjoy thinking about throughout the week.

  22. 22



    Looks like David Pogue is looking for something like this, so I told him about your code.


    We’ll see if he’s interested.


  23. 23

    Mike Carlucci

    Just heard about this on the TWIG podcast. Sounds pretty cool, though I have no webapp background to even attempt to use it.

  24. 24


    Got it up and running on http://ival.us just a project for now.. not the main purpose for purchasing the domain… (ival.us = visual)

  25. 25


    I’m a noob, so I have to ask: What do I do when PuTTY tells me “ln: creating symbolic link ‘twitalytic/webapp’ to ‘alyxportur.com/tweets’: No such file or directory”? (pls email me via capryor@alyxportur.com so I can get this fixed fast. Tks!)

  26. 26


    Gina, great work – thanks!

    Quick suggestion: have a suffix at the beginning of your table names. For example, “owners” becomes “ta_owners”, “tweets” becomes “ta_tweets”, etc…

    My host only gives me one MySQL database that all my apps can access. Other apps have this suffix, such as “mt_*” for Movable Type tables, “g2_*” for Gallery2 tables, etc… so it makes it much easier to work with the tables.

  27. 27



    For those of us not quite at the level of installing our own servers and then this software onto the servers, do you have any advice or recommendations? I would love to first and foremost archive and have access to my tweets. I would love the ability to run some analytics as well, but more importantly would be having access to my whole history.

    Jeff Jarvis gave one solution using Twistory and Google Calendars, but it only goes back 1 week’s time. While I haven’t been tweeting for years, it has been a few months and 1 week’s worth is much less than I’d like.

    So, any tips or suggestions?



  28. 28


    Hi Gina,

    Great idea here, I stumbled across the project via the Mine!Project blog.

    I’ve installed the app locally and got as far as trying to add my Twitter account into the system – for some reason the app isn’t generating an oAuth token, I’m getting some php Warnings. I haven’t got much more time to spend on it now, but it looks very useful, will try and see if I can figure out the token problem


Comments are closed.