Tool suites for archiving instant messaging history

Since the Library of Congress is archiving tweets, and the practice of archiving IRC chatroom logs is common, well, why not.

The storage is not a problem - any DB or even CSV file should do.

But I've not seen a good tools to make these DBs nicely searchable and accessible for non-tech people (everybody out there knows SQL? I think no).

So, what software could be used for providing useful access to such archives? Would be great if it could also provide anonymized stats for researchers (i.e. replacing nicknames with numbers etc)

wizzard0

• software
• digital-born
• accessibility
• presentation

Comments

Answer by raffaele messuti

generally irc logs are plain txt files, each row containing the timestamp, the user, and the message.

when log is not provided by the irc server the solution is to use an irc bouncer (another irc server sitting between your client and the irc server) that logs for you. znc is pretty simple to setup

since these logs are text files you can use any technology (even grep) to search it.

i should recommend omega, a small cgi based on xapian

Comments

• wizzard0: Omega is good, but the IRC is not the only IM solution. I have hundreds of megabytes of (mostly corporate) log files in different formats from everything from Skype to Jabber to YM - and look for turning them into something **unified**, usable & searchable.
• raffaele messuti: then you should write your own parser for these different logs, and use elasticsearch. with few lines of javascript you can setup a search interface. look at https://github.com/okfn/facetview/ if suit your needs