UK Web Archive Technical Update - Autumn 2023
First published as [this UK Web Archive blog post]( TBA .This is a summary of what’s been going on since the 2023 Q2 ...
Andy Jackson
geek
Twitter Facebook Google+ LinkedIn Github StackoverflowFirst published as [this UK Web Archive blog post]( TBA .This is a summary of what’s been going on since the 2023 Q2 ...
First published as this UK Web Archive blog postThis is a summary of what’s been going on since the 2023 Q1 report.Th...
Carefully moving files between storage systems is a critical part of digital preservation. If the number and size of ...
First published on the UKWA blog…My main goal for the conference was to support the adoption and development of share...
Recently, the Library of Congress’s excellent Web Archive Team sent out a request to gather information about how web...
First publicised in this UK Web Archive blog post.This is a summary of what’s been going on since the 2022 Q4 report....
As the ZIP scanning issue keeps getting updated, I realised I made some errors in my analysis of how DROID works.I’ve...
Following on from the previous post, I was experimenting with Siegfried and found it to be even faster than I was exp...
In the last few days, I’ve been going through the process of updating my Nanite wrapper for DROID, which I built to m...
A recent comment from the #DHNB2023 conference caught my eye… If you just follow web archiving friendly standards, y...
Usually we don’t let search engines index web archives. When sites are still live, we don’t want the archived versio...
This is a summary of what’s been going on since the update at the start of the autumn. 2022 Domain Crawl Completion ...
I’m not really into new year resolutions, but I would like to get back into blogging. I’d like to be a better writer,...
Recently, for no particular reason I’m sure, there seems to have been a renewed interest in more distributed and comm...
This is a summary of what’s been going on since the update at the start of the summer. Website Refresh 2022 Domain ...
Following on from the last quarterly update, we’ve been able to make some good progress despite being understaffed du...
This is a summary of what’s been going on since the last update, at the start of the year. Hadoop storage and replic...
During the last quarter of 2021, the technical services that make up the web archive underwent lot of changes behind ...
AbstractUnder Legal Deposit, our crawl capacity needs grew from a few hundred time-limited snapshot crawls to the con...
I love a digital preservation mystery, and this one started with question from @joe on digipres.club:A mystery file, ...
Today is the inaugural International Digital Preservation Day, and as a small contribution to that excellent global e...
Originally published on the UK Web Archive blog on the 10th of November 2017.The heart of the idea was simple. Rather...
Before I revisit the ideas explored in the first post in the blog series I need to go back to the start of this story...
AbstractAs an increasing number of government and other publications move towards online-only publication, we are for...
This is the script for the introduction I gave as part of a ‘Digital Conversations at the BL’ panel event: Web Archiv...
AbstractThe British Library has a long tradition of preserving the heritage of the United Kingdom, and processes for ...
Originally published on the UK Web Archive blog on the 8th of June 2017.It’s been over a year since we made our histo...
Following my previous post, a tweet from Raffaele Messuti lead me to this quote: “Computers, by their nature, copy. ...
So what was going on in our little experiment in data destruction? Well, to understand what happens when we open up d...
Following my proposed experiment in data destruction, a few kind readers tried it out and let me know what happened1....
Let’s start with an experiment…MaterialsFor this experiment, you will need: An USB flash drive of little importance....
I find working in digital preservation fascinating.It’s not where I expected to end up. I started off interested in c...
I came to work on digital preservation through the PLANETS project, and later the SCAPE project (for the first year) ...
Four years ago, during the 2012 IIPC General Assembly, we came together to discuss the recent and upcoming challenges...
Originally published on the UK Web Archive blog on the 15th of February 2016.Earlier this year, as part of the Big UK...
Originally published on the UK Web Archive blog on the 20th November 2015.Over the last few years, it’s been wonderfu...
A few months ago, a colleague suggested that we should come up with ways of helping people learn about the main stage...
On the first day of the IIPC GA 2015, the morning keynote was Digital Vellum: Interacting with Digital Objects Over C...
As a computational physicist working in a library, my background and training is quite different to the curators and ...
As published on the UK Web Archive blog.I gave the following presentation at the 2015 IIPC GA. If you prefer, you can...
Following Vint Cerf’s talk at AAAS, the “Digital Dark Age” is in the news again (see DSHR’s blog for a good summary, ...
As published on the UK Web Archive blog.Over the last year, we have been a part of the Big UK Domain Data for the Art...
As published on the UK Web Archive blog.This is the rough script of the demonstration I gave at IDCC15, with a few ex...
First published on the UK Web Archive blog.Like many other institutions, we are heavily dependent on a number of open...
First published on the UK Web Archive blog.The UK Web Archive started archiving web content towards the end of 2004 (...
A new OPF blog entry: User-Driven Digital Preservation. Reproduced below…We recently posted an article on the UK Web ...
First published on the UK Web Archive blog.When we archive the web, we want to do our best to ensure that future gene...
First published on the UK Web Archive blog.Among the responses to our earlier post, How much of the UK’s HTML is vali...
First published on the UK Web Archive blog.How much of the HTML in the UK web archive is valid HTML? Despite itsappar...
A new OPF blog entry: Digital Preservation War Stories. Reproduced below…I’ve started to publish some of my notes on ...
A new OPF blog entry: The Registries We Need. Reproduced below…If you’ve not already read it, I urge you to go and re...
A new OPF blog entry: Analysing the formats in the UK Web Archive. Reproduced below…Today, the UK Web Archive is rele...
A new OPF blog entry: Biodiversity and the registry ecosystem. Reproduced below…As Paul has already noted, there are ...
A new OPF blog entry: A Format Registry for SCAPE. Reproduced below…In my previous post on formats, I ended up leanin...
A new OPF blog entry: What do we mean by format? Reproduced below…Bill’s earlier post and this one from Chris Rusbrid...
A new OPF blog entry: OPF Year 1: Visualisation of development activity. Reproduced below…This is what the first year...
A new OPF blog entry: Cargo Cult Standards. Reproduced below…I’ve been keeping an eye on the #anadp11 Twitter backcha...
A new OPF blog entry: Economical Access via Normalisation. Reproduced below…I’ve finally had time to consider David R...
New OPF Blog: Building A Collaborative Format Registry Editor. Reproduced below…After Ross’s post, I thought I’d bett...
A new OPF blog entry: Format Obsolescence and Sustainable Access. Reproduced below…As David Rosenthal pointed out, a...
A new OPF blog entry: Is obsolescence overrated?. Reproduced below…I just wanted to point out a very interesting disc...
Thanks to our web archiving team (who lead the uk web archive project), I was given a day of training on using Hadoop...
A new OPF blog entry: Breaking Down The Format Registry. Reproduced below the fold.At the hackathon it was clear that...
A new OPF blog entry: In the room. Reproduced below…One of my favourite parts of the Planets project was the service ...
As well as blogging about digital preservation here, I’ve also got a blog on the Open Planets Foundation website wher...
I don’t know if it was coincidence, or by design, but the latest issue of the International Journal of Digital Curati...
Inspired by the excellent @dhnow social news service, I’ve set up a news aggregator for digital preservation. The @di...
Eddie Izzard is currently about two-thirds of the way through his attempt to run up and down the entire UK. He’s runn...
Edinburgh. Used to live there, you know. Nice place. Been a while. Until last weekend. Visited the Blythes.Outline: d...
Clearing my back-log of random things on the internet I wanted to share…Centrifugal ForceKeep your tuna dolphin-frien...
Occasionally, in comments on environmental stories and such, you see the old “we’re so small and the Earth is so big,...
As you might have noticed, I’m in the process of re-jigging this website. The new look has been chosen largely just ...
Well, there’s too many channels and too much noise.Should I twitter, or Facebook, or FriendFeed, or just collect deli...
Adobe have been working on a cool user interface for visualising how web pages have changed over time, called Zoetrop...
Science commons, the video, explaining all about how the science commons works.
Well, we all got the American president that they deserve. It’s good to hear about all the goodwill and hope in the ...
I’ve been impressed by the growing volume of free talks, books and other high-quality informationalistic edupodloads....
One of the latest impressive innovations to come out of Google is that they have speech recognition software good eno...
</param></param></embed></object> That is all.A.
I noticed this boingboing article and despaired. This ridiculous legislation, which has already been explicitly reje...
I can now verify that asking for a “Short Cappuccino” in Starbucks does indeed work, at least here in Leeds. My orde...
One of the minor pleasures of New Zealand was the Flat White. It’s essentially a cappuccino, but with a finer, cream...
Read on for the most amusing video I’ve seen all year… </embed>Where the Hell is Matt? (2008) from Matthew H...
UPDATE: I’ve reconsided my position on this, and I think I’ve underestimated the issues involved in keeping the byt...
For those of you who have trouble imaging how long 980 double decker buses would be, try imagining 1/38440th of the d...
If you’re having trouble sleeping, you could try downloading a highly exciting readable article called Lattice-switch...
After threatening it for a number of years, my Dad has finally set up his own web site. Find out more at Ron Jackso...
The Prime Minister just appeared on television to apologise for the loss of the web log entries that should have been...
For a while now, I’ve wondered what the prime numbers look like in binary, that is, if you paint the ones as white do...
Keep Calm and Carry On Originally uploaded by I like.Now that’s good wartime advice. You can buy your own poster her...
We’re back in Renwick now, and today we will drive over to Picton, drop off the car and head back to Wellington. We’...
We’re a week into our big trip, and are about to leave the north island and head over to Picton on the ferry for thre...
Andy and I are just back from our first NZ road-trip together, during which we had lots of fun exploring the Central ...
While staying in Ngaio, we visited Otari-Wilton’s Bush… “Only 5 kilometres from downtown Wellington lies a unique tr...
The southern hemisphere is a great place to view Comet McNaught as it moves away from the Sun. It can be seen clearly...
After landing in Auckland and being compelled to declare our chocolate at Customs, we caught our Wellington flight wi...
There haven’t been many updates to the site lately, and that’s because we’ve been very busy. Now it’s time to catch ...
Eilidh had her viva on Tuesday, and it went very well. After a small number of corrections are made and various admin...
Last weekend the Edinburgh Festival Fringe kicked off, and we went to see our first show - Tim Minchin. We saw his ...
We are now the proud recipients of our very first parking ticket. Great.
I’ve been slowly working on a new look for this website and I’ve just switched it on. It’s not finished, and it almo...
In 1989, while working at CERN, Tim Berners-Lee invented the Web so that scientists all across the world could share ...
On Friday, I had my second driving driving test and I passed with a mere seven minors. It’s a huge relief to have pa...
We both sat our driving tests last week (my 1st one and Eilidh’s 4th). Unfortunately I was a little too nervous and ...
You’ve probably seen the recent advert for Sony Bravia TVs - it looks really cool (if you’ve not seen it, here’s a l...
This weblog tends to go a bit political once in a while. This time, however, I’m going for politainment and rehashin...
Just a quick blog to say thank you all for the lovely cards, phone calls, emails and texts Eilidh & I have receiv...
Spring is finally here, I’ve survived my first winter for a few years, and things are going well. Really well. Eili...
Here’s the plan. In order to distract you from the fact that there have been no upda...
Just to let you know that the website and my anjackson.net email may have been behaving strangely over the last few d...
Well, a lot has happened since I last updated the site. I’ve been home a few times, had two weddings to go to and, s...
Well, various problems for Air New Zealand have left me 4hrs late. The connecting flights have been held, but I’m n...
I’ve been a bit worried, as for the last week flights from Wellington airport have been severely delayed due to foggy...
Well, thanks for the feedback. After a number of people explained I was being insane, and probably guaranteeing that...
The time has come for the fourth annual bi-hemispherical dash, and I’m looking for some help with a particularly trou...
I guess if you were brought up here, you must get more used to it. But personally, I just can’t imagine getting used...
Crikey. How out of touch am I? On a similar note to the Hitchikers thing, it turns out that Wallace & Gromit wi...
I’ve long been a fan of the work of Douglas Adam’s work. I read The Hitchikers Guide To The Galaxy sometime in 6th f...
Well, it’s all been rather quiet on the old blog lately. I had some bad news recently that meant I really wasn’t in ...
Well, it’s been a hectic couple of weeks, but the work has come along quite well. We’ve even managed to finish the p...
Agh. My head asplode. They’ve finally done it. Apple have finally released an affordable Mac: the Mac mini. This ...
Just a quick entry to say Happy New Year to you all, and wish you all a grand 2005. I’ve not made any resolutions, at...
It’s pretty near impossible to take it all in. I’ve been following the internet news coverage of the Indian ocean ts...
Well, you might have noticed by now that I’ve been fiddling with the design of this site. The main problem was the b...
Well, I finally went over to Industrial Research Limited and gave them a mildy re-written version of the talk I gave ...
I’ve recently finished reading Bill Bryson’s A Short History of Nearly Everything, and I must say I was very impresse...
This has to be the quote of the day, sent to me by Sarah who saw it in an Edinburgh newspaper over the weekend (The S...
Here’s a slide from the talk I gave to the group last Thursday. It went well, and it was probably the most relaxed ...
Jarvis says it better than me.:It seems quite ironic that at a time when whole departments in radio and TV stations a...
Well, I thought it was about time I added some new photos to the site. It’s not like I havne been taking gazillions ...
So, firstly, I feel like I should apologise for the previous political ranting. Yes, I know it’s more complicated th...
I have enabled email submission of pages. This is just a test to see ifit works. It’s not publicly accessible, by t...
I doubt I am alone right now. Just one of millions of non-U.S. citizens pounding various election result webpages (e...
If we ignore, for the moment, the disturbing sight of democracy being vigorously throttled by the self-righteous prop...
Well, I’ve made it to Wellington, and have even turned up at work full of vim and vigour, or possibly full of somethi...
Made it to LA with a minimum of fuss. It was actually a quite pleasant flight, with some fine views of Greenland and...
Monster track has to be Wow from Final Straw by Snow Patrol. It really rather hits the mark. In fact the whole albu...
Work at EPCC is over and I’ve made it home, stopping off along the way for a family get together at Centre Parcs in S...
Well, I can’t get out of it now. I’ve just picked up my ticket to New Zealand, with a free stopover in LA to see Cat...
Well, my work visa for New Zealand has come through, and I should be able to buy my ticket at the weekend. It all lo...
I’ve not written for a while, and a few different things have got backed up, so I’ll bunch them all together into one...
Well, after a slightly wobbly week (these malaria tabs are not without their wierd side effects), I’m looking forward...
Well, work’s going fairly well, and I’m still managing to enjoy being back! It probably helps that the weather is pr...
I guess it’s fair to say that the travelling is over now. Back to my old job, in my old office, with a new mix of ol...
I’ve now had a few days running about and catching up with family, touring around from London to Penzance, Sutton-in-...
The flight was pretty long, but went well. It was great to see cloud-covered Europe again! David picked me up from ...
Well, I’ve managed to sneak onto the Singapore Airport wireless internet service for free, which is helping while awa...
I’ve had a fine couple of days in Phenom Penh, zooming around on the back of motorcycles driven almost too fast and a...
Well, after three solid days pottering about Siem Reap and Angkor, I think I’ve seen enough temples to last me a life...
After three days of fairly hectic travelling about, I’ve made to to Siem Reap in Cambodia, close to the famous temple...
Well, I had a pleasant enough time on Ko Chang. The odd beer, a quiet little full-moon party, meeting the occasional...
Well, I’ve made it to the island of Ko Chang. Spend last night drinking beer and chatting to some locals on the beac...
I’ve enjoyed relaxing in Chiang Mai, and some of the temples are really very fine indeed. Doi Suthep lived up to mos...
Well, I’ve made it to Chang Mai, and found an okay place to stay. The train ride was alright, but after having the s...
Well, feeling a little better today, and generally had a more successful Bangkok experience. Booked a train to Chang...
Note: I’ve not finished editing and proofing this story yet, but feel free to have a read anyways.I’ve done many thi...
Made it! Mildly hectic afternoon but I’m really starting to enjoy this travelling game. Ended up in a place called t...
Well, technically speaking, given that I’ve just passed through Singapore immigration (and am pottering around on som...
Queen Victoria I mean, which appears to be impossible. I dunno, you travel a quarter of the way around the world, to...
Well, crikey, my last night in Wellington. Tomorrow morning I fly to Singapore for a few days, then most of a month ...
While hanging out with David, Jenny, Andy and Michelle, a heated argument broke out. The subject was one of those cl...
Well, it’s all going a bit crazy at the moment. I’ve booked my flights home (see dates on the right-hand side under ...
Many years ago, I created an online database of links related to the RISC OS platform called Nutshells. I thought it...
Inspired by Graham’s efforts, I thought I’d tell you about some rather fine music I’ve discovered round these parts (...
Well, crikey, gave a first-year lecture today. My boss couldn’t be there, so he asked me to cover for him. It was a...
Well, crikey. Things were pottering along quite nicely and then all sorts of ups and downs appear from nowhere. I w...
Just to balance up after yesterday’s entry, I thought I’d tell you about a rather nifty new feature of Google. The s...
I rather like this image, taken from Google News. Their news pages are generated automagically, which is bound to ca...
Well, I’m back in Wellington. And after a fairly long day of travel I’m just itching to get back to work tomorrow. ...
Milford Sound, with clear blue skies. I am clearly too lucky by half. It was just so beautiful - awe inspiring. Lo...
Well, after being duly impressed by the Haast Pass, and staying in Makarora for a night (very nice - middle of nowher...
Crikey. I am blessed. Went onto the Franz Josef glacier today. The west coast gets 5-8 metres of rain a year - so ...
Well, after a rather hectic few hours, I’ve made it to Greymouth. The TranzAlpine crossing was fantastic - it really...
Well, I’ve got some accomodation booked for the first few nights. I was worried for a second as Christchurch seems t...
Crikey. I’ve just booked a Magic Bus Transalpine Experience, so it looks like next week’s trip is all sorted. Sure,...
Great quotes during this trip… “I’m hungry like the wolf.” “It was like buying pies with rainman.” “But I’ve never...
Well, I’m off to Taupo with Stewart (my flatmate) and Kim (a visitor to the lab) this weekend. Should be good. It’s...
Was looking for some cake recipes, as I am cooking cakes for friday-buns-on-thursday this week.During the process I d...
Well, finally tracked down the optimization bug. A weird Intel chip + gcc issue that can be described as an ‘extende...
Horrible day. One of the codes I’ve inherited behaves very bizarrely. Usually it works, but sometimes if it’s compi...
Came across this quote, which I rather like. ‘Women are meant to be loved, not to be understood.’ from The Sphinx W...
Well, I’ve finally confirmed my leaving date for the job here at Victoria University Wellington. I’ll be finishing w...
Well, I’ve moved the [Drupal] website into place as the actual anjackson.net website. Not nearly got all the content...
Well, I’ve more or less finished munging the old website style into a skin for [Drupal], and it’s looking pretty good...
Well, this [Drupal] thing looks like it might just do the trick. Currently trying to move my preferred web page styl...
I mean, this really is rather fine…No tax on custard.And this is pretty good too.Anj