it's deja vu all over again

Six years ago, when the Scotland's People website was relaunched with an entirely new interface, there were serious problems when it went live. I wrote about these problems on a number of occasions. (here, here, here and here). Over the course of the next six months or so, the developers gradually fixed some of these problems (see here and here).

Obviously, when a major new website is launched, you can expect some minor problems, but it was clear that many of the problems should have been picked up during testing before the new site was launched. It was also clear that some problems could also have been detected had there been proper engagement with site users.

So it's more than a little dispiriting, if perhaps not entirely surprising, to find ourselves back in the same situation with another "upgrade" to the website. In the last few years, the National Records of Scotland have not done themselves any favours in terms of public relations, with what could charitably be described as a reluctance to engage with users, which led me to prepare an open letter - signed by over a hundred genealogists, researchers and historians - to the head of NRS, Paul Lowe, to which the response was underwhelming to say the least.

On Wednesday, the Scotland's People site was down (and the centres closed) for a new upgrade. Communication about this was typically poor. I'm aware of one person who had a booking at one of the satellite centres who, when they contacted the centre to rearrange their visit, was told that the centre knew nothing of this down time!

So it was with a degree of trepidation that I visited the revamped website on Thursday when it came back online. And the problems were not hard to find.

One common technique Scottish researchers use is to search for the death of a married woman under two surnames, her maiden name and her married name. This makes it much easier to identify the correct record before you pay for it. And this was a major problem in 2016. So I searched for a death I knew was there - Margaret Carstairs died in Crail in 1857 age 49. This was the result:​

Now I knew that Margaret's married name was Graham, so I repeated the same search changing only the surname:

Notice that the reference details are identical - so this is the same record, indexed under both Carstairs and Graham. So if I searched the records using both surnames, I should get this record again. Here's what happens though:

When they relaunched the site in 2016, this search was also broken, but in a different way - if you searched for a death with two surnames, it returned all results with either surname (a Boolean OR) rather than results with both surnames (a Boolean AND). This time round, the search fails altogether.

This is far from the only issue though. The SP home page has a big friendly box to get you started, where you can enter a name and a date range and search across all the records

Running that search shows the following results summary:

If you then click through to the any of the results for Old Parish Registers, it passes the date range as 17010101 to 17100101 (in other words 1 January 1701 to 1 January 1710). What that means is that any events recorded from 2 January 1710 to 31 December 1710 will be missed, even though they are there.

In 2016, I wrote about the strange results for Coupar Angus. This bug was fairly easy to explain - for some reason Coupar Angus had been assigned to the county of Forfar, whereas all other parishes in the same county were assigned to Angus. (Angus and Forfarshire are two different names for the same county). A minor issue, but one that is simply down to the way the data was recorded.

So I was curious to see if Coupar Angus was behaving itself. I searched for deaths in Coupar Angus between 1855 and 1900, and this is what was returned:

I'm sure we'd have heard if the good folk of Coupar Angus had discovered immortality, so this result seemed unlikely. So I removed the date range to see if that made any difference and suddenly the deaths were revealed:

But even that's not the end of the problems for Coupar Angus. I repeated the search removing the sort order and got the following results:

Last time I checked, Auchterarder and Perth were not Coupar Angus, so quite why those results appeared there is beyond me.

One issue with the original launch was the way filenames were assigned to saved images. Back then, when you saved an image, the files were simply given a datestamp as the filename - not very useful if like me you have thousands of images downloaded from Scotland's People. Five months after the relaunch, this was fixed so that images were automatically given meaningful filenames that allowed you to identify the record type, the year, registration district and so on.

Unfortunately, that problem is back - now the default filename is "ScotlandsPeople_imagenameraw", which is not very helpful at all, and would rapidly become unmanageable if you save large numbers of files.

At the bottom left of each search results page is a list of handy links to other record sets in the same category (Statutory Registers, Church records and so on). Every single one of these links was initially broken, although they now appear to be working. Although this problem has now seemingly been resolved, checking for broken links is a basic feature of pre-launch testing, but clearly it wasn't done.

A search for Carstairs births in Crail 1926-1950 produces 217 results, none of which are from Crail.

There are however 9 results from Dunfermline, which is at least in the same county - Fife - as Crail. However, if you then search for Carstairs births in Dunfermline in the same period, you get the following results:

So there are Carstairs births in Dunfermline in this period, but you can't find them by searching for, well, Carstairs births in Dunfermline.

There are other issues, but I think by now you're getting the picture, and this post is long enough as it is. It's abundantly clear that once again, the NRS have launched an update that is not fit for purpose. It's hard to see how these bugs could have slipped through a properly designed testing regime. The implication is that the testing regime was at best inadequate. Some at least of these problems could readily have been avoided had there been advance consultation with users. As someone who's used Scotland's People and its forerunner Scots Origins since it was first launched in 1998 - I was one of the first 100 people to register with the site - I've previously offered to test updates free of charge. One of the key demands of our open letter was the establishment of a meaningful user group, something that the NRS management do not seem inclined to agree to. Perhaps this latest avoidable fiasco could change their minds.

For users, my advice would be not to trust any negative results and either postpone your research until they fix these problems, or else think creatively when searching - or be prepared for long trawls through irrelevant results.

PS None of this bodes well for the much-delayed launch of the 1921 census, which according to the NRS' own deadline is due to launch in the next 35 days. Fingers crossed they don't mess that up as well.