Category: National Records Of Scotland

it's deja vu all over again

26/11/2022

Six years ago, when the Scotland's People website was relaunched with an entirely new interface, there were serious problems when it went live. I wrote about these problems on a number of occasions. (here, here, here and here). Over the course of the next six months or so, the developers gradually fixed some of these problems (see here and here).

Obviously, when a major new website is launched, you can expect some minor problems, but it was clear that many of the problems should have been picked up during testing before the new site was launched. It was also clear that some problems could also have been detected had there been proper engagement with site users.

So it's more than a little dispiriting, if perhaps not entirely surprising, to find ourselves back in the same situation with another "upgrade" to the website. In the last few years, the National Records of Scotland have not done themselves any favours in terms of public relations, with what could charitably be described as a reluctance to engage with users, which led me to prepare an open letter - signed by over a hundred genealogists, researchers and historians - to the head of NRS, Paul Lowe, to which the response was underwhelming to say the least.

On Wednesday, the Scotland's People site was down (and the centres closed) for a new upgrade. Communication about this was typically poor. I'm aware of one person who had a booking at one of the satellite centres who, when they contacted the centre to rearrange their visit, was told that the centre knew nothing of this down time!

So it was with a degree of trepidation that I visited the revamped website on Thursday when it came back online. And the problems were not hard to find.

One common technique Scottish researchers use is to search for the death of a married woman under two surnames, her maiden name and her married name. This makes it much easier to identify the correct record before you pay for it. And this was a major problem in 2016. So I searched for a death I knew was there - Margaret Carstairs died in Crail in 1857 age 49. This was the result:

Now I knew that Margaret's married name was Graham, so I repeated the same search changing only the surname:

Notice that the reference details are identical - so this is the same record, indexed under both Carstairs and Graham. So if I searched the records using both surnames, I should get this record again. Here's what happens though:

When they relaunched the site in 2016, this search was also broken, but in a different way - if you searched for a death with two surnames, it returned all results with either surname (a Boolean OR) rather than results with both surnames (a Boolean AND). This time round, the search fails altogether.

This is far from the only issue though. The SP home page has a big friendly box to get you started, where you can enter a name and a date range and search across all the records

Running that search shows the following results summary:

If you then click through to the any of the results for Old Parish Registers, it passes the date range as 17010101 to 17100101 (in other words 1 January 1701 to 1 January 1710). What that means is that any events recorded from 2 January 1710 to 31 December 1710 will be missed, even though they are there.

In 2016, I wrote about the strange results for Coupar Angus. This bug was fairly easy to explain - for some reason Coupar Angus had been assigned to the county of Forfar, whereas all other parishes in the same county were assigned to Angus. (Angus and Forfarshire are two different names for the same county). A minor issue, but one that is simply down to the way the data was recorded.

So I was curious to see if Coupar Angus was behaving itself. I searched for deaths in Coupar Angus between 1855 and 1900, and this is what was returned:

I'm sure we'd have heard if the good folk of Coupar Angus had discovered immortality, so this result seemed unlikely. So I removed the date range to see if that made any difference and suddenly the deaths were revealed:

But even that's not the end of the problems for Coupar Angus. I repeated the search removing the sort order and got the following results:

Last time I checked, Auchterarder and Perth were not Coupar Angus, so quite why those results appeared there is beyond me.

One issue with the original launch was the way filenames were assigned to saved images. Back then, when you saved an image, the files were simply given a datestamp as the filename - not very useful if like me you have thousands of images downloaded from Scotland's People. Five months after the relaunch, this was fixed so that images were automatically given meaningful filenames that allowed you to identify the record type, the year, registration district and so on.

Unfortunately, that problem is back - now the default filename is "ScotlandsPeople_imagenameraw", which is not very helpful at all, and would rapidly become unmanageable if you save large numbers of files.

At the bottom left of each search results page is a list of handy links to other record sets in the same category (Statutory Registers, Church records and so on). Every single one of these links was initially broken, although they now appear to be working. Although this problem has now seemingly been resolved, checking for broken links is a basic feature of pre-launch testing, but clearly it wasn't done.

A search for Carstairs births in Crail 1926-1950 produces 217 results, none of which are from Crail.

There are however 9 results from Dunfermline, which is at least in the same county - Fife - as Crail. However, if you then search for Carstairs births in Dunfermline in the same period, you get the following results:

So there are Carstairs births in Dunfermline in this period, but you can't find them by searching for, well, Carstairs births in Dunfermline.

There are other issues, but I think by now you're getting the picture, and this post is long enough as it is. It's abundantly clear that once again, the NRS have launched an update that is not fit for purpose. It's hard to see how these bugs could have slipped through a properly designed testing regime. The implication is that the testing regime was at best inadequate. Some at least of these problems could readily have been avoided had there been advance consultation with users. As someone who's used Scotland's People and its forerunner Scots Origins since it was first launched in 1998 - I was one of the first 100 people to register with the site - I've previously offered to test updates free of charge. One of the key demands of our open letter was the establishment of a meaningful user group, something that the NRS management do not seem inclined to agree to. Perhaps this latest avoidable fiasco could change their minds.

For users, my advice would be not to trust any negative results and either postpone your research until they fix these problems, or else think creatively when searching - or be prepared for long trawls through irrelevant results.

PS None of this bodes well for the much-delayed launch of the 1921 census, which according to the NRS' own deadline is due to launch in the next 35 days. Fingers crossed they don't mess that up as well.

4 Comments

What to expect from the 1921 census

31/8/2022

1 Comment

Work is well advanced for the release of the 1921 census records later this year and we are confident we will publish the census before the end of the year. We know customers are eagerly waiting this exciting release and we will provide a release date as soon as we can. pic.twitter.com/nxCUu5fmcw
— NatRecordsScot (@NatRecordsScot) August 31, 2022

Today the National Records of Scotland broke radio silence on the much-delayed publication of the 1921 census. I won't go into the details of the sorry saga surrounding the release of these records - suffice to say this has not been the NRS' finest hour. Indexing is about 60% complete. The contract was signed in January, and the NRS are now saying they are "confident" they will publish before the end of the year, whereas a few months ago they were saying they "will" publish in the second half of this year, it seems likely we'll have to wait a few months yet before we can finally see the first census after World War One.

It seems opportune therefore to set out what to expect when the census is eventually released. From the first census in 1801, the amount of information gathered increased regularly, making the census returns more useful for historians and genealogists alike. The 1921 census continued that process, including more information than its 1911 counterpart.

The individual household schedules are not being published - instead, what we will see are the enumeration books. It's important to understand the difference - the household schedules were filled in by the residents, usually by the head of household. The enumeration books are transcriptions of those schedules carried out by enumerators - individuals employed by the census office. So unlike the English equivalent, you won't be able to see your ancestors' handwriting.

Field	Notes
Schedule number	As in previous census, a sequential number for the household within the Enumeration District
Address	The address of the house or building. Standard abbreviations may be used, as may dittos
House rooms	The number of rooms in the house as a whole
House persons	The number of people in the house as a whole
Holding rooms	The number of rooms in the house occupied by the individual household/family group
Holding persons	The number of people within the household/family group
Name	Full name of each person in the household. 'Ditto' or 'Do.' may be used.
Relationship to head of household	Self-explanatory. Abbreviations to be used: Head - head of household Daur - daughter Rel - relative G-son - grandson G-daur - granddaughter Vis - visitor Serv - servant Br - boarder Lr - lodger
Age	Given in years and months
Marriage / orphanhood	An extension from previous census, with the following abbreviations used: S - single M - married W - widowed D - divorced BA - both parents alive FD - father dead MD - mother dead BD - both parents dead
Birthplace	For people born in the UK, county followed by burgh or parish. For people born outwith the UK, country followed by province or state. (Examples given in instructions: Canada, Quebec; India, Bengal; France, Charente)
Nationality	For people born in the UK, no entry required For people born in the British Empire, but outwith the UK Vis for visitor, or Res for resident For people born outwith the British Empire: Vis - visitor, or Res - resident and Born Brit - British subject by birth, or Nat Brit - British subject by naturalisation, or 'French', 'United States citizen', 'Chinese', 'Pole', 'Czecho-Slovakian' etc etc as the case may be
Gaelic	Left blank except for people aged 3 and upwards who can speak Gaelic. Options are: G - Gaelic only G&E - Gaelic and English
Occupation	Abbreviations discouraged, except for H D - Household duties
Value	Employment status. Options are: E - employer W - worker (employee) OA - own account (i.e. self-employed) Ret - retired OW - out of work People describing themselves as retired and out of work were to be recorded as retired only.
Employment	Name of the employer, if any, and a description of the business. For domestic servants (including such workers as gamekeepers), the word Private is to be recorded
National health insurance	If the person has national health insurance, the letter M or F to indicate that they are male or female If the person does not have national health insurance, column to be left blank
Dependent children number	Number of dependent children
Dependent children ages	Ages of dependent children. Boxes for each age 1-15, with an x placed in each applicable column. If there are two or more dependent children in the same age band, the xs are to be placed one above the other.

The other change in terms of what the NRS are producing, is that once the indexing has been completed and the images published on Scotland's People, a complete transcription of all the fields will be produced as part of the contract. This is something academic historians have long been pushing for, as it makes all sorts of historical research possible in a way that isn't feasible with indexed images alone. As yet, though, it's not clear if this transcription will be made publicly available when it is eventually complete.

1 Comment

it's deja vu all over again

What to expect from the 1921 census

Old Scottish

Archives

Categories