Census occupations missing - again, now 1901

djb2 · February 24

This was an annoying "feature" introduced last summer - occupations were not being shown on 1851/61/71/81/91 censuses, though they were still showing on 1901/11 censuses. I was hopeful that it was just a bug that would get rectified.

The problem is that now the 1901 census no longer shows occupations either!

https://www.familysearch.org/ark:/61903/1:1:X9FV-TYD?lang=en

No occupations are shown, but reference to the underlying XML clearly shows them:

https://www.familysearch.org/platform/records/personas/1:1:X9FV-TYD?lang=en

There was a precious discussion that was helpful, but has since got closed, so I can't add to that one, instead creating a new one.

Added to the 1911 census now only showing industry (not occupation), one wonders whether it's just bad programming or whether there's some plan here to push everyone over to the expensive websites?

There was an update to the 1891 census in February, but nothing to the 1901 one, so the problem is some change in the viewing technology (css file?), not the underlying database.

(And as for the appalling forum editor that tries to access inserted links, then loses them, and deletes inserted images randomly, so I've had to have several goes at typing this…!)

Previous discussion here:

https://community.familysearch.org/en/discussion/165700/census-occupations-missing

djb2 · February 27

Hmm.

Perhaps no-one else is bothered about this? or perhaps no-one else has the same problem, it's just me? perhaps no-one else looks at the occupation fields?

I've tried contacting familysearch support, but so far no response…

MandyShaw1 · February 27

This is clearly yet another occurrence of a particular metadata label being present in the underlying data, and retrievable via the Record Persona API, here https://www.familysearch.org/platform/records/personas/X9FV-TYD, but not being reflected on the Record display for a particular collection (here the offending label is PR_OCCUPATION_OCCUPATION_ORIG, which seems a bit weird in itself).

The situation hasn't really changed since the previous post you quoted.

@SerraNola can you help move this forward please?

djb2 · February 27

Thanks Mandy.

"The situation hasn't really changed since the previous post you quoted."

…well, it's got worse, since it now affects 1901 & 1911 as well!

MandyShaw1 · February 27

We really need to identify

a) metadata label(s) that were showing on 1901 and 1911 before but that have now ceased to do so, and/or

b) records where metadata label(s) in the underlying data have been modified recently.

Can you help with scenario a) at all @djb2?

I should have some relevant metadata stashed that I can compare between now-ish and say 6 months ago, which might help exemplify any scenarios of type b). Probably won't have time to look until Sunday afternoon though.

djb2 · March 4

Unfortunately, I don't have any xml data or metdata saved away - I only found out the xml data existed when you pointed it out back in August! Thanks for that, by the way - I have used it to find occupations for some people where there's some particular reason I need/want to check, but it's too time-consuming to do it for everyone. And I certainly haven't saved/screenshotted the data.

Incidentally, I did get an answer from fs:

"Thank you for contacting FamilySearch to report the disappearance of some of the information showing in the indexed data for the England and Wales Census records. Given that the loss of data seems to be across-the-board in these census records, we believe that the change was made by FindMyPast.com, as they are the holders of these records. You might want to check to see what the index looks like on their website. You will also be able to view the census images from findmypast.com. You can use the site without creating an account at a FamilySearch center."

I have replied that I don't think it's to do with the indexes, since the data exists - just that it might now be differently labelled, so the display code needs to change to match. It's bad of fmp if they really have changed (and broken) the way in which data is sent to fs, but one would have thought fs could be looking at dealing with it by now!

MandyShaw1 · March 4

Thanks for reminding me, I'll have a look at this again asap, and I'll look at the data on FMP for comparison purposes.

I've built (and keep up to date) an analysis database of fairly random profiles and their change logs, notes, sources, etc., plus the relevant Record metadata - it's really useful for this sort of thing.

djb2 · March 4

I've now received a reply from familysearch:

"Thank you for your follow-up. We appreciate your concern that the folks who work on records at FamilySearch should have a way to correct the xml associated with the England and Wales Census records. Unfortunately, we in support have no direct way to alert them to the issues.

In order to get issues to the people who can fix things, reports about records issues need to come in one of 2 ways:
--From use of a Feedback button on a page showing the problems. Since no feedback button shows on the pages with the census transcriptions, this option is not open to you.
--Through the FamilySearch Community. This would be your best option. Here is how to report the issue there:[…]"

…Which I think brings me back here:-)

MandyShaw1 · March 4

It's not necessarily the xml that's the problem, hopefully they realise that!

Paul W · March 4

@SerraNola

That occupation detail can be really helpful in determining the likelihood of whether the individual you have found in one census is the (same) person you think he is - especially when there are a number of individuals of the same name and even age, who lived in the same area in the same period!

This doesn't appear to relate to "permissions" (e.g., Find My Past suddenly withdrawing approval for FamilySearch to openly publish occupations), so would be grateful if you could confirm if you will pass (have already passed) this on for further investigation. Thanks.

SerraNola · March 4

This is one I've had on my To Do list to take a look at. I can't find evidence that it has ever been brought to the attention of anyone in engineering. No need to convince me of the importance—I have Welsh Jones and Evans ancestors. Thank goodness one of them was a grocer.

Mandy, I would be grateful for any pertinent data that I can take to engineers.

MandyShaw1 · March 5

@SerraNola this is where I've got to.

I've compared current data (well, from a couple of weeks ago) with data from August 2024.

1901 and 1911 England and Wales censuses
Distinct personas in August 2024 database	11519
Of which Occupation related label(s) present	6048
Of which labels are different in current database (these are also the only ones with changed citation dates)	272

As far as I can see, the only Personas that currently show Occupation on the display are the ones with PR_OCCUPATION labels, and the only Personas that currently show Industry on the display are the ones with PR_OCCUPATION_INDUSTRY labels.

The 272 all previously had PR_OCCUPATION with or without PR_OCCUPATION_INDUSTRY. These appear now to have been replaced by PR_OCCUPATION_OCCUPATION_ORIG and PR_OCCUPATION_INDUSTRY_ORIG respectively. Obviously this means the Occupation/Industry information does not show on the display at all.

Looks to me as if some mappings got changed when the most recent batch of updates was loaded into the FS database, but the display logic has not yet been changed to match.

Here are a selection of the 272:

Persona	August Citation Date	August Occupation Related Label(s)	Recent Citation Date	Recent Occupation Related Label(s)
X7GM-7H3	18/07/2019	PR_OCCUPATION	13/02/2025 05:14	PR_OCCUPATION_OCCUPATION_ORIG
X7GM-7H7	18/07/2019	PR_OCCUPATION	13/02/2025 05:14	PR_OCCUPATION_OCCUPATION_ORIG
X7GM-7HM	18/07/2019	PR_OCCUPATION	13/02/2025 05:14	PR_OCCUPATION_OCCUPATION_ORIG
X7GM-7HQ	18/07/2019	PR_OCCUPATION	13/02/2025 05:14	PR_OCCUPATION_OCCUPATION_ORIG
X7GM-7HS	18/07/2019	PR_OCCUPATION:PR_OCCUPATION_INDUSTRY	13/02/2025 05:14	PR_OCCUPATION_INDUSTRY_ORIG:PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRD	23/07/2019	PR_OCCUPATION	13/02/2025 06:45	PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRF	23/07/2019	PR_OCCUPATION	13/02/2025 06:45	PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRJ	23/07/2019	PR_OCCUPATION	13/02/2025 06:45	PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRN	23/07/2019	PR_OCCUPATION	13/02/2025 06:45	PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRV	23/07/2019	PR_OCCUPATION:PR_OCCUPATION_INDUSTRY	13/02/2025 06:45	PR_OCCUPATION_INDUSTRY_ORIG:PR_OCCUPATION_OCCUPATION_ORIG
X7JG-TRX	23/07/2019	PR_OCCUPATION	13/02/2025 06:45	PR_OCCUPATION_OCCUPATION_ORIG
X7JR-VVB	23/07/2019	PR_OCCUPATION	13/02/2025 17:10	PR_OCCUPATION_OCCUPATION_ORIG
X7JR-VVY	23/07/2019	PR_OCCUPATION	13/02/2025 17:10	PR_OCCUPATION_OCCUPATION_ORIG
X7KW-M78	22/07/2019	PR_OCCUPATION	13/02/2025 14:03	PR_OCCUPATION_OCCUPATION_ORIG
X7KW-M7H	22/07/2019	PR_OCCUPATION	13/02/2025 14:03	PR_OCCUPATION_OCCUPATION_ORIG

I have also found quite a few Personas where the Occupation uses a PR_OCCUPATION_ORIG label. These go back further (my examples mostly have citation dates in 2022). These obviously aren't reflected on the display either. (Some of them are accompanied by PR_OCCUPATION_INDUSTRY_ORIG's which clearly are also missing from the display.)

Here are a couple of these:

Persona	August Citation Date	August Occupation Related Label(s)	Recent Citation Date	Recent Occupation Related Label(s)
XWZW-FYV	24/03/2022	PR_OCCUPATION_INDUSTRY_ORIG:PR_OCCUPATION_ORIG	24/03/2022	PR_OCCUPATION_INDUSTRY_ORIG:PR_OCCUPATION_ORIG
X4M2-8Z4	29/03/2022	PR_OCCUPATION_ORIG	29/03/2022	PR_OCCUPATION_ORIG

Let me know if you need any more evidence.

SerraNola · March 5

@MandyShaw1 Thank you! I will present this next week—this week everyone is on overload with RootsTech.

djb2 · March 10

Many thanks both @MandyShaw1 and @SerraNola for looking into this, I look forward to an update when you're able to!

djb2 · June 3

Looks like something's changed over the last few days - I'm seeing quite a few occupations coming through now, on most 1851/1861/1871/1881/1891/1901/1911 censuses.

MandyShaw1 · June 3

Excellent, thanks a lot to the engineers (or whoever fixed it) @SerraNola!

djb2 · June 4

Indeed, many thanks, it's much appreciated!

Paul W · June 4

I'd like to add my thanks, too. The missing occupations issue could well have led to quite a lot of incorrect merges, where two individuals have identities (other than occupation) that are so similar.

I wonder how this problem ever arose, however? Not, apparently, due to any contractual issues (Find My Past would have had to imposed restrictions with these records), so how / why was this (Occupation) field suddenly withdrawn from view, when it still remained in the "underlying" data, as illustrated in previous posts?

Sadly, this is not the first time that a similar issue has arisen, and this can prove to be very detrimental to the quality of data that finds it way into Family Tree. To whom it may concern: please take care when tweaking the coding!

Census occupations missing - again, now 1901

Answers

Categories