Duplicate Indexes from Nov 10
In the last couple days I've run into quite a few volumes of records (of various types) in the Bergamo, Italy images that seem to have had their indexes duplicated on 10 November. Here is one such case:
https://www.familysearch.org/ark:/61903/3:1:3QS7-99F6-Z3TT?view=index&lang=en&groupId=&personArk=%2Fark%3A%2F61903%2F1%3A1%3A6JYK-KP36
When looking at the history of the original, the index was created earlier this year. Then suddenly a bunch of duplicate indexes were created on 10 Nov. These are causing major problems, including: Multiple hints, multiple duplicate records to attach, changes that happen in one copy but don't happen in the other, lots of extra work and confusion, etc.
How do we get this information to the engineers so they can figure out what went wrong and fix the problem from the back-end?
Answers
-
@SerraNola can you help here please?
0 -
Bumping this in hopes that someone has a good answer here.
0 -
Is this issue the same as this one?
0 -
No, unfortunately it does not appear to be the same issue. In my case the record is still there and valid, has just had a second set of indexed data added to each page. So it is the index that is duplicated, not records or images. I've found that previously when the father was named, he was not given a surname, because in the records they are not named with surnames (Italian children at this time virtually always received the father's surname when the father was known) but in the new indexes they seem to populate the father's surname based on the surname of the child.
Here is a snapshot:
There are 2 births on the page, and each one now has 2 sets of indexed data. The first one here has already been linked to all the individuals indexed, but the second (new) one is not yet linked and is hinted at 3 of them. The father's surname has been added. This effectively doubles the number of people indexed from each record, with most having the same data, but the fathers having slighly the same data.
Another problem is that when I edit incorrect indexes or add additional information that was not originally indexed, it didn't appear in the new indexes and I would need to correct 2 records now. Plus all the extra hints would show the old information, confusing others and making it more likely they change the data on the person in the tree to match the new (incorrect) index.
I would really like to figure out how to ask someone at FS if this was intended behavior, and if so then why and how to deal with all the extra data, and if not then how soon they can fix it. Since the problem started 10 Nov, the sooner they fix the problem the less disruption there will be.
Thanks.
1 -
This appears to be a quite common, general problem. My main research involves English counties and some few cases (Northumberland comes to mind) multiple records have been added with the exact same data and relating to the exact same original source.
I wish FamilySearch had the resources to check records when adding them to the main database, as there is a lot of duplication of identical material that it should be possible to avoid if such checks were made. In some cases, the additional identical records have been added at different times, but I wonder if it is a case of someone at FamilySearch "pressing the same button twice" (or more) when the same date (e.g. 10 November) is involved?
I find the problem is creating far too much clutter in the Sources sections of the profiles I am working with. I'm okay with adding multiple sources for the same event (e.g. a marriage) if the source / indexing project is completely different. However, it is frustrating to be having to add (we are told we should still do this) truly identical sources - well, "identical" in every way, apart from their URL reference.
0 -
@marcoreid1 I found this double indexing in ten State Archive Civil Registration collections for Italy: Bergamo, Bologna, Brindisi, Enna, Foggia, Isernia, Milano, Palermo, Salerno, and Siracusa. All were updated between 10 November and 20 November. Not sure if they will try to fix these, but reporting—I hope, will prevent future occurrences.
1 -
@SerraNola Thank you for the additional information. Other than the "Feedback" button on specific pages or records, is there a separate method you're aware of for reporting widespread issues like this that I could use to report the problems?
0 -
@marcoreid1 No, you did exactly right to report it in both places.
0 -
Hi all, I posted the case of duplicated indexes about Ostuni (the mentioned by @Nyx773
I said records instead of indexes in the title of my post… sorry, my bad.
I rushed to attach my records as soon as I found my comune was indexed. Then, last week I had many more hints.
It'll be a lot of rework. Some of my profiles will have more than 40 sources.
However, in the first set of indexed data, most death records have indexed the father-in-law of the deceased. The name of the father-in-law is almost never a death record, but they created the father-in-law with the surname of the spouse of the deceased. I have been deleting this extra person, unless of course, the father-in-law was actuallu named in the record. Another more complicated error I found a couple of times was a misalignment of the image and data set. As far as I can remember I found it on marriage records only. I believe it happened when the indexed data of the first record in the book was aligned with the image of the index. So, the fix could be to add the correct information for that image and attach to the family tree, but can you imagine do that for the whole film? I reported the case and was able to add the correct information to the image and attach the record, but for the last 3 or 4 months I no longer can edit those records.
Choosing one to keep and deleting the extra set of indexed data would be the most risk and time consuming option. I believe the best to do will be to attach the second set as well.
1 -
I have come across this multiple times recently. Here is a link to a page of duplicated indexes in the 1875 NY State Census. There are no entries when viewing the change histories for each individual, not even a creation date. Can a mod weigh in on what best practice is for dealing with these? Is @CeciMindelli correct in thinking we should avoid deleting them?
0 -
A couple months ago, when I was reviewing the sources in a profile, when I tried to open the image of a source, it showed the message that the index was removed as it was a duplicate. I guess one user marked it as a duplicate and another attached it. But there are a lot more duplicates than deletions. My guess is that it's fairly impossible to index a whole film in a timely manor without any errors, and deleting duplications is very time consuming.
Therefore, I'm coming to the conclusion that the best way to handle this issue is to attach the duplicates :) I prefer to delete the duplicate, however, I recognize that this is a bit risky. Maybe we shouldn't incentivize people to delete a duplicate, especially if we keep the wrong indexing data with a major error, like data/image misalignment, and delete the correct one.
What are your thoughts?
0 -
I did try to delete one of the duplicate indexes for just one person about 5 days ago, as a test. Unfortunately, the person still even today has a hint to the deleted duplicate indexed values, but when I click on it I simply get the message "Something Went Wrong - Unfortunately, something went wrong and we are unable to display the record. Try refreshing the page, or come back later." No option to dismiss it exists. I'm afraid that having multiple of these per individual would be even more destructive to people wanting to view and adjudicate hints than most other solutions.
I'm also finding that this duplication isn't happening in ALL the volumes of records for the area I'm working in, just most of them. And I can't figure out why some are duplicated and some are not.
However, if we do nothing then each individual will have a bunch of non-adjudicated hints and it makes it hard to see the correct ones that should be handled. And if I go attach the second set to people, it means should also make the corrections/additions to the index that I previously made to the first index. This is effectively impossible to go back and do for indexed data already processed. At the very best, even if I attach them yet again and don't correct/edit them, the duplication will still cause much confusion and lots of extra searching and researching. This is not only a waste of the community's time and efforts, but also will cost FS a lot more money to push down all the images that will be opened to resolve the duplicates.
For me (and I realize I'm well beyond the typical user as in the last year I have attached ~30,000 sources and created ~6,000 individuals), this problem has stopped me dead in my tracks. I don't want to move forward and waste copious amounts of my time doing work that is either duplicative or will be destroyed if FS decides to correct this issue, just because nobody at FS will talk to us and be responsive to the community. I believe they should evaluate the problems they've created and either reverse them or tell us why they feel it is best to keep them the way things are. But there's just no real way to reach them that I can find. It is very disheartening. 😭 😡0 -
I'm just a volunteer user/contributor to Family Search, not an employee and not (sadly) "on a mission" but I have Always found FS volunteers, employees, moderators, engineering, missionaries (don't want to leave out anyone) to be kind, courteous, knowledgeable, helpful and responsive in every situation and I don't even speak computer!!! Can you imagine having to deal with questions and comments from literally 10's - maybe even 100's of thousands of people across the globe every single day??? Personally, I'm truly grateful for all their efforts and abilities.
0 -
My two penn'orth: there are clearly many very helpful FS staff and volunteers, and we get a lot of great assistance from mods and other FS staff members here on Community. But if we ask any even slightly 'strategic' question it tends to be difficult to get an answer. Another point I'd make is that the mods here tend as I understand it to be genealogists rather than IT/systems specialists. Finally, it is good to remember that the mods typically work Utah office hours.
1 -
@maryellenstevensbarnes1
I'm glad you've always found the answers you are looking for. I must be doing something wrong since I've not found any way to receive any communication about this issue after several weeks of trying. I've reported the issue through the feedback button a number of times (including my email address), I've posted on forums, I've looked for any other means of communicating with someone at FS who might be able to actually look at the back end and see what is happening and why, all to no avail. No communication, no acknowledgement that anyone who could look into it is even aware of any of this attempted communication.
So, since you've always been able to get your answers, maybe you can help me understand what I'm missing and doing wrong in trying to get some kind of response from FS. Because at this point I'm starting to feel my only option is to pack up the car and drive to the main FH Library in Salt Lake and in person request (from the very nice people there) to talk to people until I can reach someone who can call and talk to the right person. But it shouldn't come down to that.
There should be a proper structure in forums where issues (that clearly are technical, need FS response, and aren't already covered in existing answers) can be passed by designated employees or moderators to internal FS POCs for a response, and an acknowledgement of such (and hopefully soon an answer) posted back to the forums. There should also be at least some kind of acknowledgement when the Feedback button is used and an email address given, that the feedback is received by someone, which isn't the case. Without these kinds of responses, some users (like me) will start to feel like FS is being unresponsive. Because nobody is responding.1 -
I can only speak about my personal experience, but:
1.I wouldn't personally use Feedback if I wanted a response, since I don't think it is intended to be a two-way channel.
2.For any problem that can't be resolved immediately here in Community, I would reckon to tag the appropriate moderator, in this case @SerraNola who looks after indexes, records and images. They can engage with the engineers and other FS teams as needed and will usually be able to track the issue and update us when stuff is fixed (or if for some reason no action can be taken). What I don't think they can do is set engineering priorities (happy to be corrected on this).
3.If I didn't know who the right moderator was, I would hit Flag and choose an appropriate flagging option e.g. Report a Bug. Flagging requests moderator attention.
1 -
I understand your frustration and why it may feel like you’re being ignored. FamilySearch receives a tremendous amount of feedback from many sources, making timely responses challenging.
We are still looking into your issue. Since my response on November 22, our research has continued, and we've reviewed similar cases. It might be an important clue that in these Italian collections, one duplicate was originally indexed in Italian. If you have any further insights, please let us know. I should mention that the team handling indexing errors currently faces a significant backlog, which is why we aim to provide as much detail as possible.
As both a moderator and genealogist, part of my role is to ensure engineers fully understand the patron’s experience with each issue. We truly value your honest feedback.
2 -
@MandyShaw1 Thanks for writing about your FS community/staff experiences and @SerraNola thanks
0 -
@SerraNola thanks for the additional information and update. And to others for their insights.
I'm sure there is a vast amount of work being done, and sometimes it is easy to discount that when it isn't readily visible. I'm close to this issue, both due to the time I've spent and my family connections involved. And I've already seen and cleaned up various new duplicate people and incorrectly changed/updated data fields that have occurred as a result of this issue. And so I'm definitely motivated to see this issue resolved as soon as possible.
Regarding the proper channels for communicating issues, which mods have what responsibility, and other things mentioned by MandyShaw1 in her post: Is there some kind of pinned post or commonly available document describing which mods have certain responsibility areas, when it is best to flag certain things, or such? I couldn't find any, but maybe I missed it, so if one exists I would appreciate someone pointing me to it.
As to my having further insights on this particular issue, I'll share what else I've been able to learn in case it is useful: In recent days looked in detail at all of the 170 image groups (which I'll call volumes for simplicity) from the town of Vall'Alta (Bergamo province), comprising about 6,000 images. I also sampled 100 volumes (7,500 images) from the larger nearby town of Albino, to see if there was consistency (yes, there is) in the patterns between these places. These are all the registers for births, marriages, and deaths, but none of the announcements, indexes, or supplemental files (allegati). I found the following patterns that I believe are germane here:- About 89% of the volumes are currently indexed. 51 of those were indexed for the first time in November, apparently during the process that created this issue. So that's great to have more indexed volumes that were previously un-indexed.
- Of all the volumes I examined, 77 (28%) had duplicated indexes, and those duplicates seem to have occurred on only 2 dates: 10 Nov and 25 Nov, 2025.
- Every one of those duplicated volumes had originally been indexed on 4 Apr, 2025. There were no duplicated volumes with any other date of previous indexing, despite about 4 other dates appearing among the un-duplicated volumes. Not every volume indexed on 4 Apr, 2025 was duplicated, but the 77 duplicated volumes represent 70% of all volumes initially indexed on 4 Apr, 2025.
- The date range of all available volumes (indexed or not) is 1866-1944, but all the duplicates occur from 1901 to 1925. Not all volumes in that date range have duplicated indexes, but the vast majority (77 of 87, or 89%) of them are duplicated.
- In cases where a duplicate exists but the original was altered by a user (myself or others) prior to the 10 November date of the first duplication, the new index data is an exact match for the original data. So no corrected/edited data from the original index was passed into the duplicate index.
While I'm happy to share the data by volume (image group) if it is of any use to anybody, I suspect it can more easily be obtained internally.
Thank you for your help!0 -
Re 'Is there some kind of pinned post or commonly available document describing which mods have certain responsibility areas, when it is best to flag certain things, or such?'
We have asked for this, or alternatively generic usernames that we can tag, previously, but so far only an expanded and definitely improved flagging options list has materialised. Flagging @Sam Sulser in case you have not seen the detailed discussion on this thread.
0 -
@MandyShaw1 We would prefer that you use the reporting process in place. That is really the best way to get us. You can add details in your note and the post is included. All the reports go into the same place, where we can easily see them and we can assign them to someone if needed. This provides us a good way to track open issues and not lose site of them. Sam ☺️
2 -
@marcoreid1 Thank you for taking the time to share your research data. This information matches the indexing timelines I've tracked in other provinces, suggesting the duplication may have happened during a system migration on those dates. I'll share your findings with the engineers and contact you if more details are needed. As I noted earlier, I am unable to provide an estimated resolution time because this team is currently dealing with a backlog but I will try to relay the urgency.
Regarding issue reporting, I don't make those decisions. My role is to collaborate with records engineers, investigate problems, and respond to patron questions.
0 -
Will do.
It would really help imo if we could still see our flag reports after submission, or at least have visibility that we had flagged a particular comment, is this perhaps something that could be configured in Vanilla?
1
