Unable to find
Hi!
I've requested access to the FamilySearch API for an indexing project my family has been working on. We have access to some records and are recording them in a Google Sheet. I am working to automate the process of transferring the data from the Google Sheet to FamilySearch (it will involve basically creating new people, attaching records, and the likes; all people added will be reviewed by real humans to ensure there are no errors).
It's been a few months now and it looks like my request to the API wasn't approved, and so I've continued to develop this automation process using Selenium for Python. And I've got it mostly working, but I'm running into a few frustrating tricks that are making it much harder than it needs to be.
There are some elements on FamilySearch that are easily accessible by XPATH, for example in the top bar. If I inspect, copy the full XPATH, and then CTRL + F search for it, then the element will be highlighted.
Copying:
Pasting:
However, if I try to do the same with any element after this main tag,
I can still copy the XPATH but I cannot search for it and find it. Selenium can't find child elements of this main tag by XPATH or anything else either.
So I've resorted to sending a bunch of tab keys and enter keys to select buttons, but this is really hard because I can't implicitly wait for elements to appear, I have to count how many times to press tab, it is easily breakable by small changes in the site, and I can't read content on the site (e.g., to know things like if it's detected possible duplicates).
I've attempted using .switch_to in Python, but <main> isn't an <iframe> or a <frame> so that didn't work. I've tried with both the ChromeDriver and GeckoDriver for Firefox. Any thoughts?
Thanks!
Anson
Answers
-
Your advanced technological insight is impressive. It is wonderful that your family is working on your own indexing project and that you want to use advanced technology to move the information from your Google sheets into the Family Tree program, but please use the simplified, basic process provided by the Family Tree software to enter the information you gather.
You can use the Add Unconnected Person link which is found at the bottom of the Recents menu when you are in Family Tree to create the new records for individuals that are included on your sheets. Using this option will give the system a chance to see if we already have records for the individuals you wish to create. Because our Family Tree program is designed to contain only one record for each individual, please help us to avoid duplication by using the designed software process for entering new records.
Once you have created a new record, open the details page to add any additional information you may have for the new individual.
While there is much advanced technology which may allow quick entries, the best way to be sure nothing is missed and that you do not create duplicate records is to simply follow the prescribed method of entering one individual at a time using the software as designed.
3 -
I am very surprised at your comment:
"...we ask that users...nor add a lot of data not related to your own family to FamilySearch Family Tree"
Most of my current work is unrelated to my direct family, so are you seriously suggesting I am infringing some formal, FamilySearch advice on the issue?
I have never been aware that FamilySearch sets any restrictions on what individuals, or data, we can add to Family Tree, so hope I am misunderstanding your suggestion. I would be grateful if you could clarify.
1 -
And yet the US1910Project and others continue to do so.
0 -
I still totally disagree. Where is the printed, FamilySearch advice on this?
As Aine says, there is the US 1910 Project (creating thousands of duplicates) that FamilySearch is happy to allow to continue. Also, indexing involves volunteers collecting names totally unconnected to their families.
Please clarify if this is your personal opinion, or kindly provide a link to the advice that I should only be adding data on my own family - even extended to quite distant relatives.
I think many LDS members will agree with your point, as their work primarily relates to adding names to take to the Temple, but the Church / FamilySearch has made this an open-edit program for public use. To this end, I am only too pleased to add names that members of the Church (possibly related to me in some way) can take to the Temple. On the other hand, why should I not add individuals in whom I have a just general interest, as part of what happens to be my main pastime - genealogy?
2 -
When I first read the original question, I fully expected that staff or someone in charge would either say that FamilySearch does not share the API or that the question would be closed with a message that someone in charge would be following up in private. I never expected the question to take this turn.
0 -
It is an important topic, however. Where is the line between an "unapproved extraction project" and needing to add every person in a village to be able to figure out which of the 30 Mary Smiths in town you are related to? What does this mean for One Name Study people who are working through hundreds of unrelated people all over the world who just happen to have the same last name and adding them to Family Tree? Is it fine to add and fix information in Family Tree in any amount as long as there is a FamilySearch index already in existence? As far as the US1910Project, since that data is public domain and really without any record custodian to worry about, limitations would probably not apply there, despite the concerns regarding its effect on Family Tree.
2 -
All people we are adding are related to us.
0 -
Unfortunately, the US1910Project is now adding relationships that have nothing to do with the 1910 census - for people who were not even alive to be enumerated in 1910.
0 -
I expect @N Tychonievich is referring specifically to projects such as the life work of Julius Billeter. The OP's family indexing project sounds similar.
Billeter was a Swiss immigrant to Utah who became a notable 19th Century LDS genealogist. He returned to Switzerland where he spent years making highly condensed extracts of parish church records for use by LDS families of Swiss descent. Since Family Tree came online many family historians and genealogists have spent enormous amounts of time entering Billeter's records into Family Tree and trying to reconcile each one of them with historical records.
I am a participant in that reconciliation process, and there are times I wish we had just ignored Billeter and worked from scratch with indexed scans of the original historical records. But of course starting over is not an option. It is not an option chiefly because Billeter's extracts were the basis of many LDS ordinances performed in the late 19th Century and throughout the 20th Century, all long predating Family Tree.
Just one of Billeter's surnames that I work on has thousands of duplicate records and thousands of tree fragments still waiting for reconciliation, even after many years of work by myself and others.
0 -
Here is how I would achieve your objective. Export from the Google Sheet to a plain text file in GEDCOM format, then upload the GEDCOM file to Family Tree. Do this in very small batches, perhaps even just one profile at a time.
What is the scope of your project, in geographic, temporal, and familial terms? There might or might not already be many profiles of the same persons on Family Tree.
0 -
The policy is stated in the Help Center Article: Can I do my own indexing of a microfilm and add or donate the data to FamilySearch?
The contents of that article:
"FamilySearch does not allow personal indexing projects. We cannot take donations from such projects. Doing so would risk violating FamilySearch's contracts with the record custodians.
To help make more records available for all to search online, please help us with current indexing projects at https://www.familysearch.org/indexing/."
Obviously open to various interpretations--as the rather heated argument here shows.
0 -
Unfortunately, this Help Center article is so vague that people will still do anything they want. In particular because the question, "Can I do my own indexing of a microfilm and add or donate the data to FamilySearch?" does not appear to have anything at all to do with Family Tree, just with contributing a new historical record database.
1 -
@Gordon Collett Given that the only way an individual could add the data to FamilySearch would be by either uploading a GEDCOM to Genealogies or adding to Family Tree, the article is probably clear enough. But I can take it to the authoring team to see about beefing up the language.
0 -
When reading this the thought of transferring data into GEDCOM format and importing as a GEDCOM file occurred to me. I have successfully converted spreadsheet records into GEDCOM files many times in the past.
However, I do not recommend this. For some reason unknown to me, the system for identifying possible duplicates when using the Compare facility with uploaded GEDCOM files is extremely poor. On several occasions I have added a person using this route only to open the new person in Family Tree and be told there is a possible duplicate with data virtually identical to the new person.
1 -
There are two separate projects different parts of my family are working on (with thousands to tens of thousands of names each), but both are confined to a fairly small geographical area (e.g., a town in one case and a region in Italy in another). Sometimes we have found existing duplicates, but usually not.
0 -
That sounds like a great approach. One of the things I would like to automate, however, is the linking to the original sources, getting the created URLs, etc. Do you know if that is possible with GEDCOM? Are there any resources you would recommend? We plan to have each auto-created entry be checked by a human for correctness and to ensure there are no duplicates.
Anyway, I have the whole process working (for adding an individual and his or her parents) by sending tab and enter keys to the browser, but I was just curious why these tags are unfindable after that <main> tag, it almost appears that the site was somehow designed to avoid Selenium automation.
This is an interesting discussion! I appreciate all the thoughts so far.
0 -
A GEDCOM file is plain text but it is capable of handling links. Certainly URLs can be put into fields. I do not pretend to be sufficiently expert to understand fully the way links work. However, if, for example, you download a GEDCOM from Ancestry and then upload it again (a method I have used to combine Ancestry trees when Ancestry says it can't be done!) then links to Ancestry records and other links that were in the original Ancestry tree are retained in the new tree.
1 -
Uploading a GEDCOM to Family search is a two part process. First you upload one to the Genealogies section where it is stored for you as a static file and searchable by other researcher. It will include sources but I don't think it allows live URLs.
To get people into Family Tree from there, however, you need to review each person one by one, checking against possible duplicates and either adding just part of the information or completely adding a new person. The routine is quite strict in what people can be added to Family Tree. If there is not enough information on a a record for the possible duplicate routine to check the record, you are not allowed to add it. It works pretty well for a small, say twenty to thirty person, family groups. For more than 100 people, it can quickly become a confusing mess and disaster. Also, any notes or sources in the GEDCOM file do not transfer into Family Tree. You have to go back into Family Tree and add them by hand later.
GEDCOMs are definitely not a good idea for a large project.
Even for a small project, GEDCOMs just give you a bare bones starting framework in Family Tree which can save a little typing time but require being willing to go back into Family Tree and finishing up each person.
3 -
To follow on other's remarks, your best strategy will depend on whether FamilySearch has many historical records of these people, or not, and Family Tree profiles, or not.
If there are many historical records, my suggestion would be to work directly in Family Tree, using the Google Sheet as a guide. You'll need to attach all the available historical records already in Family Tree, before using the Source Box to add any additional sources.
If there are few or no historical records, then it would make sense to first import as complete a GEDCOM as you can make for each individual.
Either way, you will find duplicates and they will need to be merged. I used to search for duplicates before I created any new profile but I learned I was wasting effort because often duplicates are not findable until after much detail has been entered on the profile and standardized.
1
