IMPORTANT PROJECT TIMELINE UPDATE
⭐️⭐️Project Timeline Update ⭐️⭐️
We have some exciting updates to share – we are making some changes to help accelerate the completion of the project.
First, let me say how much we appreciate the work that you all are doing! You have joined over 180,000 other volunteers in this monumental effort! We appreciate your joining us on this journey.
Next, in order to accelerate the publication of the census, we will be completing the volunteer review of the project by Labor Day, September 5th. We hope that we will reach about 25% of the Family Review by that point - we're at 14% now, and the work we all have done and will continue to do over the next couple months is still vital! The remaining Family Review will be completed through another review process so that we can bring the publication of the census to everyone more quickly.
Tomorrow, we will start transitioning some states to this accelerated review process. At that time, about 15 states will become unavailable for family review, then a few more in a few weeks, etc, until September 6 when we remove the final states. If a state you have been working on becomes unavailable, please consider reviewing one of the remaining states.
We are excited to see the publication of the 1950 US Census continue to accelerate, and we will all celebrate when the reviewed index is published! Thank you for joining us in this monumental effort to ensure that nobody in the census is lost to time.
The states being removed tomorrow are as follows:
Mississippi
South Carolina
North Dakota
West Virginia
Nebraska
Washington D.C.
Maryland
Georgia
Connecticut
New Jersey
North Carolina
Tennessee
Maine
Kentucky
Louisiana
Comments
-
I think this is suspicious.
0 -
Can you elaborate what this new accelerated review process is? IS it the same one that was used in Minnesota? If it works so well, why is it not being used in all the sates?
0 -
@WinertKevin, as was noted above, it will be a slow transition. Here is a quote I will share here for others to read:
⭐ Why publishing the 1950 US Census at an accelerated rate is important ⭐
Our announcement of the acceleration of the publication of the 1950 census has been, understandably, quite a surprise. It seems to imply that we care more about speed than quality. But let me share a little more about the direction FamilySearch is moving towards in the coming years. FamilySearch does care about speed - but we care more about helping people find their family, which requires speed AND quality.
In the next few years, FamilySearch will be transitioning most of our published records into a state where they can be continuously improved by those who are searching the records. At the same time, we will begin adding computer generated indexes at an unprecedented rate. This opens up so many possibilities!
Moving the census into publication sooner allows us to transition it into its next phase where we can have a better experience editing and fixing all the problems we saw that the volunteer tool wasn't sophisticated enough to let us fix. The current experience with the 1950 review will end in September, but we will be continuing to improve the census after it is published. Publication will allow more freedom to go into the records and improve what is important to each of us.
The exciting part is what computer-generated indexes open up for us - the ability to publish billions of records that would otherwise take decades to index. They may be of lower quality than what we see with general web indexing, but we will have the opportunity to review and improve those indexes continually. This shows how vitally important that contributors are and will continue to be to this entire process.
If the engineers had to keep fixing the current 1950 review tool for our project, they couldn't get to finishing the new editing tool that will enable this "review and improve" model for ALL of our records. This transition opens up more possibilities for us all.
Change can be hard...it's all going to take some getting used to. You've all come to enjoy doing the volunteer project the way it is now, and we are all so grateful for you. That doesn't make it easier, but we invite you to ponder these ideas and keep asking questions!
3 -
Thanks for the heads up about the future of FS Indexing. But it leaves me wanting more information. The change will be easier if we know more about where FS is heading and how long we can expect the transition to take.
From this quote, I don't understand entirely where FS Indexing is heading. Will researchers be expected to correct AI-generated records as they encounter them rather than OR in addition to a group of volunteers doing the same thing?
I hope that FS will share a fuller picture of the future model for Indexing at Family Search and ideally ask for feedback from the Indexing community before it is "set in stone" and released
The 1950 Census Review has undoubtedly been a disrupter of the status quo, and hopefully, our experience with it will inform the developers of the envisioned new platform. Please let us in on it sooner rather than later.
Thanks.
1 -
I'm sure this acceleration is necessary from a business perspective, but I'm still not clear on how this acceleration was executed -- is it a digital acceleration process (and if so, I'd love to learn more -- the very use of AI to decode 70-year-old handwriting is in itself fascinating), or did you outsource additional indexing to, say, India, or Indonesia, or the Philippines for these specific states? And why have you left the largest states at the time -- New York, California, Pennsylvania, Illinois, and Ohio -- out of your acceleration program?
1 -
One concern here is that records for low-quality forms will be unfindable. The present AI doesn't handle very well forms that are out-of-alignment, that have lots of extraneous marks, or very poor handwriting. I agree that the AI is on the cusp of being good enough, but how can we discover where it fails?
1 -
I'm guessing we're never going to get any answers, for any of our questions, for very obvious reasons.
1 -
For those in this thread, I'm sorry I am very sick but felt I needed to get on here this morning. I don't have the answers that you are looking for as I am not an engineer. I post what I am told to post. Please be patient and have faith in the system. Thank you.
4 -
Thanks. "Trust, but verify." Get well soon!
1 -
"I post what I am told to post. Please be patient and have faith in the system. Thank you."
Bless you, and I sincerely hope you feel better soon. However: "I post what I am told to post."
Yee-ouch! So, no one is able to address our concerns, because of horrible controls coming from some "system." What a nightmare.
1 -
The concerns above were expressed right before the weekend. I think we should let the next week start before getting too paranoid. I'm sure that the new changes are stressing out everyone at FS regardless of how well-planned they were. (I was a software engineer for 30 years.)
3 -
Well, 'we' waited out the weekend, AND a few extra days - NO further info ☹️
(But I'm NOT sure what those 'obvious reasons' are/were.) I guess we have no 'Need To Know"
0 -
Maybe @Heather R Jacobs could repost her Facebook post here to give a glimpse of part of the (near?) future of "Indexing."
1 -
Here is the post that I made:
💠 When will we be able to edit every field? 💠
With the announcement of the 1950 US Census volunteer process being completed by Labor Day, we've mentioned that soon we will have the ability to edit every field in the records. We thought you might like a sneak peek into what that will look like.
There will be new views that lets you see the information in the records. One will show the names down the right side of the page. Later, for some records, a view will be added to be able to see it as a table at the bottom of the image. You will be able to click on each name to see the details and edit the information. The editing history is being preserved to help with the integrity of the search.
Thank you to all for you amazing efforts. What you have done is paving the way for the future of review and we couldn't have done it without you.
2 -
Thank you, @Heather R Jacobs . Your post may raise other questions/issues, but it answers some of what @CaptBob is curious about by giving us an idea of the future.
Some of the questions/issues it raises are the exact nature of "Volunteer" conventional Indexing and Reviewing in the context of the above, the timing (at least in general terms) for the release of the above software and support structure, and the nature and timing of the eventual phase-out of our current practices and software, and the interactions between the old and new, while they co-exist.
1 -
@John Empoliti, thank you so much! And thank you, @Heather R Jacobs.
1 -
For the 1950 Census, PLEASE fix the locality database for searching below the State Level.
Whatever Database is being used for place names must have been made in "1800." The Counties I look at for Pennsylvania, I cannot find almost all the towns or townships, I want to review. I type in a town name, such as my Home town... which appears in all Census 1790-1950, but it does not come up as a suggested town to choose..
I have been wanting to review by town and township all along, but have NOT, because of whatever ridiculous locality database FamilySearch is using for the 1950 Census..
Why not use the locality database for the States as is used to suggest place names in Family Tree, or FamilySearch Catalog?
Fix the problem and I will review.
0 -
@Elmer, if I understand correctly, you are wanting to review certain towns or townships in Pennsylvania, but your search comes up with no matches to review them. Could you please provide me with the counties, towns or townships that you are trying to find. I need that information so I can check out the database, etc. It could also be possible that those areas have already been reviewed. I am anxious to help you if I can, but I need more information for replication.
Thank you so much for help us!!
0 -
Regarding the question about the states, additional states will be removed, a few at a time, over the coming weeks. The next set of state removal will be this coming Tuesday, and the states removed will be:
- Vermont
- Delaware
- New Hampshire
- South Dakota
- Rhode Island
@John Empoliti This transition will likely take years. The software Heather mentioned will be out in the next couple months, but it will take years to transition collections into this new tool. Indexing will exist for a long time. We just don't have a good sense right now for the transition timeline now, being at the very beginning of this process, other than that it's years, not months.
@Jon Neville FamilySearch as a general rule doesn't discuss its business practices. I am guessing the real concern here is the quality of what the index will be like? We acknowledge that our volunteers give us the absolute best quality indexes we have. This accelerated review might not give us the same quality our volunteers would give us. However, the accelerated review allows us to publish the index so that we can transition it into this new mode of editing.
3 -
Thanks for the clarification/refinement, @Janell Vasquez .
2 -
With all this rush, rush, hurry up going on, I just do not trust the integrity of the files that are going to be published for the world to see. I do not understand what it is exactly that Family search is trying to accomplish here, except...rush....rush...hurry up! I have already found several errors on 1950 census records for my family in various different parts of the country, and I have made those corrections through reporting them to Ancestry. And these are records that were supposed to have been checked. But this is based on me already knowing who these people are and where they live. Anyone new just researching their families may not be able to find the particular ancestor they are looking for if the name on the census was never checked or corrected. I get the change is hard thing, but change should be for the good not just to rush, rush, hurry up! Quality is pretty darn important with these files.
I am already starting to have my doubts about if all this work I am doing is even being processed through your system. I sure hope all this family review we are doing is not actually being ignored.
0 -
Currently the (indexed?) 1950 US Census appears to be available on MyHeritage (for months), Ancestry (with the computerized index), National Archives - etc. etc. (Google: "Available 1950 census indexes online")
It appears that FamilySearch is now 'simply' trying to refine the indexing process to make it more easily (and correctly) searchable. FS has been able to enlist a LARGE number of volunteers to achieve this goal. (I 'believe' that's the primary GOAL of THIS FS ongoing project.)
IMO, it will not (never) be 'complete' - until the online corrections are instituted, after this indexing project is completed.
The whole project could actually be simply a way for folks to 'help' everyone by serving as volunteers, hopefully providing them with a meaningful & useful project to allow them to serve others - and report that "I indexed 112,345 people in the 1950 census"
In other words . . . .
0 -
@Teri Luna I understand your frustration and lack of trust and am so sorry. I have copy and pasted from the 1950 US Census Project Frequently asked questions to help you understand what the process is since we are in contract with Ancestry on this project:
What is the difference between what NARA is doing and what FamilySearch and Ancestry are doing?
NARA released a name-only index on April 1. They are asking for volunteers to help improve this name index. You can learn more about this at https://archives.gov/1950census.
Ancestry’s handwriting recognition technology will create a comprehensive index of the census records, including all of the available fields. The 1950 US Census Community Project hosted on FamilySearch will enable volunteers to help review and improve this comprehensive index. This is a separate project from the NARA project.
Will the index and images be published on FamilySearch or Ancestry?
Both! After a state has been fully reviewed by FamilySearch volunteers, the index will be provided back to Ancestry, and both websites will publish the full index and images.
Thank you so much @CaptBob for you great explanation.
0 -
My reading from various sources is that there are four stages for the FS census: (1) microfilm, (2) AI-generated indexes, (3) temporary database being checked/corrected by humans, (4) "Final" searchable database. About half of the states are in stage 3—these are not yet searchable on FS. The other half have been moved to stage 4—some were fully checked but many are now being moved without full checking.
It's clear that even with 180,000 volunteers, full checking of the 150,000,000 records would take at least a year, perhaps even 18 months. If the stage-2 database is, say, 75% accurate for all important fields, why wait to make it available much longer? Really, the only true requirement is that 95% or so is accurate enough to be findable. Then everyone with an FS account can become a quality-improvement volunteer if they so choose.
In today's world, dissatisfied users go somewhere else (just look at some of the critical comments above). So FS is probably right in hurrying this along.
0 -
Ah:
Regarding the question about the states, additional states will be removed, a few at a time, over the coming weeks. The next set of state removal will be this coming Tuesday, and the states removed will be:
- Delaware
So, the two hours I just spent on households in Delaware is worthless? Moot?
I should just stop reviewing the 1950 census now, right?
0 -
@Jon Neville, no, you should keep reviewing! Your work is not moot and worthless! The more that can be done, the better. Thank you.
1 -
Well, thanks very much for your encouragement -- I think I'll stick to the big states from now on, though.
PS: Happy to know you're doing better! Hope whatever you were suffering from recently is over and done with.
0 -
Thank you so much, Jon. Yes, I recovered quickly! Back to normal energy. I appreciate your concern. Keep up the great work.
0 -
Okay, I guess we will see if the planned accelerated processes produce a usable product.
Like others, I am skeptical that a computer OCR can produce an index that will allow researchers to find the missing links and brick wall persons we are seeking.
Corrections after publication sounds like an inefficient and incomplete process to me. Many of these erroneous records may never be found in a search and therefore in turn, never be corrected!
I decided at the beginning of this project to focus on the area where I was born and raised - because I still recall the names on the storefronts, the names I read all the time in the newspaper, and the names our family encountered at school, church, and other interactions. I also remember the street names and the house numbering conventions used in the old home town. While my review was not 100% perfect (would have been a heck of a lot closer if I had been allowed to back up to previously reviewed family groups on the page I was working on - final full page review before submitting would have been extremely helpful), I would say my work corrected a massive number of mistakes made by the computer OCR and the volunteers (in name review)! Guess I'm glad the review web page is telling me off and on that there are no more pages matching my search criteria. Perhaps I'm close to reaching my personal goal.
Speaking of the "no more pages matching my search criteria," I'm a little puzzled to find that after I supposedly run out of pages, when I come back and do the search again, magically there appears to be more pages to review. Now I know some of these are pages that get recycled back into the queue because the review process didn't allow me to complete all families on the page (another topic, but this happens usually, when the OCR misreads the dwelling serial number for one family group and merges two families together that shouldn't be together. The programming cannot recover from me effectively splitting what it decided was a single family group.) So we get those pages back into the queue. Understandable. For other pages, it is not as obvious why they were not available for my search criteria and sneak back in. Makes me wonder if the Header Review process is actually still ongoing and new pages are available once in a while.
Sorry I diverged into multiple topics, but to me they are all related.
Isn't the whole point of the index to aid researchers in actually finding the correct people in the pages? Obviously we will have to deal with whatever changes in processes that are arbitrarily decided by the higher ups, but to keep changing strategies in the middle of this project for the sake of speeding up publishing and possibly sacrificing accurate search results sounds like bad strategy to me. The powers that be are also effectively slapping the volunteers in the face saying, yeah, thanks, but we think we can do the job better (?) and faster than you can. May not be the intent, but that's how it comes across. Just saying. So, when the 1960 Census gets released, what then? Guess you'll be on your own.
0