Rating Source quality w/ Tags using letter grade to classify each Source. Objective criteria used. E
Comments
-
m said: I used letter-grade, but we could also use a numbering system and make more categories.
I expect the wording of each to be expanded as the system gets tweaked.0 -
Adrian Bruce said: But how do you know it's the wrong person? If the source, or the source/person combination has been rated A+ - why would most people look to check it's the right thing? It's encouraging people to accept what's there without critical thinking if all we do is assess the origins of the document.
An assessment system that ignores these other aspects seems to be missing a dimension or two.0 -
Adrian Bruce said: But a Death Certificate is an original and therefore A....
What this illustrates is that the original image rating is separate from the Primary or Secondary information rating and shouldn't be stuffed into one single letter value. The ideas are important but stuffing at least 3 different ideas into one single letter is not a good idea.0 -
Adrian Bruce said: Rating a Source-Record on its own is only part of the story. It also needs to be compared against the fact that it purports to support - that's why the ESM system has 3 different ratings.0
-
m said: How do you know a Source currently Tagged as "Name, Burial" actually has the name and burial? You look.
If a Source is Tagged as "Name, Burial" does not have those things, you can change the Tag.
Similarly, if a Source is Tagged as "A+" and it does not have an Image of an original record, you can change the Tag.
Simple.0 -
m said: Oh, you are talking about an Image of a Death Certificate. That did not even occur to me, for some reason. I was thinking about something else.
If it is an FS indexed record in this system it would be an A or A-.
But maybe Death Certificates since they can have errors due to people giving wrong info, we should just decide as a forum which level of general quality we want Death Certificates to have and include them in that level and write "Includes Death Certificate."
I'm expecting/hoping the whole system will be tweaked / molded by the forum.0 -
m said: Sources can offer contradictory evidence.
Let's say 3 Sources with contradictory evidence are Tagged as "Burial."
If one has the image of the original burial record, it might be Tagged as A+.
If one is Geni and says burial is in say a different location, and let's say it does not cite a record for this individual so it might be Tagged as F.
If one is a book and says burial is in a different location, and let's say mentions at least 1 record for a fact such as a birth/marriage/death for this individual, but the book was written in 1950 is outdated and several things have been disproven in the book, so it might be Tagged as D.
Then I think that each patron can "eyeball" these ratings and get a general sense of what is available for that ancestor on "burial" and a general sense of the quality of each.0 -
m said: Another example of Sources offering contradictory evidence would be:
Let's say 3 sources are Tagged "Birth."
Let's say all 3 are FS indexed Census records for different time periods. They all are Tagged as quality A.
Let's say all 3 have a different Birth state listed: NY, PA, NJ.
Then each patron can "eyeball" these ratings and get a general sense of what is available for that ancestor on "birth" and a general sense of the quality of each.0 -
Jeff Wiseman said: As far as the "Quality" of a source goes, it begs the question, WHICH quality?
E.g., a Death certificate is a primary source for death information, it it also frequently contains, birth information, which is at BEST secondary information. You can't assign a single rating to a source because different parts of it can have large difference in trust levels.
I've discovered that during a period of many years in southern Ohio, you just can't seem to trust ages or birth information on the marriage license of a young woman's first marriage. A very LARGE percentage of them have lied in order to get married when they weren't legally old enough. And yet the date of the marriage is a primary piece of data.
Also, GEDCOM X has already defined a set of source quality values to be used. A lot of companies would have to be convinced to change them in the standard. Either that or you just create your own proprietary method that would not convey across other genealogical databases.0 -
Adrian Bruce said: The 3 ESM criteria that I list also have the advantage of already being recognized by a number of genealogists. Their adoption (and I've no idea if GEDCOM-X mentions them) would also avoid the invention of yet another scheme.
Your reference to the ages on marriage documents makes me realize that the ESM scheme does not cover that aspect - the reliability of the information relevant to the point at issue. That's not an accidental gap, I suspect. The 3 ESM criteria are objective criteria that, if adopted, would produce values that most people (who understood the concepts) would agree with - that's because they are objective.
ESM didn't encode a reliability value because, I'm guessing, the 3 objective criteria were the input to a process intended to produce an assessment of that reliability and that assessment is way too complex to be easily encoded in one single character. Sentences of text are far more appropriate.
Further, the 3 ESM values are objective and 2 genealogists are unlikely to disagree over their values. Whereas the final reliability assessment is subjective and 2 people might easily disagree depending on all sorts of things - the starting point of their knowledge, for instance.0 -
Tom Huber said: Wow. Talk about getting a lot of opinions for a very good topic.
The idea of source quality is important. It gives each user an opportunity to rate the quality of the source. It doesn't matter if the user is experienced or a novice (and anywhere between). What they are saying through the rating they give the source is the usefulness and accuracy of the source to that particular user.
An experienced user may be more critical of the source while a novice may think it is great.
The ability to break down the rating according to the contents is specific. If a source is great, but lacks an included image of the original, it is not going to get an A+ (or should not).
Now, what do I mean by great. Is the index accurate to the original source? In most cases, an A only rating means there is no attached image.
But... if there is a known error in the index, then at best, the quality would be A-.
Most original records (or indexes) would get some level of "A" (A+, A, or A-) rating.
B, C, and D are more about the record type, specifically published trees, histories, other sites, and so on, that are sometimes used as sources. This would include an Ancestry tree, or a GEDCOM file. If no sources are provided, the that source would rate an F. But if a source to an original record was included, then B, C, or D would apply.
Very few published trees or family histories would rate a B, but if they did, then they would be heavily sourced back to original records. Only some Ancestry Trees could qualify, and they would be extensively researched with sources, photographs, and other supporting documentation. Only a few trees are this good.
A rating of C would requite at least one source back to an original record.
A rating of D would require at least one source back to an original record, but would contain known (proven) errors in the data and/or sourcing. Among the Pennsylvania Mennonites, there are several volumes published by Ira Landis. Because of the details, I originally would have rated the overall work in the B or C category, but analysis against published sources dropped its rating to a D. The biggest problem is that for families at the time of publication, there was a significant amount of hearsay involved with Ira's work.
A rating of F says that there are no records cited in the source. Many GEDCOM files fall into this category, because they did not have sources embedded in the file. The GEDCOM ingest into FSFT falls into this category, as well, since the ingest fails to carry sources into the uploaded genealogy (Another reason why FS should not allow GEDCOM files to be ingested into the massive tree -- no sourcing).
The Gustav Anjou genealogies fall into the proven fraud category (!). So do the published histories derived from such genealogies.
The X rated source should not be attached as a source for a person's profile. The reason is because the source fails to mention the person. SAR and DAR applications are often used when they do not apply to the individual. My Anthony ancestor falls into this area because he did not serve as a Revolutionary Soldier.
In the past, we generally rate a source as being primary, secondary, or "other" but for the most part, the definitions vary from researcher to researcher. Note that many primary and secondary sources (the way I define them) could get an A rating of some kind.
A lot more could be said to further define the lower ratings, providing samples of why the rating applies. For example a D would apply to a gravestone with a bad date or name spelling (a mistake made during the creation of the gravestone). In the case of older gravestones, readability becomes an issue and can impact the quality rating.0 -
Tom Huber said: Assigning this quality rating applies to the source as it is attached to an individual's profile. One thing is missing: fabricated information, such as a burial date that FS adds to the Find a Grave index when the original Find a Grave entries do not have burial dates. Thus, if the burial date is carried forward in any way to the profile, the source rating drops needs a rating less than F, but higher than ! or X.0
-
Tom Huber said: What assigning a quality to an attached source would or should do is to encourage FamilySearch from adding unsubstantiated / fabricated data to the index.0
-
m said: Definitely, and that needs a thread to bring that up. Has anyone made a thread on fabricated data in the index?
But at least under a Rating Source Tag system, you could change the Tag of an FS indexed Source away from A to something appropriate for fabricated FS index burial Sources that the forum would decide to add into the rating system, such as A- F ! X or some other -- (this forum could also expand the rating system to have more levels and use a number system.)0 -
m said: "GEDCOM X has already defined a set of source quality values to be used."
OK, what are they? Maybe we can incorporate them. I'm on the GEDCOM X website and I don't see any mention of them.0 -
m said: Lying on a marriage record...
that is already covered in this system:
A- = "record with error(s)" -
indexed record ...mentioning this individual by name...AND contains at least 1 proven error = A-
0 -
m said: And for lying on a marriage record there would be a changelog and you would write "lying on a marriage record..." on the changelog --- for the purposes of collaboration --- so more people would become aware of the lying on that marriage record.0
-
m said: ESM
The Rating Source Tag system will give a 2nd dimension, not the full 3 dimensions...but it will help patrons see the full 3 dimensions.
The point is to make it so specific that patrons will be able to "eyeball" the general quality of the Sources present for something, say a birth, and then when they look at the Sources tagged "Birth" and they see 4 Fs that means in general the Sources for birth so far for that ancestor aren't looking so hot; if they see an A+ they know in general the Sources for birth so far for that ancestor are looking promising and they want to make sure to look at that one.
Examples from above for contradictory evidence. Notice how the "eyeballing" of the quality of Sources for one thing gives you a 2nd dimension...(which will help you see the 3rd dimension):
Sources can offer contradictory evidence.
Let's say 3 Sources with contradictory evidence are Tagged as "Burial."
If one has the image of the original burial record, it might be Tagged as A+.
If one is Geni and says burial is in say a different location, and let's say it does not cite a record for this individual so it might be Tagged as F.
If one is a book and says burial is in a different location, and let's say mentions at least 1 record for a fact such as a birth/marriage/death for this individual, but the book was written in 1950 is outdated and several things have been disproven in the book, so it might be Tagged as D.
Then I think that each patron can "eyeball" these ratings and get a general sense of what is available for that ancestor on "burial" and a general sense of the quality of each.
Another example of Sources offering contradictory evidence would be:
Let's say 3 sources are Tagged "Birth."
Let's say all 3 are FS indexed Census records for different time periods. They all are Tagged as quality A.
Let's say all 3 have a different Birth state listed: NY, PA, NJ.
Then each patron can "eyeball" these ratings and get a general sense of what is available for that ancestor on "birth" and a general sense of the quality of each.0 -
Paul said: Sorry, but I don't get any of this! The only important factor to me in rating a source is the accuracy of the information it contains.
In which case, why, 'A+ = "image &/or text of original record"' ?
I have copies of original records that contain inaccurate information, why should I rate them so highly just because of the nature of the "evidence"? For example, say the image of a headstone records the wrong date of death, or a parish register baptism entry has the wrong name for the mother?
Any such ratings would be of no use at all, because I would want to check, where possible, for consistency across a range of sources (including even "hearsay") and weigh up the overall evidence, not base my conclusions / ratings on an individual source.
Interesting that census sources are being left out of the discussion. Successive census records often record a different age (hence year of birth) and birthplace for many of my relatives. How do I rate the 1851, 1861 & 1871 census sources attached to my relative if the detail on each is totally different? In certain cases, I am sure I will never get to find out the exact location of a relative's birth.
You just cannot categorise on the basis on the type of source. Nor can you avoid the fact that an A+ source cannot be rated such if you are not 100% sure it has been attached to the right person.
As I have stated so often previously, FamilySearch's record hints (including possible duplicates) have caused me so much wasted time, in having to detach / unmerge records that inexperienced users have trusted because they have believed these were reliable suggestions.
What is the difference here? An inexperienced user sees an A+ rating for a source and doesn't hesitate in accepting its integrity, so sees no need to validate the detail it contains.
Apart from "m", would someone else please try to convince me this is not the totally unworkable and misleading scheme that it appears to my mind?0 -
m said: Paul, please see Tom Huber's more elaborate explanation of this system above.
As to A+:
The highest rating goes to the Image of Original Record...
(....whether or not the text is included).
That's it.
[Because most people would agree that is the most accurate Source.]
Any proven error on the Image of Original Record would force a downgrade to A-
[Because A- includes image of original record with proven error.]0 -
Adrian Bruce said: Re GEDCOM-X and its linkage to the ESM Process Analysis Model.
I have found https://github.com/FamilySearch/gedco... - this only appears to define types of Derivative and Authored Narratives - I can't see how / if it applies to originals.
In addition, the list of Derivatives (abstract, transcription, translation) is thin compared to ESM:
"materials that offer alternate versions of the original—typically transcripts, translations, abstracts, extracts, nutshells, indexes, and database entries. The best derivatives will preserve all the essential details of the original. Still, errors are frequent."
I may have totally missed other references and I certainly can't find anything about Primary / Secondary. Maybe it's elsewhere...0 -
Paul said: With respect, m, I wanted someone else to explain to or convince me of the validity of your suggestions. Your example of downgrading from an A+ to an A- just confirms my feelings about how silly it would be to implement your idea:
(1) Still give it an "A-" even though you know if contains errors?
(2) In many cases, only serious research will show there even is an error.
Most people will accept the details contained in a source - especially if there is an image of the original record, say a parish register, but many parish clerks (for instance) are known to have been virtually illiterate.
I must admit, on the "source with image" basis, I have made many inputs, assuming they were factual, and it has taken a lot of work before I have found the truth to be otherwise. This is work that I could not possibly spend time in verifying for the thousands of IDs I have added sources to in Family Tree. Whilst I make such inputs on the basis of there being (at the time) no evidence to the contrary, I just do not want to be responsible for indicating to any user (especially of the trusting, inexperienced type) that I am "A+" convinced of details for which I (and I bet most other FT users) have not necessarily had time to fully validate for complete accuracy.
An A+ rating from an an experienced FT user is no better to me than one of those notorious FamilySearch Record Hints (source and all). Both look convincing, but neither are helpful to the inexperienced user, who is prone to accept anything that looks reliable (whether an A+ rating from a user or a FamilySearch hint). These suggestions might be made in good faith, but prove to be totally inappropriate.0 -
Paul said: Adrian
Whilst I read the "ESM" piece with interest, I admit I regularly challenge what ostensibly appears to be sound reasoning on the criteria for establishing evidence relating to ones genealogy. I find myself particularly irritated when the word "proof" appears. If we relied on absolute proof, most genealogists would not be able to enter detail of, say, paternity much beyond one or two generations - DNA evidence being the main consideration for offering, say, 99.9% "proof" of this.
Even without taking arguments that far (i.e. who knows who your ancestors' biological fathers really were - sometimes, even their mothers), I would always prefer the term "extremely sound evidence" to "proof".
As you can see from my other posts, I would no more wish to convince someone else of my "genealogical beliefs" (e.g. how reliable a source matched my ancestor's profile) than, say, tell them that certain religious or political beliefs are the correct path(s) to follow.
I believe every person should be free to make their own conclusions as to (as applicable to this thread) how good a source is - and that it should be only partly rated on the basis of its nature (as listed by "m"), but mostly on the known evidence, or lack of such. Recorded sources are notorious for their lack of accuracy, in spite of m seeming to believe inaccurate content is very much an exception. Often, the flaws are difficult to discover - but I am amazed m considers a source should only be slightly down-rated, even if it is found to contain errors.
To be honest, I am surprised at you and Tom treating these proposals so seriously when, to me, you can drive a truck through them.
Apologies to m for my attitude, especially when he has taken such serious efforts in presenting the arguments for his ideas. I would honestly be delighted if someone could explain why we should ever need to rate sources primarily to type, rather than quality of evidence, however.0 -
Adrian Bruce said: Re Proof - James Tanner (frequent blogger) has similar issues with the word Proof. However, he reckons that it's a legal concept and it's wrong that genealogists use it. Since I'm a mathematician by training, I know that the only valid use of the word Proof is in maths so since the lawyers have already pinched it, I can't get too upset about the use of the same word by genealogists.
Besides, any court room drama on TV always starts with a proven case that Colombo, Rumpole, etc always disprove, so I think that the general public knows full well that a Proof is never permanent.0 -
Tom Huber said: It isn't that I'm treating a rating system with particular seriousness, it is just that the current primary/secondary/other system needs to be standardized.
Perhaps m is going overboard, but I don't think so. What this is doing is forcing some kind of rating system, whether it is a sliding scale (similar to rating a response or service (from 0 to 10, for instance), or giving a letter system to it, it is badly needed with sources, especially with works like the Anjou genealogies out in the wild.0 -
Adrian Bruce said: I would still contend that trying to fit everything into a single code value is pointless when the assessment consists of several factors. For instance, trying to fiddle Death Certificate images into various places. And it's another system that'll be unique to FS, so how does it get synchronized?0
-
m said: Please read Tom's elaborate explanation and let us know where on the scale YOU would rate the quality Original Image of a Record with a proven error.0
-
m said: Tagged ratings can be changed by anyone, just like the current system of Tagging "burial" etc.
There is no need for anyone to be afraid of misleading another patron/being misled by another patron by means of a Tag.
This is a collaborative website, edits can be made. Person pages regularly get edited to become more incorrect or more correct all the time.0 -
m said: "Proof" or "genealogical beliefs"---why are these words being used?
This rating system is about giving patrons a general idea of the quality of each source currently present on an ancestor's page.
That's it.
1 example:
Ancestor 1 has a Source that links to an article Tagged with "Birth" and Tagged with "A+" meaning the Original Image of a record.
Ancestor 1 also has a Source that links to an article Tagged with "Birth" "Burial" and Tagged with "B" meaning high quality in this case because it cites more than 1 record for a fact such as a birth/marriage/death for this individual and the article is a TAG article (The American Genealogist).
I look at Ancestor 1 and decide for birth I definitely need to click on and look at those 2 sources.
Ancestor 2 has a Source that links to Geni and us tagged with F meaning low quality source because does not cite at least 1 record for a fact such as a birth/marriage/death for this individual and that is the only Source attached.
I look at Ancestor 2 and decide this Ancestor needs some work because it lacks high quality Sources.
That's it.
No "proofs" or "genealogical beliefs" are involved.0 -
m said: Death Certificates.
The Source Quality Rating Tag System could have 5 levels or 15 levels of quality...however many the forum wants.
Death Certificates - the forum would make the levels and the forum would decide where Death Certificates belong on the scale.
Are they better than Geni? Place it higher. Are they worse? Place it lower. Better than TAG articles? Place it higher. Worse? Place it lower.0
This discussion has been closed.