Rating Source quality w/ Tags using letter grade to classify each Source. Objective criteria used. E

LegacyUser

m said: Rating Source quality with Tags using letter grade to classify and "rank" each Source according to the quality of the Source based upon objective criteria. Tag offers a letter grade for patron to choose from to rate the quality of the Source and the quality is spelled out so specifically that no one can misinterpret each grade. There is a Changelog where each patron has to write a Reasoning Statement. Patrons can change the Tag rating.

Classification system rating Source quality with Tags using letter grade based upon objective criteria:
1) improves collaboration,
2) helps experienced users see at a glance which Sources to concentrate on,
3) helps beginner users learn about differences in Source quality,
4) improves accuracy of each person page over time.

Classification system rating Source quality with Tags using letter grade based upon objective criteria:

A+ = "image &/or text of original record" -
contains either the image or the text of an original record mentioning this individual by name (including gravestone, obituary). = A+

A = "indexed record" -
indexed record mentioning this individual by name. = A

A- = "record with error(s)" -
indexed record OR image OR text of original record - contains either the image or the text of an original record OR indexed record mentioning this individual by name (including gravestone, obituary) AND contains at least 1 proven error = A-

B = "high quality" -
cites more than 1 record for a fact such as a birth/marriage/death for this individual OR quotes text of a record of this individual. = B

C = "medium quality" -
cites at least 1 record for a fact such as a birth/marriage/death for this individual. = C

D = "outdated but quality" -
mentions at least 1 record for a fact such as a birth/marriage/death for this individual, but source is outdated and several things have been disproven, lowering overall quality. = D

F = "low quality -
does not cite at least 1 record for a fact such as a birth/marriage/death for this individual. =F

! = "proven fraud" -
has been identified as a fraud by a credible professional genealogical source. = !

X = "not related to individual" -
no mention of the individual by name of the person it is attached to, or the name of one of his/her family members. = X

Please give suggestions to improve this Source quality rating system using Tags with letter grades based upon objective criteria.

GOAL FOR EACH LETTER GRADE: quality is spelled out so specifically that no one can misinterpret each grade.

LegacyUser

Adrian Bruce said: Classification has lots of merit. But I'm a little lost about what you are suggesting to classify here.

Your A and A+ could apply to the Historical Source Record itself - but B mentions "cites more than 1 record" - so that would appear to be applicable only to the Vital Event itself where multiple records could be cited for 1 event, but with just one rating for the Vital Event?????

LegacyUser

m said: 1 example:

Here is a Source that I have in the Source section of multiple ancestors and also in the Document of multiple ancestors. I'm placing the Document link (because the Source link at the original website is so long):

https://www.familysearch.org/photos/a...?
2019 version

https://www.familysearch.org/photos/a...?
2013 version

When you read this source, you see that it cites more than 1 record (it cites dozens of records for dozens of people related to birth/marriage/death). It also contains 2 images of original records related to birth/marriage/death.

If the person page is happens to be the person page that one of the original images applies to OR quotes text of a record for that exact person, you would select:

A+ - image of original record.

...and if the person page happens to be one in which more than 1 record is cited for that exact person for birth/marriage/death OR quotes text of a record for that exact person, you would select:

B = "high quality"

...and if the person page happens to be one in which only 1 record is cited for that exact person for birth/marriage/death, you would select:

C = "medium quality"

LegacyUser

m said: The general idea being, for example, that this Source contains 2 images of original records related to birth/marriage/death.

Images of original records related to birth/marriage death are the highest quality source, so for those few ancestors that the 2 images of original records apply to, those are of highest interest to patrons.

LegacyUser

Paul said: Sorry, but no - I'd rather make my own judgements. Presumably, FamilySearch would rate its record hints quite highly: but this leads to trusting, inexperienced users accepting poor suggestions without the verification required.

I've found some extremely good looking sources to attach to an ID: they look like perfect matches at first, then you realise they're for a cousin - same name, same age, same area.

Apart from when totally ridiculous mismatches are involved, I believe it is impossible to rate sources in this manner. The most convincing can often be totally off-target. It's up to each user to pass judgement based on checking and rechecking, not on what someone else finds thoroughly convincing. In fact, some of the most reliable of my inputs have no sources as back-up: none are available in FamilySearch, so I just give detailed reason statements (in Notes or against the Vitals) as evidence.

LegacyUser

m said: Sources for the wrong person should not be allowed to remain on a person page. They should be detached and possibly reattached to the correct person.

LegacyUser

m said: The above system would apply to Sources attached to the correct person.

Currently we have a Tagging system that allows us to Tag "birth/marriage/death" to each source.

So the above system would be the same except you Tag the quality of the source.

LegacyUser

Paul said: But how do you know a source has been attached to the correct ID? Say I have a William Fenwick who I know was born around 1720 in the county of Durham, England. The source attached for a 1721 christening in the area looks pretty convincing to me, and no doubt to others. Then I find sources for two other individuals of the same name, baptised close by: one in 1719, the other in 1722. What do I do - detach the 1721 source from "my" William Fenwick altogether? Reduce the "A" rating I had originally assigned to the source?

I genuinely have encountered situations like this, where only further, carefully researched evidence might enable some positive identification. No, I don't want to lull anyone in a "false sense of security" over my very convincing attachments - it's up to other users to examine and try to pull apart my conclusions, if they wish. Don't necessarily trust me - I've been undertaking this research for 35 years, but can still make incorrect judgements!

LegacyUser

Adrian Bruce said: Sorry but I still don't understand exactly where this rating is going to go in the proposed idea.

Is it supposed to go on the Historical Record for the source (if it's a FS source)?

Is it supposed to appear in the Sources tab of the profile for each person, and appear against the source that's being cited at that point?

Is it supposed to appear against the Vital Event referred to in the source?

Is it supposed to appear in the Vital event entry, against the source that's tagged against the vital event?

LegacyUser

m said: The website where that source comes from is:
https://sites.google.com/site/beyondo...

and the link to that document is on the home page in case anyone is wondering. It's a very long link.

LegacyUser

m said: This thread is not about "what to do in the rare instance you are not sure if a Source is attached to the wrong person or not."

The topic of this thread is the above system of rating Source quality.

(This thread assumes the Source you are applying this rating system to is attached to the correct person.)

LegacyUser

m said: Another example:

https://sites.google.com/site/beyondo...

The italics are the original text of a baptism record.

"Bautizado 9 febrero 1559, iglesia de Nuestra Castillo, Fuente Ovejuna, Spain, Juan, hijo de Juan Lopez de Villa Sana y de Isabel Ruiz."
Source: Iglesia de Nuestra Señora del Castillo, Fuente Ovejuna, Spain, Bautismos, Libro 2, f. 85v, 9 Feb 1559

So already this Source is looking to be high quality. What else does it have? There are some more italics and looking at one of them I can see that it is a quote from an Inquisition record in which his wife is named:

"...Catalina de Villanueva, 'muger de Capt. Juan López Olguín,' who provided the same testimony and gave her age as 'close to fifty years' (born circa 1576). Both were described as 'besinos fundadores de Santa Fe' ('founding vecinos of Santa Fe')."
Source: Archivo General de la Nación, Inquisición, tomo 356, f. 303.

So we have text of original records related to both marriage and birth.

And for both we have complete record citations. So this is a very high quality Source.

Patrons are very interested in both marriage and birth.

We would give it a high rating for collaboration purposes.

In the above system it would be B "high quality" because it cites more than 1 source for this exact individual related to birth/marriage/death OR quotes text from a record related to birth/marriage death for this exact individual.

[In other words, the original image of the baptism record itself would be rated as A+, the indexed record on FS of the baptism record would be rated as A if no errors or A- if errors, and then this is such a high quality Source that it comes in next with a B "high quality" rating.]

LegacyUser

m said: Adrian,

I was thinking it could go in a similar location to where the current system for Tagging Sources with "birth/death/burial" is.

But I suppose it depends on the Engineers or on where the Forum would want it to go.

LegacyUser

Adrian Bruce said: This explanation helps me understand a bit more about what you're trying to convey. However, the explanation seems to apply only to secondary sources that cite other sources, i.e. are compilations / conclusions.

I struggle to see its relevance to primary sources, or secondary sources that are not compilations / conclusions.

LegacyUser

m said: When you attach a Source it would be convenient to Tag it "burial" and say also Tag the quality as well. What if you attached an image of the original burial record? So Tag that "A+" highest quality possible---for collaboration purposes.

Patrons are very interested in burials.

LegacyUser

m said: The following section:

[In other words, the original image of the baptism record itself would be rated as A+, the indexed record on FS of the baptism record would be rated as A if no errors or A- if errors, and then this is such a high quality Source that it comes in next with a B "high quality" rating.]

..does not explain its relevance to primary sources or secondary sources that are not compilations / conclusions?

LegacyUser

Adrian Bruce said: That set of phrases seems fine - it's when the rest come in that I struggle.

Take an image of a parish register of a baptism. It contains "the image ... of an original record mentioning this individual by name ... = A+ "
So that's A+.

But how does "B = "high quality" - cites more than 1 record for a fact such as a birth/marriage/death for this individual ... " get into the picture? An image of a parish register doesn't cite *any* other records, so it doesn't rate B, or C, or...???

You can't just say, "Oh it's A+ so forget the rest" - that strongly suggests that we have at least two concepts being mixed up here.

The A+ rating should surely be applied at the level of the person within the Historical Record itself - which is the Historical Record in FamilySearch.

LegacyUser

m said: I'm not sure I understand the question. But the point is for us to hash out and create a Source quality system that

1) uses objective criteria
2) takes into account to every possible source
3) is so specifically spelled out that any beginner can use it

LegacyUser

m said: So the highest level will be A+ and I think that everyone can agree that should be when there is an image of an original record (whether or not text is also given).

LegacyUser

m said: I was thinking that FS indexed records should be A or A- (A- being if there is an error----we all know indexed records with errors).

I think everyone can agree that FS indexed records have to be high on the list.

LegacyUser

Adrian Bruce said: I think one of my concerns is that the proposal conflates at least two concepts and, because of this conflation perhaps, has to apply them in places where duplicate work could result.

1. Whether the source "contains either the image or the text of an original record" is important. That can be assessed with little possibility of dissent. Further, it is a property of the source-record-entity itself, can be entered once, and should be stored with the source-record-entity for the benefit of everyone.

2. Whether the source mentions "this individual by name" is a different concept - that is only relevant to the combination of person-profile-entity and source-record-entity - basically at the point where, on a person's profile, a source-record is attached to a person-profile. Whether the source mentions "this individual by name" is something that can be debated - it might contain a "John Doe" but does it contain this "John Doe"?

3. The A+/A/A- ratings appear to refer (mostly?) to original documents, whereas B thru F seem only to refer to compilations (what Elizabeth Shown Mills refers to as Authored Narratives). That suggests to me that we have two distinct assessment schemes that have been pushed into one. Things get problematic perhaps when we look at a death certificate that records someone's date of birth. No sources are quoted for that, of course - so is it an F ("does not cite at least 1 record for a fact such as a birth/marriage/death for this individual") or an A+ ("contains either the image or the text of an original record")?

And of course the relative quality of date-of-death information and date-of-birth information on a death certificate is well known to be radically different ,requiring different source-quality values - which then pushes the source-quality-rating to be held against the 3-fold combination of vital-event, person-profile-entity and source-record-entity.

LegacyUser

m said: Then I was thinking that next (B) would come very very high quality sources that aren't FS sources and aren't images of original records....do you get my drift?

...and that everyone would be in agreement with that.

LegacyUser

m said: Great. Tweak the system and improve it.

The point is for us to hash out and create a Source quality system that

1) uses objective criteria
2) takes into account to every possible source
3) is so extremely specifically spelled out that any beginner can use it

It's about collaboration and so that patrons can "eyeball" Source quality in the same way they "eyeball" say a burial Tag.

LegacyUser

Adrian Bruce said: I would strongly recommend that, rather than FS creating its own scheme of things, it takes a look at Elizabeth Shown Mills' Evidence Analysis Process Map (see https://www.evidenceexplained.com/con... ) where there are what I would describe as 3 dimensions, that would be held as 3 different items to avoid conflation of concepts:

1. Source-records are classed as Original Record / Derivative Record / Authored Narrative (Derivate refers to various copy / extract processes)

This would appear to be held once per source-record if implemented.

2. The Information within the source-record is then assessed whether it is Primary knowledge (first-hand), Secondary knowledge (second-hand) or Unknown.

That is more complex because the information within the source-record can be a mix of information depending on which bit of the source we're looking at - the classic example being the information on a death-certificate is usually primary for the date of death and secondary for the actual or implied date of birth.

I think that this would be held against the 3-fold combination of vital-event, person-profile-entity and source-record-entity to get round the mix issue.

3. The Evidence - is it Direct? Indirect? Negative?

I think that this would also be held against the 3-fold combination of vital-event, person-profile-entity and source-record-entity, but as a separate item from 2.

I don't have enough experience of applying ESM's concepts to know whether they would satisfy M's quality ratings for Authored Narratives. And bear in mind as well that Original Documents showing Primary Knowledge and Direct Evidence can be plain simple wrong - so maybe that is a 4th dimension?

LegacyUser

m said: So the highest level will be A+ and I think that everyone can agree that should be when there is an image of an original record (whether or not text is also given).

I was thinking that FS indexed records should be A or A- (A- being if there is an error----we all know FS indexed records with errors).

(I think everyone can agree that FS indexed records have to be high on the list.)

Then I was thinking that next (B) would come very very high quality sources that aren't FS sources and aren't images of original records....do you get my drift?

(...and that everyone would be in agreement with that.)

These categories would have the stuff patrons are highly interested in.

LegacyUser

Adrian Bruce said: "This thread assumes the Source you are applying this rating system to is attached to the correct person" But that's missing a hugely important issue - the whole combination of source-record / person created through inadequately assessed record hints.

LegacyUser

m said: If someone has attached a source for the wrong person, you just detach the source.

Am I missing something?

LegacyUser

m said: Hearsay would go under:

F = "low quality -
does not cite at least 1 record for a fact such as a birth/marriage/death for this individual. =F

Death certificate might go under:

F = "low quality -
does not cite at least 1 record for a fact such as a birth/marriage/death for this individual.

And maybe the wording of F should be changed to:

F = "low quality -
does not cite at least 1 record for a fact such as a birth/marriage/death for this individual. Includes Death Certificate.

OR possibly under C and change wording to:

C = "medium quality" -
cites at least 1 record for a fact such as a birth/marriage/death for this individual. Includes Death Certificate = C

Great. Tweak the system and improve it.

The point is for us to hash out and create a Source quality system that

1) uses objective criteria
2) takes into account to every possible source
3) is so extremely specifically spelled out that any beginner can use it

It's about collaboration and so that patrons can "eyeball" Source quality in the same way they "eyeball" say a burial Tag.

LegacyUser

Juli said: I would rate indexed sources at about C- ("passing, but barely"), in general, unless verified with a connected image, in which case I'd cite the image and the rating of the index would be moot.

Also, the "quality" (trustworthiness? accuracy?) of a source document is completely independent of the quality of the _attachment_.

For example, I recently learned that my great-grandfather had another sister who emigrated to Chicago. She married back in Burgenland, so she's in most U.S. records with her married name: Karolina Gamauf. There was another Karolina Gamauf who emigrated from Austria to Chicago; she arrived about a decade later, and was half a year younger. The other Karolina was naturalized in 1943, while "mine" was naturalized in 1944. The one clear difference is that the other Karolina was unmarried. Images of the original ship manifests are available for both arrivals. They have some minor problems -- placenames slightly misspelled, ditto marks unclear as to scope, that sort of thing -- but as sources go, they're about as good as you can get: definitely in the "A" range on your scale. But attaching the unmarried Karolina's arrival to the married one would still be completely wrong, rating an F- by any definition.

So echoing Paul's and Adrian's confusion: which part are you trying to rate, the sources, or the profiles?

LegacyUser

m said: I'm trying to rate the Sources.

How would you make your ranking list?

The point is to come up with a system so each patron can grasp (in general) the quality of a source.

So the forum would have to tweak and tweak the system until we got it just right.

LegacyUser

m said: Also, I don't understand the focus on Sources attached to the wrong person.

If someone has attached a source for the wrong person, you just detach the source and possibly attach it to the correct person.

Am I missing something?