DATE: problem with GEDCOM 7.x definition for BEF and AFT

[albertemmerich]

in our German group we have a discussion about the DATE payloads with BEF and AFT.
GEDCOM 5.5.1 defined:
AFT = Event happened after the given date.
BEF = Event happened before the given date.
This is interpreted by most users as the given date not be included.

Now GEDCOM 7.0 has modified the definition to:
BEF x = Exact date unknown, but no later than x.
This includes x.

Example:
GEDCOM 5.5.1
2 DATE BEF 1899
was interpreted as on any date up to 31 DEC 1898, but not on 1 JAN 1899 or later.
GEDCOM 7.0
2 DATE BEF 1899
is defined to be: on any date up to 31 DEC 1899 (included), and not on 1 JAN 1900 or later

How to handle this when converting GEDCOM 5.5.1 to GEDCOM 7.0 ? To have the same interpretation in GEDCOM 7, we have to write in GEDCOM 7.0:
2 DATE BEF 1898
decreasing the year by 1??

If one application does that, another does not, "same" data are represented in GEDCOM 7.0 in different ways.

Questions:

1. How should any conversion 5.5.1 to 7.0 handle this?
2. Is there any recommendation, how an application should show the data to the user?

For 2. we started to discuss using "<" for "BEF" from GEDCOM 5.5.1; and "<=" for "BEF" from GEDCOM 7.0 ....

Albert (Emmerich)

[dthaler]

Example: GEDCOM 5.5.1 2 DATE BEF 1899 was interpreted as on any date up to 31 DEC 1898, but not on 1 JAN 1899 or later.

My view (and my implementation) is that such an interpretation is not safe.
I've seen many examples of gedcom files that use "BEF 1899" when there is some event (month and day unknown) in 1899 at which the specified event was known to occur before. For example, if a land sale occurred in 1899 (month and day unknown) in which both the husband and the wife signed as grantors, then the marriage was entered as "BEF 1899". However in reality the marriage could have occurred in 1899, earlier in the year than the land sale. Hence it is only safe to interpret BEF as including the date when the month+day is not specified. Another way to interpret it is conservatively that "BEF 1899" is equivalent to "BEF 31 DEC 1899" (as opposed to equivalent to "BEF 1 JAN 1899").

[kiwi3685]

This change is pointless. It will never solve the problems of mis-use or mis-interpretation. It's more likely to make it worse.
You seem to forget that 99% (at least) of GEDCOM users will never read the "rules" let alone understand them.
All this change achieves is further complexity to very simple before / after terminology.

If you need to add further clarification to a BEF or AFT date just add a note.

[brezelman]

As a user, I always interpret BEF and AFT as including dates. Since these specifications cannot be approximate (cannot be combined with ABT/CAL/EST), they are usually specified as a conclusion based on the date of another event. As a user, I then use the exact date of the original event as the BEF or AFT to indicate the appropriate dependency. I can't imagine that the majority of users would then always add or subtract one to indicate the dependent date exclusively, nor would I hope that the software would do that. I would thus agree with @dthaler that the interpretation of the German GEDCOM group is probably not what is present in the majority of existing GEDCOM files and that the definition now established is the more reasonable one.

[Norwegian-Sardines]

I’ve alway used BEF/AFT to not include the date provided. Maybe it is a language use or some custom in a location, but BEF/AFT has never been inclusive of the date, particularly if in the future a TIME is included in the use of a DATE. This is particularly important when building a time line of events.

Saying the following makes no sense to me: “ that the definition now established is the more reasonable one.”. NO, It muddies the water more.

Like so many times in GEDCOM the designers try to not make a stand as to the meaning or definition of a word or tag, and leave it up to the user/application designer to decide the use. This attitude causes interfacing between two programs impossible when the program’s come up with their own interpretation. Interfacing is why GEDCOM is valuable, and not being precise in a definition defeats that value.

The definition should have been made more precise, for example:
Exact date unknown, but no later than x. Including x
Or
Exact date unknown, but no later than x. Excluding x.

I would thus agree with @dthaler that the interpretation of the German GEDCOM group is probably not what is present in the majority of existing GEDCOM files and that the definition now established is the more reasonable one.

Without surveying all applications I would not make this assumption.

Some software makes no designation of difference between FROM/TO and BET/AND. There is a difference and it messes with interpretation of the user base.

[Norwegian-Sardines]

Example: GEDCOM 5.5.1 2 DATE BEF 1899 was interpreted as on any date up to 31 DEC 1898, but not on 1 JAN 1899 or later.

My view (and my implementation) is that such an interpretation is not safe. I've seen many examples of gedcom files that use "BEF 1899" when there is some event (month and day unknown) in 1899 at which the specified event was known to occur before. For example, if a land sale occurred in 1899 (month and day unknown) in which both the husband and the wife signed as grantors, then the marriage was entered as "BEF 1899". However in reality the marriage could have occurred in 1899, earlier in the year than the land sale. Hence it is only safe to interpret BEF as including the date when the month+day is not specified. Another way to interpret it is conservatively that "BEF 1899" is equivalent to "BEF 31 DEC 1899" (as opposed to equivalent to "BEF 1 JAN 1899").

Not all “BEF” or “AFT” dates are only a year. If the date-value is always interpreted to be “inclusive” of the date-value then when the date-value is precise we can have a problem.

When creating a time line of events imprecise dates must be given a “sort value” and as you describe “BEF 1899” would logically have “BEF 1 JAN 1899” so than a date of “1 JAN 1899” sorts after it.

Therefore, using a year only example seams to make sense, when more precise dates are use, including the date-value in a “BEF” is not correct.

[Norwegian-Sardines]

This change is pointless. It will never solve the problems of mis-use or mis-interpretation. It's more likely to make it worse. You seem to forget that 99% (at least) of GEDCOM users will never read the "rules" let alone understand them. All this change achieves is further complexity to very simple before / after terminology.

If you need to add further clarification to a BEF or AFT date just add a note.

Many of the developers of software have not even understood the “rules”.

If you need additional clarification of the BEF date use the DATE.PHRASE

2 MARR
2 DATE BEF 1899
3 PHRASE Before the couple purchased the land, exact date unknown.

[giotodibondone]

Gramps forum: Misleading date range labels? (Before, After, Between)

[Norwegian-Sardines]

Thanks for the inclusion of this discussion! I think the new definition should be changed back to the old wording. If I said “be at my house before Monday, or I’ll leave without you”, and you showed up on Monday, it is a good chance i would be gone! “Before” should always mean “not on”!

[fisharebest]

I think we should ask which is more useful, rather than which is more "correct".

If an individual was baptised on 28 MAY 1800, you want to be able to say they were born before then - e.g. BEF 28 MAY 1800.

But since they could have been born and baptised on the same day, it needs to be inclusive.

[Norwegian-Sardines]

I agree that they could have been born and baptized on the same day! However, since you don’t actually know what date the person was born on, the accuracy of the birth date is already in dispute!

I could use the before date or more likely a phrase without a date and still make the point that they were born before they were baptized.

‘
1 BIRT
2 DATE
3 PHRASE Unknown birth date, but on or before their baptism of 28 MAY 1800.

If you just have to have a real date, you could also say BEF 29 MAY 1800 and also be correct!

[webtrees-pesz]

I completely agree with Ken.

The change in the meaning of the BEFORE addition in the date specification from GEDCOM 5.5.1 (happened before the given date) to GEDCOM 7.x (happened not later than the given date) was, in my opinion, a serious mistake that distorts date specifications from earlier versions!

BEFORE means “before” and not “on or before.”

If this information is required in the future, then a new suffix should be created for it.

Suggestion: ONBEF
or
Convert all date additions to technically precise terms such as LE (<=), GE (>=) LT (<), GT (>), RANGE ... ..., (FROM ... TO ...) PERIOD ... ... (BET ... AND ...).

[tychonievich]

Creating new tags is possible as part of 7.1. I'm not immediately seeing where we'd have the sources to need both LE and LT, but I'd be open to being proven wrong.

Changing the meaning of BEF to mean "before the beginning of" instead of "before the end of" would be a breaking change, and as such not possible before 8.0.

[tychonievich]

As the person who wrote the current 7.0 wording, here is why it was written as it is. I'm not claiming we made only correct decisions, but I will try to reveal the decisions we made and why we made them.

5.5.1's text was "BEF = Event happened before the given date."

That text is ambiguous: does it mean before the beginning of the given date or before the end of it? This is especially significant when the date itself is imprecise. Does BEF 1865 mean "< 1865-01-01 00:00:00" or "< 1865-12-25 23:59:59" or something in between?

We looked to see if we could find consistency of interpretation in the wild. It didn't take long to find .ged files that had things like a source dated 1865 with text that referred to someone as deceased, and where that was the source of a DEAT.DATE BEF 1865; and other files with similar courses going with of a DEAT.DATE BEF 1866 instead. Existing consensus could thus be ruled out. Nor were these isolated examples; searching through the files various members of that team had available for DATE BEF, DATE AFT, and DATE BET and checking their sources showed many instances of each interpretation.

If we were to remove the ambiguity, we were thus forced to pick something that did not align with the intent of many existing files. Two options were available: we could either cause people who interpreted it as "before the beginning/after the end" to find their data in 7.0 was not quite as precise as they had intended, or we could cause people who interpreted it as "before the end/after the beginning" to find their data in 7.0 was now more strict and possibly asserting falsehoods. This left three options

1. Leave them ambiguous
2. Clarify them in a way that lost precision in some files
3. Clarify them in a way that changed some files to assert falsehoods

Option 3 was clearly out: at first glance that "clarification" was the most natural interpretation of the words in 5.5.1, but we had ample evidence that it was a more strict interpretation that many extant .ged files assumed and hence would amount to a breaking change.

After multiple rounds of discussion, we decided that option 2 was superior to option 1 because it did not propagate past ambiguity into future versions of GEDCOM.

[Norwegian-Sardines]

That text is ambiguous: does it mean before the beginning of the given date or before the end of it? This is especially significant when the date itself is imprecise. Does BEF 1865 mean "< 1865-01-01 00:00:00" or "< 1865-12-25 23:59:59" or something in between?

I don’t understand this question/issue! How can anyone think that before 1865 means it still happened in 1865? DEC 1865 is not before 1865! Just like my example “be at my house before Monday” does not mean on Monday! How do we interpret “Before he entered the service he was a shoe maker”!

We looked to see if we could find consistency of interpretation in the wild. It didn't take long to find .ged files that had things like a source dated 1865 with text that referred to someone as deceased, and where that was the source of a DEAT.DATE BEF 1865; and other files with similar courses going with of a DEAT.DATE BEF 1866 instead.

Are you saying that two different transmissions of the exact same individual with the same source artifact could not agree on the “before” year? Did the source artifact just indicate the individual died in 1865? If so then the BEF directive should not be used at all. If the source artifact did not indicate the death but by inference suggested the person was already dead, was the artifact itself only dated with the year of 1865, then we can’t infer that the death happened in 1865 but, about or before 1865 but not before 1866 because that would include 1865! Using an imprecise date to infer another date makes that date more imprecise!

I get what Greg is saying that we don’t have a way to include a given day or before in the current design (others have said this too), but we are talking about using dates from a known event to estimate an unknown date. It is an estimate that says “I have no proof or record of an exact date, so I’m saying it happened before (or more likely had already happened by) something I know.” This is a good use of the PHRASE!

Maybe adding a new indicator of HBY (Happened BY) or OBY (occurred by) when we see a source that references something that was already in place but when another dated event occurred, such as “Jane was born on 1 MAY 1880 at the Benton family farm”. And this is the earliest source referencing the Benton family farm, so we could say:

1 PROP Benton Family Farm
2 DATE HBY 1 MAY 1880

THE BOTTOM LINE, Do whatever you think is right, I’m done with this line of reasoning, GEDCOM has set their understanding of the word “before” to be inclusive of the date given and I’ll live with it!

[tychonievich]

Are you saying that two different transmissions of the exact same individual with the same source artifact could not agree on the “before” year?

No, just similar source data resulting in different recording. I conducted a web search for filetype:.ged "DATE BEF and read through dozens of such files looking for ones with sources; some other members of the drafting group looked at files that had been uploaded by users of their tools. Many such files didn't have sources, but those that did often had the general form of (a) source asserts some status is/is not the case on a given date; (b) associated GEDCOM data shows the event that starts/ends that status with a date BEF/AFT the date in the source (the inclusive version interpretation) or BEF/AFT a date one off from the date in the source (the exclusive version).

[Norwegian-Sardines]

Thanks for the answer and process!
Here is my take, Because the source artifact specifies a date for one fact/event (1 DEC 1880), but not explicitly specifies the date for another, any dates inferred by the source artifact but not specified for any other fact/event are “estimates”, unsupported dates, and therefore should not hold the same weight as one with real evidence.

Therefore since the unsupported fact is only an estimate of when the fact/event occurred making the formerly unspecific BEF and AFT arguments specific, make little sense to me! Yes the unspecified fact/event could have occurred on the same day as in Greg’s example, but since we dont have any evidence (we are just guessing), any use of that information is less than valuable. This gets even more complicated when the source artifact does not specify an exact date for the fact of the source but just a month/year or only a year. These are already inexact estimates to begin with and have less value for estimating any other date for an unspecified fact/event. Logic would indicate that we should not try to use an estimated date to infer another estimated date, using transmissions that use illogical processes to formulate a conclusion takes us down an unsound path.

This is where I am at a loss for understanding how people interpret and use source artifacts. But as I said, if the committee likes this interpretation and want to reinforce the word BEFore and AFTer to be inclusive of the date value, then so be it. The confusion will only continue and some with reject the definition.