Changes

Jump to: navigation, search

GEPS 043: Improving GEDCOM support for Places

1,788 bytes added, 16:45, 14 June 2019
no edit summary
Related Bugs
* {{Bug|688}} (support GedcomELGEDCOM 5.5EL) <== The big one
* {{Bug|10160}} (export of Place with ADDR)
* {{Bug|8699}} (Blank Place when only ADDR is present)
References:
* The Gedcom GEDCOM L PLAC/_LOC agreements ( http://wiki-en.genealogy.net/GEDCOM/PLAC-Tag )
* A list of place types we might want to consider incorporating ( http://gov.genealogy.net/type/list )
* Detecting GEDCOM 5.5EL ( https://www.tamurajones.net/DetectingGEDCOM5.5EL.xhtml )
Goal: avoid data loss
Note: This has been updated in anticipation of the [[GEPS 045: Place Model Enhancements]], we now assume that we can store much more of this into the Gramps enhanced places. Since it is not easily possible to separate the two GEPS, the commits for this GEPS is included in the GEPS045 work! == Gedcom GEDCOM Import ==
=== Current situation ===
Place Name and Title fields are always stored the same unless otherwise noted.
=== Proposals ===
#) Support Gedcom GEDCOM EL extensions for PLAC._LOC, and a few others related to places.#) Use only one mechanism for ADDR records. Do away with Place, type 'Details' and, Location Object(place Alt Location).#* It has been suggested that Gramps developers want to deprecate the Location (Alt Location) structure of our place object. In this case the proposal is to convert stand-alone event ADDR to places with Type 'Address'. Events that also have the PLAC tag would create the a place with Type 'Address' from PLAC ADDR data, and add an alternate name with the ADDR then create a second (enclosing) place from PLAC data. In either case, the ADDR data would be converted to a single line of comma separated elements (if not already available). This could result in some data loss if the ADDR record is not consistent. Also, if both the PLAC and ADDR records had postal codes, the PLAC code would end up in the Gramps place 'code' field, and the ADDR postal code would be on the end of the alt-name.#) Don't force fit PLAC.FORM records to Location objects, just make appropriate places with type as value from the FORM. Enclosed by hierarchy defined by Gedcom L FORMGEDCOM 5.5. Any place types in 1 FORM list after 'country' would not be part of hierarchy. Gedcom L actually says to use first 4 levels, { place, county/district, state, country} but I've got a few files where leading levels are {address, street, city,…}. Postal codes are expected after 'country' level and don't participate in enclosed by hierarchy, but should be recognized if at another position. Have found another file with {2 FORM Town, Area code, County, Region, Country, Subdivision} for example, which needs to work.
#: The following Example would create a Gramps place, Named '123 Main', type 'Street' with code '77375' enclosed by
#: another Gramps place, Named 'Houston', type 'City' enclosed by
<nowiki> 1 RESI
2 PLAC 123 Main, Houston, Harris Co., Texas, USA, 77375
3 FORM Street, City, County, State, Country, Postal Code</nowiki>  
=== Issues ===
* Location xref (_LOC.TYPE) structure allows multiple record for place type seems likely to allow an enormous list of types. Far more than the current Gramps PlaceType list. We import them as Custom Gramps types, with a date but that has poor results for eachinternationalization. Gramps The GEPS045 work allows only oneplace types to be renamed, and also shows the original GOV place type number as part of the Preferences/Place Types configuration panel.* Location xref (_LOC) structure allows multiple postal codes, with a date for each. Gramps allows only onenow supports Place Attributes where we can store the postal code. The Attributes DO NOT support a date, with no so a date, if any, is added to the attribute value (Example: "12345; Date: 2011-04-05").* Gramps has no direct support of ABBR_GOV, since the GetGOV Gramplet has decided to store the GOV ID in the Gramps ID field of a place, ABBRthe _GOV entry will do the same.TYPE, _FPOST, _FPOST* The _DMGD tag also has a _DMGD.DATE, _POSTand _MDGD.TYPE fields.DATE The associated demographic data will be placed in an Attribute value, _GOVwith the date and type, _FSTAEif any, _FCTRY, _MAIDENHEAD, _LOC.TYPE added to the attribute value (Hierarchical relationshipExample: "12345; Date: 2011-04-05; Type:Population"), _DMGD, _DMGD.DATE, * The _AIDN, tag also has a _AIDN.DATE, and _AIDN.TYPEfields. These are proposed to The associated Administrative data will be stored placed in a Note.* Gramps does not support Event references from Place objects. These are proposed an Attribute value, with the date and type, if any, added to be stored as xrefs in a StyledText Notethe attribute value (Example: "County Precinct 4; Date: 2011-04-05; Type:Precinct").
* The potential exists for place names defined in ADDR, PLAC, and _LOC records to be different. If this occurs, then multiple Gramps alt place names will be set for the place. The primary name will be set from the _LOC record.
* The potential exists for certain items to be defined both in the PLAC and _LOC records (where the Gedcom GEDCOM L specification says they should be in the _LOC record). If this occurs, both the PLAC.FORM and the _LOC versions of hierarchy will take prioritybe imported into the Gramps place
=== Details ===
In embedded PLAC records the following is in the Gedcom GEDCOM L standard. This proposal suggests that items labeled 'new' be imported by Gramps.
<nowiki>n PLAC <PLACE_NAME> {1:1}
+1 FORM <PLACE_HIERARCHY> {0:1}
+1 FONE <PLACE_PHONETIC_VARIATION> {0:M} New Place Alt Name(1,2) Note+2 TYPE <PHONETIC_TYPE> {1:1} New Place Alt Name Language(1,2) Note+1 ROMN <PLACE_ROMANIZED_VARIATION> {0:M} New Place Alt Name(1,2) Note+2 TYPE <ROMANIZED_TYPE> {1:1} New Place Alt Name Language(1,2) Note
+1 MAP {0:1}
+2 LATI <PLACE_LATITUDE> {1:1}
+1 <<NOTE_STRUCTURE>> {0:M}
+1 _LOC @<XREF:_LOC>@ New cross ref to _LOC
+1 _GOV <GOV_IDENTIFIER> {0:1} New (1, 2) NoteStored in Gramps ID for the place+1 _POST <POSTAL_CODE> {0:M} New (2)Place Code fieldAttribute, type 'Postal'+2 DATE <DATE_VALUE> {0:1} New (1,2) Noteappended to place Attribute value+1 _FPOST <FOKO_POSTCODE> {0:M} New (1,2) Notealready deprecated, not supported+1 _MAIDENHEAD <MAIDENHEAD_LOCATOR> {0:1} New (1Place Attribute,2) Notetype 'Maidenhead Locator'+1 _FSTAE <FOKO_TERRITORY_IDENTIFIER> {0:1} New (1,2) Notealready deprecated, not supported+1 _FCTRY <FOKO_STATE_IDENTIFIER> {0:1} New (1,2) Notealready deprecated, not supported</nowiki>
1) Since Gramps Places don't have Attributes, I propose that these items be placed in a single Gramps note (one per place (or place reference in the Event)). The note will have the various Gedcom GEDCOM fields and the contents included.
2) If the _LOC is present, these are also expected in the _LOC structure. Between both structures and in some cases multiple copies, and a duplicate avoidance strategy is used. If the sub-record is the same as one already present in the note, it is dropped. If different, it will be ignored hereadded.
The cross referenced _LOC record is defined below and is entirely new. While these are expected at the end of the Gedcom GEDCOM file per the EL agreement, Gramps can support them at any position.
<nowiki>0 @<XREF:_LOC>@ _LOC
1 NAME <PLACE_NAME> {1:M} Place Name
2 DATE <DATE_VALUE> {0:1} Place PlaceName Date
2 _NAMC <PLACE_NAME_ADDITION> {0:1} Place Name Note (31)2 ABBR <ABBREVIATION_OF_NAME> {0:M} Note (1)Place PlaceName PlaceAbbrev3 TYPE <TYPE_OF_ABBREVIATION> {0:1} Note (1)Place PlaceName PlaceAbbrev PlaceAbbrevType
2 LANG <LANGUAGE_ID> {0:1} Place PlaceName Language
2 <<SOURCE_CITATION>> {0:M} Place PlaceName Citation (4)1 TYPE <TYPE_OF_LOCATION> {0:M} Place Type (2)PlaceType2 DATE <DATE_VALUE> {0:1} Place PlaceName PlaceType Date (2)2 <<SOURCE_CITATION>> {0:M} Place PlaceType Citation (4)1 _FPOST <FOKO_POSTCODE> {0:M} Note (1)Deprecated, not supported2 DATE <DATE_VALUE> {0:1}} Note (1)Deprecated, not supported1 _POST <POSTAL_CODE> {0:M} Place Code field(6)PlaceAttribute2 DATE <DATE_VALUE> {0:1} Note (1)appended to Place PlaceAttribute Value2 <<SOURCE_CITATION>> {0:M} Place PlaceAttribute Citation (4)1 _GOV <GOV_IDENTIFIER> {0:1} Note (1)Stored in Gramps ID for the place1 _FSTAE <FOKO_TERRITORY_IDENTIFIER> {0:1} Note (1)Deprecated, not supported1 _FCTRY <FOKO_STATE_IDENTIFIER> {0:1} Note (1)Deprecated, not supported
1 MAP {0:1}
2 LATI <PLACE_LATITUDE> {1:1} Place Latitude
2 LONG <PLACE_LONGITUDE> {1:1} Place Longitude
1 _MAIDENHEAD <MAIDENHEAD_LOCATOR> {0:1} Note (1)Place PlaceAttribute1 EVEN [<EVENT_DESCRIPTOR>|<NULL>] {0:M} Event (5)Place EventRef2 <<EVENT_DETAIL>> {0:1} Event (5)1 _LOC @<XREF:_LOC>@ 0:M Place Reference PlaceRef (Enclosed)2 TYPE <HIERARCHICAL_RELATIONSHIP> {1:1} Note (1)Place PlaceRef PlaceHierType2 DATE <DATE_VALUE> {0:1} Place Reference PlaceRef Date2 <<SOURCE_CITATION>> {0:M} Place PlaceRef Citation (4)1 _DMGD <DEMOGRAPHICAL_DATA> {0:M} Note (1)Place PlaceAttribute2 DATE <DATE_VALUE> {0:1} Note (appended to Place PlaceAttribute Value2 TYPE <TYPE_OF_DEMOGRAPICAL_DATA> 1:1) appended to Place PlaceAttribute Value2 <<SOURCE_CITATION>> {0:M} Place PlaceAttribute Citation (4)2 TYPE <TYPE_OF_DEMOGRAPICAL_DATA> 1:1 Note (1)1 _AIDN <ADMINISTRATIVE_IDENTIFIER> {0:M} Note (1)Place PlaceAttribute2 DATE <DATE_VALUE> {0:1} Note (appended to Place PlaceAttribute Value2 TYPE <TYPE_OF_ADMINISTRATIVE_IDENTIFIER> {1:1)} appended to Place PlaceAttribute Value2 <<SOURCE_CITATION>> {0:M} Place PlaceAttribute Citation (4)2 TYPE <TYPE_OF_ADMINISTRATIVE_IDENTIFIER> {1:1} Note (1)
1 <<MULTIMEDIA_LINK>> {0:M}
1 <<NOTE_STRUCTURE>> {0:M}
1 <<CHANGE_DATE>> {0:1}</nowiki>
1) Since Gramps Places don't have Attributes, I propose that these items be placed in a single Gramps note (one per place). The note will have the various Gedcom GEDCOM fields and the contents included. When there are multiple possible similar notes (place name notes, enclosed by notes) the associated place Name or enclosed Gramps_ID (as a styledtext StyledText xref) will be included in the local heading.
2)Gramps does not support multiple place types. If more than one type is found, the TYPE and TYPE.DATE will be put in the Note. 3) Place Name Addition. Prepend to place name on import. 4) The citation will be directly on the Place, however, the note will have a heading and citation xref. 5) The event will be created as normal, and the note will have a styledtext xref to it. This is somewhat dangerous in that these events will appear to be 'unused objects' in a 'Check and Repair' or 'Unused Object' scan, and might be deleted. Also if filters are used, these events might not be included in desired output. 6) If more than one Postal code is encountered only the first is stored in the Gramps place code field. Additional encounters are stored in Note. == Gedcom GEDCOM Export ==
=== Current situation ===
Addresses attached to submitter, persons, repos, etc. are exported as ADDR record.
Events are exported with BOTH PLAC and ADDR records. The ADDR record uses CITY, STAE, CTRY sub records to build a place hierarchy.
Places export with the PLAC.NAME filled in with the Gramps Title field, the contents depending on the Preferences/Places/Automatic Place Title setting.
Places with 'code' export this in the ADDR structure.
The ADDR sub record (CITY, STATe etcPlaces with Alt-Locations; the Alt-Locations are NOT exported.) is filled out if with enclosed by places with exactly matching place types These are expected to be deprecated soon.
=== Issues ===
* ADDR structure is not legal GedcomGEDCOM. No first line.
* PLAC.NAME etc. should not depend on automatic title generation setting.
* While exporting ADDR and PLAC is arguably legal, it is redundant.
* No attempt to export place alt name, enclosure, place type, date, language, URL, citations, media information.
=== Proposal ===
# Support PLAC.FORM. Use the Gramps enclosure information to create full comma separated PLAC.NAME and PLAC.FORM filled out from the Gramps place type. Postal Code from place code field attribute at end of list when present. PLAC.FORM will be included with each PLAC, NOT in header, since Gramps is not guaranteed to have consistent place types.# Support PLAC._LOC and _LOC xref. This, while not standard Gedcom GEDCOM is defined by Gedcom GEDCOM L and is legal under standard GedcomGEDCOM. So it should be safe for other programs to import, even if they don't support it. This can encode several Gramps structures as detailed below.# Drop export Export places of Type Address as an ADDR record.# Place names for export no longer depend on preferences.  
=== Details ===
The following would be exported at each Event. We include PLAC.FORM for compatibility, and also include PLAC._LOC for more complete export.
+2 LONG <PLACE_LONGITUDE> {1:1} Place Longitude
+1 _LOC @<XREF:_LOC>@ cross ref (Gramps place ID)
+1 _GOV <GOV_IDENTIFIER> {0:1} Exported if Gramps ID looks like a GOV ID
+1 <<NOTE_STRUCTURE>> {0:M} When notes are present</nowiki>
<nowiki>
0 @<XREF:_LOC>@ _LOC Gramps place ID
1 NAME <PLACE_NAME> {1:M} Place Name PlaceName (1)
2 DATE <DATE_VALUE> {0:1} Place PlaceName Date
2 _NAMC <PLACE_NAME_ADDITION> {0:1} Not used
2 ABBR <ABBREVIATION_OF_NAME> {0:M} Not usedPlace PlaceName PlaceAbbrev3 TYPE <TYPE_OF_ABBREVIATION> {0:1} Not usedPlace PlaceName PlaceAbbrev PlaceAbbrevType
2 LANG <LANGUAGE_ID> {0:1} Place PlaceName Language
2 <<SOURCE_CITATION>> {0:M} Not usedPlace PlaceName Citation1 TYPE <TYPE_OF_LOCATION> {0:M} Place TypePlaceType2 DATE <DATE_VALUE> {0:1} Place PlaceName PlaceType Date2 <<SOURCE_CITATION>> {0:M} Not used1 _FPOST <FOKO_POSTCODE> {0:M} Not usedPlace PlaceType Citation
2 DATE <DATE_VALUE> {0:1}} Not used
1 _POST <POSTAL_CODE> {0:M} Place codeAttribute of AttributeType:POSTAL2 DATE <DATE_VALUE> {0:1} Not usedextracted from Place Attribute Value if present2 <<SOURCE_CITATION>> {0:M} Not usedPlace Attribute Citation1 _GOV <GOV_IDENTIFIER> {0:1} Not used1 _FSTAE <FOKO_TERRITORY_IDENTIFIER> {0:1} Not used1 _FCTRY <FOKO_STATE_IDENTIFIER> {0:1} Not usedExported if Gramps ID looks like a GOV ID
1 MAP {0:1} When LAT/LON are present
2 LATI <PLACE_LATITUDE> {1:1} Place Latitude
2 LONG <PLACE_LONGITUDE> {1:1} Place Longitude
1 _MAIDENHEAD <MAIDENHEAD_LOCATOR> {0:1} Not usedPlace Attribute of AttributeType:MAIDN1 EVEN [<EVENT_DESCRIPTOR>|<NULL>] {0:M} Not usedPlace EventRef2 <<EVENT_DETAIL>> {0:1} Not usedEvent
1 _LOC @<XREF:_LOC>@ 0:M Enclosed Place Reference (2)
2 TYPE <HIERARCHICAL_RELATIONSHIP> {1:1} Not usedPlace PlaceRef PlaceHierType2 DATE <DATE_VALUE> {0:1} Place Reference PlaceRef Date2 <<SOURCE_CITATION>> {0:M} Not usedPlace PlaceRef Citation1 _DMGD <DEMOGRAPHICAL_DATA> {0:M} Not usedPlace Attribute of AttributeType:DMGD2 DATE <DATE_VALUE> {0:1} Not usedextracted from Place Attribute Value if present2 <<SOURCE_CITATION>> {0:M} Not usedPlace Attribute Citation2 TYPE <TYPE_OF_DEMOGRAPICAL_DATA> 1:1 Not usedextracted from Place Attribute Value if present1 _AIDN <ADMINISTRATIVE_IDENTIFIER> {0:M} Not usedPlace Attribute of AttributeType:AIDN2 DATE <DATE_VALUE> {0:1} Not usedextracted from Place Attribute Value if present2 <<SOURCE_CITATION>> {0:M} Not usedPlace Attribute Citation2 TYPE <TYPE_OF_ADMINISTRATIVE_IDENTIFIER> {1:1} Not usedextracted from Place Attribute Value if present
1 <<MULTIMEDIA_LINK>> {0:M} When Media are present
1 <<NOTE_STRUCTURE>> {0:M} When Notes are present
=== Questions ===
# Gedcom GEDCOM L recommends that when 'user defined tags' (those that are preceded by an '_') are present, that an explanation of these tags is placed in the header of the exported Gedcom GEDCOM file as a Schema. I was unable to find any example Gedcom GEDCOM files that included this feature for _LOC tag, although I did find some very old Gedcom GEDCOM files with the _SCHEMA tag, mostly exported by FTW vers 2.
#: Do we want to include it for export?
#: Author recommends NOT to include.
#: Gedcom GEDCOM L Schema for the _LOC tag is included below.
#: <nowiki> 0 HEAD
...
5 CONC GEDCOM/PLAC-Tag#Location_Records
</nowiki>
# Should Gramps be exporting GEDCOM custom tags? These tags (starting with an '_' example: "_LOC") are defined by GEDCOM specifications as legal ways to extend the Gramps specification. Programs that do not recognize these tags are supposed to ignore them except for some sort of warning to the user. Gramps does this for our own import by including these in an error dialog after import, as well as notes in the database associated with the object when possible. Gramps already exports a few of these tags for other purposes.
# How do we want to include this into Gramps?
#: Could be The code is developed in Gramps under master, branch for release at 5.1 timeframe, or developed as an Addon which would replace current libgedcom.py for 5.0 branch. Or even as a 'bug' for gramps50 branch
=== Comments ===
Round trip Gedcom GEDCOM in to Gedcom GEDCOM out will not be transparent for all the fields in the Gedcom GEDCOM L _LOC record. Data should not be lost, but some data will get converted to Notes, which would be inconvenient for users.
==See also==
*[[GEPS 006: Better Place handling]]
*[[GEPS 045: Place Model Enhancements]]
*[[GEDCOM Extensions]] (Note that ''Import GEDCOM Extensions'' are currently not supported as Gramps needs to be refactored.[http://gramps.1791082.n4.nabble.com/Geneanet-Gramps-importing-woes-tp4673883p4673965.html])
)
[[Category:GEPS|M]]
266
edits

Navigation menu