15,091
edits
Changes
From Gramps
m
{{out of date[[File:PlaceCompletion-Tool-Addon-50.png|page needs to be reworked and maybe split}}right|450px|PlaceCompletion - Tool - Addon - default dialog]]
A The {{man label|PlaceCompletion}} tool to bring the addon can import places in into your Gramps database in accordance with the Gramps requirements: batch add country, county; look-up latitude-longitude; set description (title); ...
Follow the installation details in the [[Place_completion_tool#Download|Download]] section for your version of Gramp.
[[Image:place_completion_tool_example1.png|right|300px|Place completion tool]]
== Place Completion tool ==
# A third option is to set attributes of all selected places. You can set the country, state, county, parish, zip/postal code and city attributes of all places in one sweep.
Clicking ''Selecting {{man button|Help'' }} will bring you to this page, clicking {{man button|Close}} will close the ''ClosePlace completion tool'' will close the window and clicking ''{{man button|Google Maps'' } when an entry is selected in the results field has the same effect as pressing tab on an entry (see above).
== Download ==*[[Addon:GetGOV]] - download place information from the [http://gov.genealogy.net/search/index GOV gazetteer].*[[Addon:GeoName]] - downloads place information from the [http://www.geonames.org/ GeoNames gazetteer].
If you use Gramps 4.1 , then use the *[[4.1_Addons|automatic installationGeoCodes]].
If you use Gramps 3.4 , then use the === Design specification ===*See [[3.4_Addons|automatic installationPlace completion tool specification]].
If you use Place/Location is a newer concept in Gramps 3.3 Many older databases only have a Place title field which is a descriptive text containing city, then use state, country. To distribute these values into the [[3.3_Addons|automatic installation]].correct attribute fields, see Parse Title Details below
stop redirect
{{Third-party pluginaddon}}{{man warn|This tool is of limited use for Gramps 5.x series.|Please read the [[Place_completion_tool#Issues|Issues]] section. You should consider using the [[Addon:PlaceCleanupGramplet|Place Cleanup Gramplet]] instead.}}
This tool helps you fill in the place attributes like county, country, ..., by allowing you to select the places you work on, and do changes on all these places with one button click.
*Conversion of latitude and longitude to a fixed data format. On import one might obtain latitude and longitude in several different formats. A conversion tool to store them all in the same format is useful.
*Construction of a uniform title/description field, from the data in the place object.
Follow the installation details in the [[Addon:Place_completion_tool#Download|Download]] section for your version of Gramp.
== Usage Instructions ==
=== Download resources ===
{{stub}}<!--page needs to be reworked and maybe split-->
The place completion tool can look up for you latitude/longitude, add county information (USA), ... . For some of this functionality, you must download data files of the countries you are interested in. Right now you have three options:
#Download geonames country files. You can do this [http://download.geonames.org/export/dump/ here freely]. {{man menu|Geonames parses fastest, so is the advised format to use.}}
#Download geonames USA state files. You can do this [http://geonames.usgs.gov/domestic/download_data.htm here freely]. This is advised for USA searches, as the data in the USA country file contains many doubles, which can be avoided by searching state per state. State info also contains county information.
#download Download GNS Geonet country files (not available for usa). You can do this [ftp://ftp.nga.mil/pub2/gns_data here freely with ftp].
Watch out, some of these downloads are '''VERY''' large, especially USA data. Only download what you need! If the download is a compressed zip file, you will need to extract the data file before you can use it.
{{man note|Note:|The geonames data of popular places is in English, so e.g. municipalities in Italy will be found, but Roma not, as this is Rome in English. To find data with these you need to search in the localised variants of the name (see below)}}
{{man warn|Warning:|DO NOT BETA TEST WITH USE ON YOUR RESEARCH DATA. EXPORT DATA FIRST TO HAVE A BACKUP. Before use [[How_to_make_a_backup#Gramps_version_3.3_and_later|backup]] your Family Trees.}}
=== Starting the tool ===
You will find the plugin under {{man menu|Tools > Utilities > PlaceCompletion...}}
=== The dialog explained ===
[[ImageFile:place_completion_tool_emptyPlace completion tool empty.png|500px600px|The Place Completion Tool]]
The Dialog consists of 4 parts:
###All in decimal notation: use the decimal system to denote lat/lon.
###Correct -50° in 50°S: a much seen error is to use - for the classical degree notation, which is wrong, and which Gramps will not be able to interpret. With this option this error is looked for and corrected.
====Part 3. Overview of the results ====
====Part 4. Actions ====
After you have checked the changes in '''Part 3''', you can now apply them all with one button click, by clicking the ''{{man button|Apply'' }} button.
== Example ==
Open the example file from the examples where latitude and longitude are empty: [http://svn.code.sf.net/p/gramps/code/branches/maintenance/gramps34/example/gramps/example.gramps example.gramps].
We will now show how the places in this file can be completed. The best thing to do is open to create a new family tree (.grdb)Family Tree, give it a name, and import the example.gramps file. This file has 852 places, which would mean a lot of manual edits if you do not use this tool!
Now, open the place view. You will see all places are of the form:
We open the ''Place completion tool'':
[[ImageFile:place_completion_tool_example1Place completion tool example1.png|500px|left|Parse the Place Name Field]]Here we have selected ''{{man label|All Places''}}, and we parse '''Change the title as into'''as {{man label|City [,|.State] State''}}. Click on ''{{man button|Find''}}, quickly scan the data if all looks ok, and then click on ''{{man button|Apply''}}. You are notified that 851 places are updated443 place records were modified. This is one less that the number of places. Indeed, one place does have a different type of title: ''Puerto Rico'' has no state information. {{-}}
=== Step 2: Look-up latitude and longitude ===
We have downloaded the GeoNames datafiles for the USA states, and will now use that to complete the latitude and longitude of the data. At the same time, this will fill up the county field.
[[Image:place_completion_tool_example2.png|500px|left|Look up lat/lon for Alaska]] In the above screenshot, you see we have selected All Places with State=AK. In the second part of the window we give that we want to search in the AK_DECI.txt file downloaded from GeoNames, using the parsing method: ''GeoNames USA state file, city search''. Note that if you want to change AK into Alaska, this would be possble. Just set state=Alaska in the set attributes section of the window.
In the screenshot, you see we have selected All Places with State=AK. In the second part of the window we give that we want to search in the AK_DECI.txt file downloaded from GeoNames, using the parsing method: ''GeoNames USA state file, city search''.
{{-}}
Note that if you want to change AK into Alaska, this would be possible. Just set state=Alaska in the set attributes section of the window.
Do this now for all the states. Always check for doubles. Eg, for state ''AL'', going over the changes, we encounter:
{{man note|Regex Help|For those new to Python and Regex please review the HOWTO here:<br /> http://docs.python.org/dev/howto/regex.html}}
#<nowiki>"City [,|.] State" is parsed by : <code>r'\s*(?P<'+city_translated +r'>.+?)\s*[.,]\s*(?P<'+state_translated +r'>.+?)\s*$'</nowikicode>#<nowiki>"City [,|.] Country" is parsed by : <code>r'\s*(?P<'+city_translated +r'>.+?)\s*[.,]\s*(?P<'+country_translated +r'>.+?)\s*$'</nowikicode>#<nowiki>"City (Country)" is parsed by : <code>r'\s*(?P<'+city_translated +r'>.*?)\s*\(\s*(?P<'+country_translated +r'>[^\)]+)\s*\)\s*$'</nowikicode>#<nowiki>"City" is parsed by : <code>r'\s*(?P<'+city_translated +r'>.*?)\s*$'</nowikicode>
Here the variables used are:
For the regex of lat/lon lookup, you need to indicate which data must be replaced with existing place attributes for the search, as well as indicate which regex groups must be extracted.
#<nowiki>"GeoNames country file, city search" is parsed with: <code>r'\t'+CITY_transl +r'\t[^\t]*\t[^\t]*\t' +latgr + r'[\d+-][^\t]*)\t' + longr + r'[\d+-][^\t]*)\tP'</nowikicode>#<nowiki>"GeoNames country file, city localized variants search" is parsed with: <code>r'[\t,]'+CITY_transl+r'[,\t][^\t\d]*\t?' +latgr + r'[\d+-][^\t]*)\t' + longr + r'[\d+-][^\t]*)\tP'</nowikicode>#<nowiki>"GeoNames country file, title begin, general search" is parsed with: <code>r'\t'+TITLEBEGIN_transl +r'\t[^\t]*\t[^\t]*\t' +latgr + r'[\d+-][^\t]*)\t' + longr + r'[\d+-][^\t]*)\t[PSTV]'</nowikicode>#<nowiki>"GeoNames USA state file, city search" is parsed with: <code>r'\t'+CITY_transl+r'\tPopulated Place\t[^\t]*\t[^\t]*\t' + countygr + r'[^\t]*)' + r'\t[^\t]*\t[^\t]*\t[^\t]*\t' +latgr + r'[\d+-][^\t]*)\t' + longr + r'[\d+-][^\t]*)'</nowikicode>#<nowiki>"GNS Geonet country file, city search" is parsed with: <code>r'\t'+latgr+r'[\d+-][^\t]*)\t'+longr+r'[\d+-][^\t]*)' + r'\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\tP\t[^\t]*\t[^\t]*' + r'\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*' r'\t[^\t]*\t[^\t]*\t[^\t]*' + r'\t'+CITY_transl+r'\t[^\t]*\t[^\t\n]+$'</nowikicode>#<nowiki>"GNS Geonet country file, title begin search" is parsed with: <code>r'\t'+latgr+r'[\d+-][^\t]*)\t'+longr+r'[\d+-][^\t]*)'+ r'\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[PLSTV]\t[^\t]*\t[^\t]*'+ r'\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*\t[^\t]*' + r'\t[^\t]*\t[^\t]*\t[^\t]*' + r'\t'+TITLEBEGIN_transl+r'\t[^\t]*\t[^\t\n]+$'</nowikicode>#<i>Read of mediawiki CSV dump. This reads the files on [http://tools.wikimedia.de/~kolossos/wp-world/pub_CSV_test3.csv.gz] (link gone) (for more information, see http://meta.wikimedia.org/wiki/WikiProjects_Geographical_coordinates) (''Contribution by nomeata'')</i>
For extraction of data you can use the same groupnames as in title parsing, so eg latgr in above should read: <nowikicode>r'(?P<'+lat_translated +r'>'</nowikicode> .
The syntax for the values that need to be used for searching in the file, eg CITY_transl, is given by : _('CITY'). You can use as substitution values:
The tool will read in the given regex, replace the substitution strings by the values in the place object, do the search, and extract the regex groups given from the result.
Resource: [httphttps://deen.wikipedia.org/wiki/Wikipedia:GEOnet_Names_Server GEOnetNames Server] == Design specification ==*See [[Place completion tool specification]] Place/Location is a newer concept in Gramps. Many older databases only have a Place title field which is a descriptive text containing city, state, country. To distribute these values into the correct attribute fields, see Parse Title Details below
== Troubleshooting ==
The tool has to load the datafile for latitude/longitude searching into memory. For large files like USA.txt, this might be impossible if you have limited memory. You can try to close as many programs running at together with Gramps, and try the tool again.
==See also==
== Download ==* If you use Gramps 5.0 , then use the [[5.0_Addons|automatic installation]].* If you use Gramps 4.2 , then use the [[4.2_Addons|automatic installation]].* If you use Gramps 4.1 , then use the [[4.1_Addons|automatic installation]].* If you use Gramps 3.4 , then use the [[3.4_Addons|automatic installation]].* If you use Gramps 3.3 , then use the [[3.3_Addons|automatic installation]].* If you use Gramps 3.2 , then use the [[3.2_Addons|automatic installation]].
=== Manual install ===
If you still use Gramps 2.2.5+, then you will need version 1.0 of the Place Completion Tool. You find it at [http://cage.ugent.be/~bm/varia/placecompletion_1_0.tar.gz placecompletion_1_0.tar.gz].
==See alsoIssues ==* None of the Actions in the tool work for Apple Mac {{bug|10004}} in 4.2 release. This is fixed in 5.x release.*For Gramps 5.x series please see notes [[GeoCodes]https://github.com/gramps-project/addons-source/pull/107 here]** The most concerning is that this tool is quite outdated in our current place structure (for Gramps 5.x). While it works, the title change capability is of little utility for those users who use the automatic place title function. The new place hierarchy largely eliminates the need for this tool. Information is no longer duplicated, so we no longer get inconsistencies which need fixing. Also changes to all titles lower in the hierarchy can now be made by changing the name of a single place.** The whole tool is useless unless the user has already manually enclosed his places.(for Gramps 5.x)** The user can still use it for Lat/Long creation and update.(for Gramps 5.x)
* Maybe replace GoogleMaps button (eg, http://open.mapquestapi.com/nominatim/)
[[Category:Proposed Tool SpecificationsAddons]]
[[Category:Plugins]]
[[Category:Developers/General]]
[[Category:Tools]]
