Python Tutorial on Geocoding Locations

In this tutorial, I will provide a step-by-step process on geocoding location names from a csv file and retrieving their coordinates which can be useful when dealing with files with hundreds of location names. Geocoding is simply the process of inputting the description of a location (like an address or city) and getting a latitude-longitude pair as an output which can be used in plotting points on a map. In the context of digital humanities, this could be incredibly useful for mapping locations where the dataset is given with only the names of locations, and having a program that could convert those names to coordinates would save a lot of time. In this tutorial, I will be walking through my version of the program where I processed a csv file that had columns for birth locations and death locations of various artists.

Step 1: A Bit of Preprocessing

Although this program is incredibly useful, it’s not perfect and there needs to be a bit of preprocessing of the data in order for it to work correctly. First of all, the data needs to be in CSV format which thankfully is not too difficult to achieve. Next is ensuring locations are formatted correctly. For countries outside the United States, there isn’t much of a problem as long as they are formatted as follows “city, country.” For the US, they must be formatted as follows “city, state” because there are cities that share names. Unfortunately, I’m not aware of a quick solution to this besides manually changing each entry that is not formatted correctly. Finally comes dealing with empty entries in the file which also is not too difficult thanks to the pandas library in Python. With this library, you can remove all rows that have no entries at all, and you can replace empty entries with a value of your choice. To do this, you first have to ensure you have the latest version of Python and have installed the package pip in VSCode.

Firstly, install VSCode and then Python. To install pip on Windows click here, and to install pip on MacOS click here. Once all that has been installed and verified, open up VSCode and open up a new terminal by navigating to the top of the screen and clicking “Terminal” -> “New Terminal” and run the following command and then restart VSCode once it successfully installs:

pip3 install pandas

If that doesn’t work, try running this instead:

pip install pandas

Once pandas has been installed, create a new python file.

For this tutorial, I have name it “cleanupdata.py” and looks like this (ignore the folder named temporary):

In the file, import the pandas package you just installed and set up your csv file for reading like so:

import pandas as pd

initial_file = pd.read_csv('your_csv_file.csv')

You can then remove all the rows that are missing all entries, and add a value (in this case I used “None”) to rows that have some values missing:

file_with_empty_rows_removed = initial_file.dropna(how='all')

cleaned_file = file_with_empty_rows_removed.fillna("None")

Finally, in order to run the newly modified file with the geocode program, you must convert the file back into CSV format which is done with the following line:

cleaned_file.to_csv('your_output_file.csv', index=False)

In this case, ‘your_output_file.csv’ will be created and will contain the contents of your newly cleaned up file. Once all that’s been done, your workspace will look like this:

Step 2: Install Geopy and Import Necessary Packages

In order to install geopy, run the following command in a new terminal:

pip3 install geopy

or

pip install geopy

and restart VSCode.

Open up VSCode once again and create a new Python file in the same spot as your other files, I have named mine “latlong.py” and import the following packages into your file:

from geopy.geocoders import Nominatim
import time, csv

Step 3: Initialize a Nominatim Client and Begin Geocoding

In Step 2, we imported the Nominatim library which we will create an instance of in order to access information necessary to geocode. We will first create a Nominatim instance as follows:

# The name you give to the user_agent variable can be whatever you want
app = Nominatim(user_agent="tutorial")

The goal of this program is to read through a csv file with location names and then create a new csv file that replaces the names of the locations with their coordinates. Therefore, we will begin by getting the coordinates from each location and storing them somewhere to then write into a new csv file. We will begin by geocoding each location and storing the coordinates in a list.

Step 4: Open CSV File and Begin Iterating

Using your cleaned up csv file from Step 1, open it and prepare it for reading like so:

with open('BirthandDeathLocations.csv', mode='r', encoding='utf-8') as file:
        csvfile = csv.reader(file)
        next(csvfile) # Skips the header column

After opening the file, initialize a list to store the coordinates and then begin iterating through each row in the csv file. While we iterate, we want to store each location name into its own variable which will be passed as a parameter to another function “getcoordinatesbycity()” which will return the coordinates of the location:

       coordinateslist = []
       for row in csvfile:
            birthplace = row[0] # Entry in the first column
            deathplace = row[1] # Entry in the second column
            coordsbirth = getcoordinatesbycity(birthplace)
            coordsdeath = getcoordinatesbycity(deathplace)
            coordinateslist.append((coordsbirth, coordsdeath))

For example, using the first row of the following image,

“Udine, Italia” would be stored in the “birthplace” variable and “Zurich, Scheiz” would be stored in the “deathplace” variable.

Step 5: Get the Actual Coordinates

The function called “getcoordinatesbycity()” is what is responsible for getting the actual coordinates.

def getcoordinatesbycity(city):

  # Part of Nominatim usage policy, unfortunately makes the program run much slower
    time.sleep(1)

    # Checks for empty entries
    if (city.strip() == "None"):
        return "N/A"
    else:
        try:
            location = app.geocode(city).raw
            latitude = location['lat']
            longitude = location['lon']

            return latitude, longitude
        # In case of timeout error
        except:
            return getcoordinatesbycity(city)

We retrieve all the data of a location by calling the function “app.geocode().raw” (remember “app” from Step 3?) which returns a dictionary with a lot of data about the location of which we want the latitude and longitude which we access via simple indexing.

As you can see, we return “latitude” and “longitude” together. In Python, this is known as a tuple and can be used to return multiple things. The function only returns a tuple containing the coordinates if the location is valid, otherwise it just returns “N/A.”

Step 6: Adding Coordinates to List

We now have a function that either returns a set of coordinates or a string “N/A” for both birth and death locations, and now we have to add them to the list we initialized in Step 4. We can simply add them with the following line after calling “getcoordinatesbycity()” twice:

            coordinateslist.append((coordsbirth, coordsdeath))

In order to keep “coordsbirth” and “coordsdeath” together, we have to add them to the list as a tuple.

Step 7: Write Coordinates to a New File

Once we have added all our sets of coordinates to the list, we can then iterate through it and write the contents to a new csv file.

with open('Coordinates.csv', mode='w') as outfile:
        outfile.write("Latitude(Birth)" + ", " + "Longitude(Birth)" + ", " + "Latitude(Death)" + ", " + "Longitude(Death)" + "\n") # Header row for new file
        for i in range(0, len(coordinateslist)):

Now, because of the implementation, there are three different scenarios for each element in the list:

• The element is a tuple that contains two tuples containing coordinates of both the birth and death locations (i.e. ((45.48686, 82.59394), (30.6858, 32.940)))
• The element is a tuple that contains one tuple of coordinates for the birth location and “N/A” for the death location (i.e. ((45.78494, 56.9384), N/A))
• The element is a tuple that contains “N/A” for the birth location and a tuple of coordinates for the death location (i.e. (N/A, (45.7950, 87.684)))

To make it easier to read, we will store each element of the list into a variable “element” and then store the birth location in a variable “birthloc” and death location in a variable “deathloc.”

            element = coordinateslist[i]
            birthloc = element[0] # First part of element
            deathloc = element[1] # Second part of element

Then accounting for the three scenarios described above, we write to the new csv file. However, before we do this, there is some more manipulation we have to do in order to write the contents of the tuple. We can simply index into the tuple to get the data we need which we can then write onto the csv file:

            if (isinstance(birthloc, tuple) and isinstance(deathloc, tuple)):
                birthlat = birthloc[0]
                birthlon = birthloc[1]
                deathlat = deathloc[0]
                deathlon = deathloc[1]


                outfile.write(str(birthlat) + ", " + str(birthlon) + ", " + str(deathlat) + ", " + str(deathlon) + "\n")

            elif (isinstance(birthloc, tuple) and not isinstance(deathloc, tuple)):
                birthlat = birthloc[0]
                birthlon = birthloc[1]
                
                # for death location
                outfile.write(str(birthlat) + ", " + str(birthlon) + ", N/A, N/A\n")


            elif (not isinstance(birthloc, tuple) and isinstance(deathloc, tuple)):
                deathlat = deathloc[0]
                deathlon = deathloc[1]


                outfile.write("N/A, N/A, " + str(deathlat) + ", " + str(deathlon) + "\n")

And thats it! Once the program is done running, you will have a new csv file that will contain all the coordinates of the locations that were inputted which can then be used for whatever you would like.

I won’t paste my entire program since this is already long enough, but you can view the entire annotated program here.

Resources

The source that inspired this program: How to Get Geolocation in Python – The Python Code

A general, beginner-friendly course on Python: Learn Python – Free Interactive Python Tutorial