I know there was the wish to discuss functional programming but we changed our mind
to this time discuss web APIs since we believe this makes for more fun exercises.
But you are most welcome to implement them in a functional way (study immutable data
structures, and the functions map, filter, and reduce).
Big thanks to Mathias Bockwoldt for the introduction and exercise suggestions below!
In our everyday life, we cope with many different graphical user interfaces (GUIs). When we wish to automate work, such user interfaces are often too complicated and interacting with them programmatically can be error-prone or very difficult.
Therefore, whenever possible, you should try to interact with application programming interfaces (APIs). APIs are also available for many requests via the internet. The most common APIs use HTTP(S) for data exchange. In principle, these are common web addresses, where the parameters of the requests are either part of the URL (so-called GET variables) or can be given using so-called POST-variables. The result is often one of the common standard formats for data, mostly json, xml, or csv, depending on the data.
The module requests is the recommended module to implement web calls to http(s) using Python.
You can install it using pip:
$ pip install requests
Please note the “s” at the end! Do not install request without “s”.
Requests allow both GET and POST requests with a very intuitive scheme.
The Norwegian meteorological institute offers a very rich API to download all kinds of historical and forecast data that is connected to weather and climate. Visit api.met.no for an overview. For us, this API has the disadvantage of delivering only xml-files that are (arguably) more difficult to parse than json or csv files. However, for example, the weather forecast for tomorrow can be requested as follows:
import requests
# This is the URL of the forecast API
url = 'https://api.met.no/weatherapi/locationforecast/1.9'
# The API expects latitude and longitude as parameters.
# The parameters given are those for Tromsø.
parameters = {'lat': 69.651944, 'lon': 18.953333}
# api.met.no wants to know who is calling them, so this will tell them
headers = {'User-Agent': 'Python course UiT'}
response = requests.get(url, headers=headers, params=parameters)
# Print some (more or less) interesting parts of the response
print('Status code:', response.status_code) # en.wikipedia.org/wiki/List_of_HTTP_status_codes
print('Content-type:', response.headers['content-type'])
print('Encoding:', response.encoding)
# Save the response content to a file (you could as well parse it right-away)
with open('api.met.no_response.xml', 'w') as f:
f.write(response.text)
The Polar Institute has many APIs for different data types. Visit data.npolar.no to get an overview. Besides others, they have historical minutely weather data for their research stations. An example for the Troll station that uses json as format is below.
import requests
# This is the URL of the API
url = 'https://api.npolar.no/weather/troll'
# The API expects a time frame and a format as parameters.
parameters = {'filter-measured': '2018-05-17T11:25:00Z..2018-05-17T11:30:00',
'format': 'json'}
response = requests.get(url, params=parameters)
# Print some (more or less) interesting parts of the response
print('Status code:', response.status_code) # en.wikipedia.org/wiki/List_of_HTTP_status_codes
print('Content-type:', response.headers['content-type'])
print('Encoding:', response.encoding)
# requests has a build-in json interpreter
json = response.json()
print(json)
Uniprot has an API to provide you with protein sequences in different formats, given that you know their unique identifier. Visit www.uniprot.org/help/api_retrieve_entries for more info.
import requests
# This is the URL of the API. Note that the unique identifier is part of the URL.
# In this case, we expect a fasta file.
url = 'https://www.uniprot.org/uniprot/{uid}.fasta'
uid = 'P04406' # human GAPDH
response = requests.get(url.format(uid=uid))
# Print some (more or less) interesting parts of the response
print('Status code:', response.status_code) # en.wikipedia.org/wiki/List_of_HTTP_status_codes
print('Content-type:', response.headers['content-type'])
print('Encoding:', response.encoding)
# Save the response content to a file (you could as well parse it right-away)
with open('human_GAPDH.fasta', 'w') as f:
f.write(response.text)
Utilize at least one of the given APIs or find some interesting API on the net. You might, for example, write a script to download a protein sequence from Uniprot or to get the weather forecast for tomorrow (maybe on Jan Mayen?).
Please play nice with the public APIs and use them reasonably. Don’t run thousands of requests within seconds and obey their limitations.