Termipankki
  1. A
    1. API Blueprint
    2. Addressability
    3. Ajax
    4. Anonymous Function
    5. App Context
  2. B
    1. Blueprint
    2. Business Logic
  3. C
    1. CORS
    2. Callback
    3. Client
    4. Column
    5. Column Attribute
    6. Components Object
      Concept
    7. Connectedness
    8. Container
    9. Control
    10. Converter
      Framework
    11. Cookie
      WWW
    12. Credentials
      Concept
  4. D
    1. DOM
    2. Daemon
    3. Database Schema
    4. Decorator
      Python
  5. E
    1. Element
    2. Entry Point
    3. Environment Variable
  6. F
    1. Fixture
    2. Flask App
    3. Foreign Key
  7. G
    1. Generic Client
  8. H
    1. HTTP Method
    2. HTTP Request
    3. Hash
      Concept
    4. Header
    5. Host Part
    6. Hypermedia
  9. I
    1. Idempotent
    2. Info Object
      Concept
    3. Instance Folder
  10. J
    1. JSON
    2. JSON Schema
  11. L
    1. Link Relation
  12. M
    1. MIME Type
    2. Microservice
    3. Migration
    4. Model Class
  13. N
    1. Namespace
  14. O
    1. ORM
    2. OpenAPI
      OpenAPI
    3. Operation Object
      Concept
  15. P
    1. Pagination
      Concept
    2. Path Parameter
      OpenAPI
    3. Primary Key
    4. Profile
  16. Q
    1. Query
    2. Query Parameter
  17. R
    1. Regular Expression
    2. Request
    3. Request Body
    4. Request Object
    5. Resource
    6. Resource Class
    7. Resource Representation
    8. Response
    9. Response Body
    10. Response Object
    11. Rollback
    12. Routing
    13. Route →
      Routing
    14. Row
  18. S
    1. SQL
    2. Serialization
    3. Static Content
    4. Swagger
      tool
    5. System User
  19. T
    1. Table
    2. Test Setup
    3. Test Teardown
  20. U
    1. URI
    2. URL Template
    3. Uniform Interface
    4. Unique Constraint
  21. V
    1. View Function
    2. Virtualenv
  22. W
    1. WSGI
    2. Web API
    3. Web Server
  23. Y
    1. YAML
      Language
Ratkaistu: / tehtävää

Hypermedia APIs

In the extra part of this exercise material we will dive into hypermedia APIs. This exercise moves away from the SensorHub example to the MusicMeta example which provides a better base for what we are about to discuss.

Full Example

Below is the full MusicMeta API example that we will be discussing in this section. Furthermore, if you haven't done so already, read through the API design extra material in order to understand what this API is trying to achieve.
musicmeta.py

Enter Hypermedia

In order for client developers to know what to actually send - and what to expect in return - APIs need to be documented. We achieved some of this goal by using OpenAPI to document the API, but we can further with
hypermedia
in responses given by the API. This way the API itself describes possible actions that can be taken to the client. For this example we have chosen Mason as our hypermedia format because it has a very clear syntax for defining hypermedia elements and connecting them to body.

Hypermedia Controls and You

You can consider the API as a map and each
resource
as a node. The resource that you most recently sent a GET request to is basically the node that says "you are here".
Hypermedia
controls
describe the logical next actions: where to go next, or actions that can be performed with the particular node you're in. Together with the resources they actually form a client-side state diagram of how to navigate the API. Hypermedia controls are extra attributes attached to the data
representation
we showed in the API design example.
A hypermedia control is a combination of at least two things:
link relation
("rel") and target
URI
("href"). These answer two questions: what does this control do, and where to go to activate it. Note that link relation is a machine-readable keyword, not a description for humans. Many generally used relations are being standardized (full list) but APIs can define their own when needed as well - as long as each relation always means the same thing. When a client wants to do something, it uses the available link relations to discover what URI the next request should go to. This means that clients using our API should never need to have hardcoded URIs - they will find the URI by searching for the correct relation instead.
Mason also defines some additional attributes for hypermedia controls. Of these "method" is one that we will be using frequently, because it tells which
HTTP method
should be used to make the request (usually omitted for GET as it is assumed to be the default). There's also "title" which can be used in
generic clients
(or other generated clients) to help the client's human user figure out what the control does. Even beyond that we can also include JSON schema representation that defines how to send data to the API.
In Mason hypermedia controls can be attached to any object by adding the "@controls" attribute. This in itself is an object where link relations are attribute names whose values are also objects that have at least one attribute: href. For example, here is a track item with controls to get back to the album it is on ("up") and to edit its information ("edit"):
{
    "title": "Wings of Lead Over Dormant Seas",
    "disc_number": 2,
    "track_number": 1,
    "length": "01:00:00",
    "@controls": {
        "up": {
            "href": "/api/artists/dirge/albums/Wings of Lead Over Dormant Seas/"
        },
        "edit": {
            "href": "/api/artists/dirge/albums/Wings of Lead Over Dormant Seas/2/1/",
            "method": "PUT"
        }
    }
}
Or if we want each item in a collection to actually have its own URI available to clients:
{
    "items": [
        {
            "artist": "Scandal",
            "title": "Hello World",
            "@controls": {
                "self": {
                    "href": "/api/artists/scandal/albums/Hello World/"
                }
            }
        },
        {
            "artist": "Scandal",
            "title": "Yellow",
            "@controls": {
                "self": {
                    "href": "/api/artists/scandal/albums/Yellow/"
                }
            }
        }
    ]
}

Custom Link Relations

While it's good to use standards as much as possible, realistically each API will have a number of
controls
whose meaning cannot be explicitly conveyed with any of the standardized
relations
. For this reason Mason documents can use link relation
namespaces
to extend available link relations. A Mason namespace defines a prefix and its associated namespace (similar to XML namespace, see CURIEs). The prefix will be added to link relations that are not defined in the IANA list.
When a relation is prefixed with a namespace prefix, it is meant to be interpreted as attaching the relation at the end of the namespace and makes the relation unique - even if another API defined a relation with the same name, it would have a different namespace in front. For example if want to have a relation called "albums-va" to indicate a control that leads to a collection of all VA albums, its full identifier could be http://wherever.this.server.is/musicmeta/link-relations/#albums-va. To make this look less wieldy we can define a namespace prefix called "mumeta", and then include this control like so:
{
    "@namespaces": {
        "mumeta": {
            "name": "http://wherever.this.server.is/musicmeta/link-relations/#"
        }
    },
    "@controls": {
        "mumeta:albums-va": {
            "href": "/api/artists/VA/albums"
        }
    }
}
Also if a client developer visits the full URL, they should find a description about the link relation. Note also that this is normally expected to be a full URL because the server part is what guarantees uniqueness. In later examples you will see we're using a relative
URI
- this way the link to the relation description itself works even if the server is running in a different address (i.e. most likely localhost:someport).
Information about the link relations must be stored somewhere. Note that this is intended for client developers i.e. humans. In our case a simple HTML document with anchors for each relation should be sufficient. This is why our namespace name ends with #. It makes it convenient to find each relation's description. Before moving on, here's the full list of custom link relations our API uses: add-album, add-artist, add-track, albums-all, albums-by, albums-va, artists-all, delete.

API Map

The last order of business in designing our API is to create a full map with all the
resources
and
hypermedia
controls
visible. This a kind of a state diagram where resources are states and controls are transitions. Generally speaking only GET methods are used to moving from one state to another because other methods don't return a
resource representation
. We have presented other methods as arrows that circle back to the same state. Here's the full map in all its glory.
MusicMeta API state diagram
NOTE: The box color codes are only included for educational purposes to show you how data from the database is connected to resources - you don't need to share implementation details like this in real life, or your course project for that matter.
NOTE 2: the
link relation
"item" does not exist, this is actually "self". In this diagram "item" is used to indicate that this is a transition to an item from a collection through the item's "self" link.
A map like this is useful when designing the API and should be done before designing individual representations returned by the API. As all actions are visible in a single diagram, it's easier to see if something is missing. When making the diagram keep in mind that there must be a path from every state to every other state (
connectedness
principle). In our case we have three separate branches in the
URI
tree and therefore we have to make sure to include transitions between brances (e.g. AlbumCollection
resource
has "artists-all" and "albums-va").

The Road to Transcendence

Consider the above state diagram. Let's assume you're a machine client. You are currently standing in the ArtistCollection node. The goal is to find and modify data about a various artists album titled "Transcendental" (collaboration of Mono and The Ocean). Which links have to be followed in order to do that? Does that path make sense to you?
For your answer, type the shortest list of link relations (use same names as in the diagram) that lead you from ArtistCollection to modifying the VA album's data.
Write the relations on a single line, separated with commas.
Varoitus: Et ole kirjautunut sisään. Et voi vastata.

Entry Point

A final note about mapping API is the
entry point
concept. This should be at the root of the API (in our case: /api/. It's kind of like the API's index page. It's not a
resource
, and isn't generally returned to (which is why it isn't in the diagram). It just shows the reasonable starting options a client has when "entering" the API. In our case it should have
controls
to GET either the artists collection or the albums collection (potentially also the VA album collection).

Enter the Maze

Create a JSON document of the MusicMeta APIs entry point. It should contain two hypermedia controls: link to the artist collection, and link to the albums collection. You should be able to figure out the link relations of these controls from the state diagram. Don't forget to use the mumeta namespace!
Return your JSON file here. Make sure your document is valid JSON and can be parsed with json.load.
Varoitus: Et ole kirjautunut sisään. Et voi vastata.

Advanced Controls with Schema

Up to now we have defined possible actions by using
hypermedia
. Each action comes with a
link relation
that has an explicit meaning, address for the associated resource, and the
HTTP method
to use. This information is sufficient for GET and DELETE requests, but not quite there for POST and PUT - we still don't know what to put in the
request body
. Mason supports adding
JSON Schema
to hypermedia controls.
A schema object can be attached to a Mason hypermedia
control
by assigning it to the "schema" attribute. If the schema is particularly large or you have another reason to not include it in the response body, you can alternatively provide the schema from a URL on your API server (e.g. /schema/album/) and assign the URL to the "schemaUrl" attribute so that clients can retrieve it. The client can then use the schema to form a proper request when sending data to your API. Whether a machine client can figure out what to put into each attribute is a different story. One option is to use names that conform to a standard e.g. we could use the same attribute names as IDv2 tags in MP3 files.
Schemas are particularly useful for (partially) generated clients that have human users. It's quite straightforward to write a piece of code that generates a form from a schema so that the human user can fill it. We'll show this in the last exercise of the course. Below is an example of a POST method control with schema:
{
    "@controls": {
        "mumeta:add-artist": {
            "method": "POST",
            "encoding": "json",
            "title": "Add a new artist",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {
                        "description": "Artist name",
                        "type": "string"
                    },
                    "location": {
                        "description": "Artist's location",
                        "type": "string"
                    },
                    "formed": {
                        "description": "Formed",
                        "type": "string",
                        "format": "date"
                    },
                    "disbanded": {
                        "description": "Disbanded",
                        "type": "string",
                        "format": "date"
                    }
                },
                "required": [
                    "name",
                    "location"
                ]
            }
        }
    }
}
Schemas can also be used for resources that use
query parameters
. In this case they will described the available parameters and values that are accepted. As an example we can add a query parameters that affects how the all albums collection is sorted. Here's the "mumeta:albums-all" control with the schema added. Note also the addition of "isHrefTemplate", and that "type": "object" is omitted from the schema.
{
    "@controls": {
        "mumeta:albums-all": {
            "href": "/api/albums/?{sortby}",
            "title": "All albums",
            "isHrefTemplate": true,
            "schema": {
                "properties": {
                    "sortby": {
                        "description": "Field to use for sorting",
                        "type": "string",
                        "default": "title",
                        "enum": ["artist", "title", "genre", "release"]
                    }
                },
                "required": []
            }
        }
    }
}

Client Example

In order to give you some idea about why we're going through all this trouble and adding a bunch of bytes to our payloads, let's consider a small example from the client's perspective. Our client is a submission bot that browses its local music collection and sends metadata to the API for artists/albums that do not exist there yet. Let's say its local collection is grouped by artists, then albums. Let's say it's currently examining an artist folder ("Miaou") that contains one album folder ("All Around Us"). The goal is to see if this artist is in the collection, and whether it has this album.
  1. bot enters the api and finds the artist collection by looking for a
    hypermedia
    control
    named "mumeta:artists-all"
  2. bot sends a GET to the artist collection using the hypermedia control's href attribute
  3. bot looks for an artist named "Miaou" but doesn't find it
  4. bot looks for "mumeta:add-artist" hypermedia control
  5. bot compiles a POST request using the control's href attribute and the associated
    JSON schema
  6. after sending the POST request, the bot discovers the artist's address from the response's location
    header
  7. bot sends a GET to the address it received
  8. from the artist representation the bot looks for the "mumeta:albums-by" hypermedia control
  9. bot send a GET to the control's href attribute, receiving an empty album collection
  10. since the album is not there, bot looks for "mumeta:add-album" control
  11. bot compiles a POST request using the control's href attribute and the associated JSON schema
The important takeaway from this example is that the bot doesn't need to now any
URIs
besides /api/. For everything else it has been programmed to look for
link relations
. All the addresses it visits are parsed from the responses it gets. They could be completely arbitrary and the bot would still work. Depending on the bot's AI it can survive quite drastical API changes (for example when it GETs the artist representation and finds a bunch of controls, how exactly has it been programmed to follow "mumeta:albums-by"?)
One really cool thing about hypermedia APIs is that they usually have a generic client to browse any API if it's valid. The client will generate a human-usable web site by using hypermedia controls to provide links from one view to another, and schemas to generate forms.

Hypermedia Profiles

By adding
hypermedia
we have managed to create APIs that machine clients can navigate once they have been taught the meaning of each
link relation
, and the meaning of each attribute in
resource representations
. But how exactly does the machine learn these things? This is a ongoing challenge for API development - for now one way is to educate the human developers by using resource
profiles
. Profiles describe the semantics of resources in human-readable format. This way human developers can transfer this knowledge to their client, or a human user of a client can use this knowledge when navigating the API.

What's in a Profile?

There's no universal consensus about what exactly should be in a profile, or how to write one. Regardless of how it's written, the profile should have semantic descriptors for attributes (of the resource representation) and protocol semantics for actions that can be taken (or a list of link relations associated with the resource). Collections don't necessarily have their own profiles, like in our example they don't. Except for album since it is both an item and a collection.
If your resource represents something that is relatively common, using attributes defined in a standard (or standard proposal) is recommended. If your entire resource representation can conform to a standard, all the better. You can look for standards in https://schema.org/. One important future step for our example API would be to use attributes from this schema for albums and tracks.

Distributing Profiles

Like
link relations
, information about your
profiles
should be accessible from somewhere. In our example we have chosen to distribute them as HTML pages from the server using
routing
/profiles/{profile_name/. Links to profiles can be inserted as
hypermedia
controls
using the "profile" link relation. For example, to link the track profile from a track
representation
:
{
    "@controls": {
        "profile": {
            "href": "/profiles/track/"
        }
    }
}
Another possibility is to use HTTP Link
header
in responses. 
Link: <http://where.ever.the.server.is/profiles/track/>; rel="profile"
However this is somewhat more ambiguous. Our album
resource
is an example that actually should link to two profiles - album and track. For this reason we have included profiles as hypermedia controls, and for collection types we have included one with every item.

Implementing Hypermedia

Hypermedia is essentially a bunch of JSON that is added to GET responses. At its core this is a rather simple matter of adding more content to the dictionaries we get from
serializing
model instances. As the Mason syntax is quite verbose, simply hardcoding these additions to response body dictionaries is a fast lane to trouble town. In this section we will discuss how to be a bit more systematic when adding hypermedia to responses.

Subclass Solution

In Mason the root type of a hypermedia response is
JSON
object which - as we have learned - is in most ways the equivalent of a Python dictionary. However if you go and define the entire response as a dictionary in each resource method separately, the likelihood of introducing inconsistencies is quite high. Furthermore the code becomes cumbersome to maintain. For any applications that produce JSON, a good development pattern is to create a dictionary subclass that includes a number of convenience methods that automatically manage integrity of the selected JSON format.
As stated before, our chosen hypermedia type for examples in this course is Mason. There are three special attributes in Mason JSON documents that we use commonly: "@controls", "@namespaces" and "@error". Just to give you an idea of what we're trying to avoid, here's how we would need to make a Mason document with one
namespace
and
control
with normal dictionaries: 
body = artist.serialize()
body["@namespaces"] = {
    "mumeta": {
        "name": "/musicmeta/link-relations/#"
    }
}
body["@controls"] = {
    "mumeta:albums-by": {
        "href": api.url_for(AlbumCollection, artist=artist)
    }
}
Putting stuff like this - and usually in bigger numbers - is incredibly messy. What we want to achieve is, instead, something like this:
body = MasonBuilder(**artist.serialize())
body.add_namespace("mumeta", "/musicmeta/link-relations/#")
body.add_control("mumeta:albums-by", api.url_for(AlbumCollection, artist=artist))
Without doubt this looks much cleaner. The MasonBuilder class would take care of details about how exactly to add the namespace and control into the resulting document. If something about that changed, making the change in the class would also apply the change to all resource methods. So, how does this look on the class itself? Something like this:
class MasonBuilder(dict):

    def add_namespace(self, ns, uri):
        if "@namespaces" not in self:
            self["@namespaces"] = {}

        self["@namespaces"][ns] = {
            "name": uri
        }
    
    def add_control(self, ctrl_name, href, **kwargs):
        if "@controls" not in self:
            self["@controls"] = {}

        self["@controls"][ctrl_name] = kwargs
        self["@controls"][ctrl_name]["href"] = href
Observe how MasonBuilder extends the dict Python class, so the way of creating a MasonBuilder is exactly the same to create a dictionary using the dict class.
Implementation detail: if you have not seen **kwargs used before, this is a Python feature called packing/unpacking. It's a wildcard catch for keyword arguments given to the function/method when it's called: all such arguments will be packed into the kwargs dictionary. So when we call this method with method="POST" the kwargs will be end up like this: {"method": "POST"}. This feature is also used in the dict __init__ method (which we inherit), you can give it keyword arguments to initialize it with a bunch of keys.
Because each object should have only one "@controls" and "@namespaces" attributes, it makes sense to automatically create this when the first namespace/control is added. We can add a similar method for the "@error" attribute:
    def add_error(self, title, details):
       self["@error"] = {
            "@message": title,
            "@messages": [details],
        }
You can download the entire class with docstrings added from below. If you're using the more elaborate project structure, this class is something that definitely belongs into the utils.py file and should be imported to other modules with from sensorhub.utils import MasonBuilder. The file below also incudes utility methods for adding PUT, POST, and DELETE controls.
masonbuilder.py
Since items in a colletion type
resource
can have their controls, you should construct them as MasonBuilder instances instead of dictionaries. This way you can add
controls
to them just as effortlessly as you can to the root object. Let's take an example of how to add the very important "self"
relation
to each sensor in the sensors collection
resource representation
. 
body = MasonBuilder(items=[])
for artist in Artist.query.all():
    item = MasonBuilder(artist.serialize(short_form=True))
    item.add_control("self", api.url_for(Artist, artist=artist))
    body["items"].append(item)
Note: instead of passing the serialize method's result to the MasonBuilder constructor like we do in this example, you can change the seriliaze method itself to initialize a MasonBuilder instance of a normal dictionary.

API Specific Subclasses

While the builder class we gave you goes quite a long way, making an API specific subclass can reduce the amount of boilerplate code in view methods a little bit more. For instance, to create a control for creating artist resources with POST, a function call like this is required:
body.add_control_post(
    "mumeta:add-artist",
    "Add a new artist",
    api.url_for(ArtistCollection),
    Artist.json_schema()
)
If you have to put multiple such method calls into your view methods, they get very bulky. It would probably be better if we could simply do:
body.add_control_add_artist()
This of course require that the gritty details of the add_control_post function call are hidden away somewhere else. A good way to hide these details is to subclass MasonBuilder, and put these convenience methods there. 
class MusicMetaBuilder(MasonBuilder):
    
    def add_control_add_artist(self):
        self.add_control_post(
            "mumeta:add-artist",
            "Add a new artist",
            api.url_for(ArtistCollection),
            Artist.json_schema()
        )
This case is very simple because the control has no variables at all. But we can also take an example with variables, like the PUT method control for tracks:
    def add_control_edit_track(self, artist, album, disc, track):
        self.add_control_put(
            "Edit this track",
            api.url_for(
                TrackItem,
                artist=artist,
                album=album,
                disc=disc,
                track=track
            ), 
            Track.json_schema()
        )
Now we can get a control for editing a track with a much simpler function call (all the variable values come from the view method's parameters). 
body.add_control_edit_track(artist, album, disc, track)
With the gritty details carefully hidden away, the view method code will stay much more compact, making it much more easier to see what the view method actually does. For instance, an artist item has quite a few controls to it, but the view method code stays quite neat:
class ArtistItem(Resource):

    def get(self, artist):
        body = MusicMetaBuilder(artist.serialize())
        body.add_namespace("mumeta", LINK_RELATIONS_URL)
        body.add_control("self", href=request.path)
        body.add_control("profile", href=ARTIST_PROFILE_URL)
        body.add_control("collection", href=api.url_for(ArtistCollection))
        body.add_control_albums_all()
        body.add_control_albums_by(artist)
        body.add_control_edit_artist(artist)
        body.add_control_delete_artist(artist)
        return Response(json.dumps(body), 200, mimetype=MASON)

Inventory Builder

In this task you're going to get a bit more familiar with dictionary subclasses by implementing one yourself. Picking up this habit will make future work on our inventory management API much leaner.
Learning goals: How to implement a dictionary subclass that maintains a Mason
hypermedia
document using convenience methods for adding
controls
.
Before you begin:
We have provided you a base file that contains the necessary resource classes, and the MasonBuilder class that was introduced above.
inventory_builder.py
You need to complete your answer into this file for the checker to work. The checker is dynamic. It constructs a
JSON
document using your InventoryBuilder and then attempts to generate requests based on the controls in that JSON document against the resources in the base template.
In order to be able to test your solution, run the following commands after setting the FLASK_APP environment variable: 
flask init-db
flask populate-db
You can also test your solution with another utility command
flask test-document
Product method: json_schema
You need to fill in the json_schema method in the Product model class in order to make the rest of the task work properly. It should return the schema for a valid product.
Class ProductConverter
Use the class from your solution to Inventory Converter.
Class: InventoryBuilder
For this task we want you to create a class that can be used to add hypermedia controls associated with the inventory manager's product resources. Your class should have a method the following methods for various
link relations
:
  • add_control_all_products
    • paramaters: -
    • rel: "storage:products-all"
    • Leads to the list of all products (GET /api/products/)
  • add_control_delete_product
    • parameters: product model instance
    • rel: "storage:delete"
    • Deletes this product (DELETE /api/products/{handle}/)
  • add_control_add_product
    • paramaters: -
    • rel: "storage:add-product"
    • Creates a new product (POST /api/products/)
    • schema required
      • handle: string
      • weight: number
      • price: number
  • add_control_edit_product
    • parameters: product model instance
    • rel: "edit"
    • Edits a product (PUT /api/products/{handle}/)
    • schema required (same as above)
Varoitus: Et ole kirjautunut sisään. Et voi vastata.

Responses and Errors

We briefly discussed Flask's
response object
in the Resource Locator task where we used it to set custom
headers
. We have now arrived at a stage where we should actually be using it for all responses. This is largely because we need to announce the content type of our
responses
, and this is done by using the mimetype keyword argument. Because we're using Mason, we need to set it to "application/vnd.mason+json". Since this will be repeated in every GET method, it'd be wise to make a constant of it (i.e. MASON = "application/vnd.mason+json"). From now on a typical 200 response would look like:
return Response(json.dumps(body), 200, mimetype=MASON)
We went back to using json.dumps because Response takes the
response body
as a string. We can also change all 201 and 204 responses accordingly. We already learned how to do the 201 response with Location header, and a 204 response is even simpler: 
return Response(status=204)
We have now resolved issues regarding responses in the 200 range (i.e. successful operations). What about errors in the 400 range? Mason also defines what errors should look like. In fact, we actually already implemented the add_error method into our MasonBuilder dictionary subclass. However, even with that, returning an error becomes a multiline effort, and we don't really want that because most of it is boilerplate and resource methods typically return errors at multiple points of their execution. Let's make a convenience function for generating errors:
def create_error_response(status_code, title, message=None):
    resource_url = request.path
    body = MasonBuilder(resource_url=resource_url)
    body.add_error(title, message)
    body.add_control("profile", href=ERROR_PROFILE)
    return Response(json.dumps(body), status_code, mimetype=MASON)
This generates a Mason error messages with a title, and one message with more description about the problem. It also puts the resource URL into the response body, just in case the client forgot what it was trying to do (sometimes actually relevant, e.g. asynchronous use cases). Now instead of writing all that whenever an error is encountered in a resource method, we can just write:
return create_error_response(404, "Not found", "No sensor was found with the given name")

Static Parts of Hypermedia

In addition to generating
hypermedia
representations for
resources
, a fully functional hypermedia API should also serve some static content. Namely:
link relations
and resource
profiles
. Also if you have particularly large schemas and would rather serve them separately from
resource representations
, these should also be served as static content. For profiles and link relations, you can send them out as static files.

As Static Files

If your project doesn't have a static folder yet, now's the time to create one. It's also recommended to create some subfolders to keep things organized, e.g.
static
├── profiles
└── schema
In order to use a static folder, it must be registered with Flask. This is done when initializing the app:
app = Flask(__name__, static_folder="static")
Static views are
routed
with @app.route. If you are storing profiles and such locally as html files, you can implement these views quite easily by using Flask's send_from_directory function which sends the contents of a file as the
response body
- you should add it to your growing from flask import line. With profiles you can use one route definition and view function for all the profiles, like this:
@app.route("/profiles/<resource>/")
def send_profile_html(resource):
    return send_from_directory(app.static_folder, "{}.html".format(resource))
The send_from_directory function is convenient enough that it will send a 404 response if the file is not found. Because link relation descriptions are not particularly lengthy they can be gathered into a single file, served in a similar manner:
@app.route("/sensorhub/link-relations/")
def send_link_relations_html():
    return send_from_directory(app.static_folder, "links-relations.html")
If you are using schema files, you can send them out in a similar manner. You will also need these URLs often in your code since almost all responses will include the
namespace
which requires the link relation URL, and likewise all resource representations have at least one profile link. Therefore you should probably at least introduce them as constants in your code, e.g.
SENSOR_PROFILE = "/profiles/sensor/"
MEASUREMENT_PROFILE = "/profiles/measurement/"
LINK_RELATIONS_URL = "/sensorhub/link-relations/"
If you are using the more elaborate project structure, consider putting these into their own file, e.g. constants.py.

Documenting Hypermedia

So we documented our API with Swagger. Then we self-documented the API with hypermedia. Now the two are no longer in sync, so the final step is to update the OpenAPI documentation to match the new and improved hypermedia API we have built. Frankly there isn't much to do, just need to update the examples in all GET method responses.

Snatching Bodies with Requests

So basically what we want to have for, e.g. the documentation of the GET method for a single artist in our API, looks like this:
parameters:
  - $ref: '#/components/parameters/artist/'
responses:
  '200':
    content:
      application/vnd.mason+json:
        example:
          '@controls':
            collection:
              href: /api/artists/
            edit:
              encoding: json
              href: /api/artists/scandal/
              method: PUT
              schema:
                properties:
                  disbanded:
                    description: Disbanded
                    format: date
                    type: string
                  formed:
                    description: Formed
                    format: date
                    type: string
                  location:
                    description: Artist's location
                    type: string
                  name:
                    description: Artist name
                    type: string
                required:
                - name
                - location
                type: object
              title: Edit this artist
            mumeta:albums-all:
              href: /api/albums/?sortby={sortby}
              isHrefTemplate: true
              schema:
                properties:
                  sortby:
                    default: title
                    description: Field to use for sorting
                    enum:
                    - artist
                    - title
                    - genre
                    - release
                    type: string
                required: []
                type: object
              title: All albums
            mumeta:albums-by:
              href: /api/artists/scandal/albums/
            mumeta:delete:
              href: /api/artists/scandal/
              method: DELETE
              title: Delete this artist
            profile:
              href: /profiles/artist/
            self:
              href: /api/artists/scandal/
          '@namespaces':
            mumeta:
              name: /musicmeta/link-relations#
          disbanded: null
          formed: '2006-08-01'
          location: Osaka, JP
          name: Scandal
          unique_name: scandal
  '404':
    description: The artist was not found
Now that's a handful. There is no way we are writing anything like this manually into the documentation files. Even if we replace the schemas with references, there's just too many things here that it's way too easy to forget something. Luckily, just like with schemas earlier, we can just pull the examples from our own code. Not quite as directly, but close enough.
Regardless of what your API is, the first step is to populate the database with enough data to have examples for everything. Then you can simply go through your routes with requests and use PyYaml to dump the responses into YAML format. Then you can just place the example into wherever you need it. Here's the basic way to do it:
import requests
import yaml
import os.path

SERVER_ADDR = "http://localhost:5000/api"
DOC_ROOT = "./doc/"
DOC_TEMPLATE = {
    "responses": {
        "200": {
            "content": {
                "application/vnd.mason+json": {
                    "example": {}
                }
            }
        }
    }
}

resp_json = requests.get(SERVER_ADDR + "/artists/scandal/").json()
DOC_TEMPLATE["responses"]["200"]["content"]["application/vnd.mason+json"]["example"] = resp_json
with open(os.path.join(DOC_ROOT, "artist/get.yml"), "w") as target:
    target.write(yaml.dump(resp_json, default_flow_style=False))
A small note on folder naming in this example. The MusicMeta API defines endpoint names for resources because it uses the same resource class for multiple endpoints (to separate single artist and VA albums). So the doc folder naming is based on endpoint names instead of resource class names. Resource class name in lowercase is simply the default endpoint name assigned by Flask Restful if none is given.
After running this for all resources, you'd then simply add the remaining details like parameters and error codes. With this as a base operation you could quite easily build your own automation machinery that uses e.g. your API test database, and updates all examples in all documentation files. Or, now that you understand the basic process, you could look into libraries that provide this kind of automation.

Inventory Engineer Finale

We're almost one with our very minimal inventory management API that has served us well for majority of the server side implementation tasks. As your final exam on the topic, we want you to turn it into a hypermedia API that uses Mason. We will stick to managing products, inventory will be Someone Else's Problem.
Learning goals: how to add
hypermedia
to API responses.
Before you begin:
Continue into the same file that was used in the previous task.
Entry Point
  • Route: "/api/"
You need to create an entry point that has one control: "storage:products-all" which should point to the products collection. Remember namespace. This is best done as a normal
view function
.
Resource: ProductCollection
  • Route: "/api/products/"
  • Methods:
    • GET - get list of all products (returns a Mason document)
    • POST - creates a new product (should already be done)
GET: you should already have the basic code for this method. However, instead of returning an array, this method should now return an object, where the array of products is placed in the "items" property. You also need to add the following hypermedia controls to each item in the array:
  • "self" - points to the item
  • "profile" - points to the product profile ("/profiles/product/")
The top level object itself should also have the following controls:
  • "self" - points to the collection itself
  • "storage:add-product" - a control for adding products, see previous task
Don't forget to add the namespace. The document is returned as JSON, with the status code 200 and using "application/vnd.mason+json" as the content type.
POST: add validation using the product schema.
Resource: ProductItem
  • Route: "/api/products/<product:product>/"
  • Methods:
    • GET - get information about the addressed product
    • PUT - modify product information
    • DELETE - delete the addressed product
GET: this method returns information about a product and controls related to it. It should use the product converter as shown in the given file. Add the following controls to response body, and change its media type to "application/vnd.mason+json":
  • "self" - points to the item
  • "profile" - points to the product profile ("/profiles/product/")
  • "collection" - points to the product collection
  • "edit" - a control for editing product information, see previous task
  • "storage:delete" - a control to delete the addressed product, see previous task
Finishing Touches
Finally you need to provide views for
link relations
and
profiles
. These do not need to contain anything in particular as long as they exist and return plain text or HTML as strings. For this task only don't use static files or redirect.
About the Checker
The checker initializes your database with some test data. After this it will run a script that acts like a hypermedia client, traversing your API from the entry point onward using link relations to find resources and actions. The testing is sequential: if your API fails to return a resource representation, all future cases will probably fail as well because the client script cannot find the hypermedia controls it's looking for.
Varoitus: Et ole kirjautunut sisään. Et voi vastata.
?
API Blueprint is a description language for REST APIs. Its primary categories are resources and their related actions (i.e. HTTP methods). It uses a relatively simple syntax. The advantage of using API Blueprint is the wide array of tools available. For example Apiary has a lot of features (interactive documentation, mockup server, test generation etc.) that can be utilized if the API is described in API Blueprint.
Another widely used alteranative for API Blueprint is OpenAPI.
Addressability is one of the key REST principles. It means that in an API everything should be presented as resources with URIs so that every possible action can be given an address. On the flipside this also means that every single address should always result in the same resource being accessed, with the same parameters. From the perspective of addressability, query parameters are part of the address.
Ajax is a common web technique. It used to be known as AJAX, an acronym for Asynchronous Javascript And XML but with JSON largely replacing XML, it become just Ajax. Ajax is used in web pages to make requests to the server without a page reload being triggered. These requests are asynchronous - the page script doesn't stop to wait for the response. Instead a callback is set to handle the response when it is received. Ajax can be used to make a request with any HTTP method.
  1. Kuvaus
  2. Examples
Anonymous functions are usually used as in-place functions to define a callback. They are named such because they are defined just like functions, but don't have a name. In JavaScript function definition returns the function as an object so that it can e.g. passed as an argument to another function. Generally they are used as one-off callbacks when it makes the code more readable to have the function defined where the callback is needed rather than somewhere else. A typical example is the forEach method of arrays. It takes a callback as its arguments and calls that function for each of its members. One downside of anonymous functions is that they function is defined anew every time, and this can cause significant overhead if performed constantly.
  1. Kuvaus
  2. Example
In Flask application context (app context for short) is an object that keeps tracks of application level data, e.g. configuration. You always need to have it when trying to manipulate the database etc. View functions will automatically have app context included, but if you want to manipulate the database or test functions from the interactive Python console, you need to obtain app context using a with statement.
Blueprint is a Flask feature, a way of grouping different parts of the web application in such a way that each part is registered as a blueprint with its own root URI. Typical example could be an admin blueprint for admin-related features, using the root URI /admin/. Inside a blueprint, are routes are defined relatively to this root, i.e. the route /users/ inside the admin blueprint would have the full route of /admin/users/.
Defines how data is processed in the application
Cross Origin Resource Sharing (CORS) is a relaxation mechanism for Same Origin Policy (SOP). Through CORS headers, servers can allow requests from external origins, what can be requested, and what headers can be included in those requests. If a server doesn't provide CORS headers, browsers will browsers will apply the SOP and refuse to make requests unless the origin is the same. Note that the primary purpose of CORS is to allow only certain trusted origins. Example scenario: a site with dubious script cannot just steal a user's API credentials from another site's cookies and make requests using them because the APIs CORS configuration doesn't allow requests from the site's origin. NOTE: this is not a mechanism to protect your API, it's to protect browser users from accessing your API unintentionally.
Callback is a function that is passed to another part of the program, usually as an argument, to be called when certain conditions are met. For instance in making Ajax requests, it's typical to register a callback for at least success and error situations. A typical feature of callbacks is that the function cannot decide its own parameters, and must instead make do with the arguments given by the part of the program that calls it. Callbacks are also called handlers. One-off callbacks are often defined as anonymous functions.
Piece of software that consumes or utilizes the functionality of a Web API. Some clients are controlled by humans, while others (e.g. crawlers, monitors, scripts, agents) have different degree of autonomy.
In databases, columns define the attributes of objects stored in a table. A column has a type, and can have additional properties such as being unique. If a row doesn't conform with the column types and other restrictions, it cannot be inserted into the table.
  1. Kuvaus
  2. Common keywords
In object relational mapping, column attributes are attributes in model classes that have been initialized as columns (e.g. in SQLAlchemy their initial value is obtained by initializing a Column). Each of these attributes corresponds to a column in the database table (that corresponds with the model class). A column attribute defines the column's type as well as additional properties (e.g. primary key).
  1. Kuvaus
  2. Example
In OpenAPI the components object is a storage for reusable components. Components inside this object can be referenced from other parts of the documentation. This makes it a good storage for any descriptions that pop up frequently, including path parameters, various schemas, and request body objects. This also includes security schemes if your API uses authentication.
Connectedness is a REST principle particularly related to hypermedia APIs. It states that there for each resource in the API, there must exist a path from every other resource to get there by following hypermedia links. Connectedness is easiest to analyze by creating an API state diagram.
Container is a virtualization concept where the virtualization is limited to the software layer. Unlike traditional virtual machines (VM) that virtualize a full computer from the hardware up, containers share operating system resources with each other and only provide an isolated run environment. Every container can define what is installed in its running environment, but they have less overhead than VMs. They are faster to work with and easy to replicate as well. Due to the shared hardware layer, it is possible for malicious containers to break out of their isolation and affect the hardware shared by all container running on the same system.
  1. Kuvaus
  2. Example
A hypermedia control is an attribute in a resource representation that describes a possible action to the client. It can be a link to follow, or an action that manipulates the resource in some way. Regardless of the used hypermedia format, controls include at least the URI to use when performing the action. In Mason controls also include the HTTP method to use (if it's not GET), and can also include a schema that describes what's considered valid for the request body.
  1. Kuvaus
  2. Example
  3. Using
A (URL) converter is a piece of code used in web framework routing to convert a part of the URL into an argument that will be used in the view function. Simple converters are usually included in frameworks by default. Simple converters include things like turning number strings into integers etc. Typically custom converters are also supported. A common example would be turning a model instance's identifier in the URL to the identified model instance. This removes the boilerplate of fetching model instances from view functions, and also moves the handling of Not Found errors into the converter.
The term credentials is used in authentication to indicate the information that identifies you as a specific user from the system's point of view. By far the most common credentials is the combination of username and password. One primary goal of system security is the protection of credentials.
Document Object Model (DOM) is an interface through which Javascript code can interact with the HTML document. It's a tree structure that follows the HTML's hierarchy, and each HTML tag has its own node. Through DOM manipulation, Javascript code can insert new HTML into anywhere, modify its contents or remove it. Any modifications to the DOM are updated into the web page in real time. Do note that since this is a rendering operation, it's very likely one of the most costly operations your code can do. Therefore changing the entire contents of an element at once is better than changing it e.g. one line at a time.
  1. Kuvaus
  2. systemctl
Daemons are processes that run independently on the background and are typically started by the system without any user interaction, or if started manually, they will keep running even if the user logs out as long as the system itself is running. The operating system and other programs can communicate with daemons through sockets, and their output is usually found in log files. Most daemons that are installed from package managers are controlled with systemctl. It's also useful to know that Supervisor allows running non-daemon processes as daemons.
Database schema is the "blueprint" of the database. It defines what tables are contained in the database, and what columns are in each table, and what additional attributes they have. A database's schema can be dumped into an SQL file, and a database can also be created from a schema file. When using object relational mapping (ORM), the schema is constructed from model classes.
  1. Kuvaus
  2. Example
Decorator is a function wrapper. Whenever the decorated function is called, its decorator(s) will be called first. Likewise, when the decorated function returns values, they will be first returned to the decorator(s). In essence, the decorator is wrapped around the decorated function. Decorators are particularly useful in web development frameworks because they can be inserted between the framework's routing machinery, and the business logic implemented in a view function. Decorators can do filtering and conversion for arguments and/or return values. They can also add conditions to calling the view function, like authentication where the decorator raises an error instead of calling the view function if valid credentials are not presented.
In HTML element refers to a single tag - most of the time including a closing tag and everything in between. The element's properties are defined by the tag, and any of the properties can be used to select that element from the document object model (DOM). Elements can contain other elements, which forms the HTML document's hierarchy.
For APIs entry point is the "landing page" of the API. It's typically in the API root of the URL hierarchy and contains logical first steps for a client to take when interacting with the API. This means it typically has one or more hypermedia controls which usually point to relevant collections in the API or search functions.
  1. Kuvaus
  2. Use (Linux)
Environment variables are values that are available in the running environment of a process, and are typically used to give processes information about the specific environment they are running in. This can include both configuration parameters, and information that is gathered from the operating system. They are most visible in shell sessions where all processes that are started from the shell inherit its environment variables but processes running without shell also have them, usually set when the process is started. Although sometimes used to replace configuration, their main purpose is to customize how a process is run temporarily. Using environment variables to store secrets is generally not advised, but sometimes done when other options are not available.
Their names are typically written in uppercase. One of the more notable variables is PATH, which indicates what directories should be searched for when the process tries to invoke an executable.
In software testing, a fixture is a component that satisfies the preconditions required by tests. In web application testing the most common role for fixtures is to initialize the database into a state that makes testing possible. This generally involves creating a fresh database, and possibly populating it with some data. In this course fixtures are implemented using pytest's fixture architecture.
  1. Kuvaus
  2. Creating DB
  3. Starting the App
This term contains basic instructions about setting up and running Flask applications. See the term tabs "Creating DB" and "Starting the App". For all instructions to work you need to be in the folder that contains your app.
In database terminology, foreign key means a column that has its value range determined by the values of a column in another table. They are used to create relationships between tables. The foreign key column in the target table must be unique.
For most hypermedia types, there exists a generic client. This is a client program that constructs a navigatable user interface based on hypermedia controls in the API, and can usually also generate data input forms. The ability to use such clients for testing and prototyping is one of the big advantages of hypermedia.
HTTP method is the "type" of an HTTP request, indicating what kind of an action the sender is intending to do. In web applications by far the most common method is GET which is used for retrieving data (i.e. HTML pages) from the server. The other method used in web applications is POST, used in submitting forms. However, in REST API use cases, PUT and DELETE methods are also commonly used to modify and delete data.
HTTP request is the entirety of the requets made by a client to a server using the HTTP protocol. It includes the request URL, request method (GET, POST etc.), headers and request body. In Python web frameworks the HTTP request is typically turned into a request object.
In computing a hash is a string that is calculated from another string or other data by an algorithm. Hashes have multiple uses ranging from encryption to encoding independent transmission. Hash algorithms can roughly be divided into one- and two-directional. One-directional hashing algorithms are not reversible - the original data cannot be calculated from the hash. They are commonly used to store passwords so that plain text passwords cannot be retrieved even if the database is compromised. Two-directional hashes can be reversed. A common example is the use of base64 to encode strings to use a limited set of characters from the ASCII range to ensure that different character encodings at various transmission nodes do not mess up the original data.
Headers are additional information fields included in HTTP requests and responses. Typical examples of headers are content-type and content-length which inform the receiver how the content should be interpreted, and how long it should be. In Flask headers are contained in the request.headers attribute that works like a dictionary.
Host part is the part of URL that indicates the server's address. For example, lovelace.oulu.fi is the host part. This part determines where (i.e. which IP address) in the world wide web the request is sent.
In API terminology hypermedia means additional information that is added on top of raw data in resource representations. It's derived from hypertext - the stuff that makes the world wide web tick. The purpose of the added hypermedia is to inform the client about actions that are available in relation to the resource they requested. When this information is conveyed in the representations sent by the API, the client doesn't need to know how to perform these actions beforehand - it only needs to parse them from the response.
An idempotent operation is an operation that, if applied multiple times with the same parameters, always has the same result regardless of how many times it's applied. If used properly, PUT is an idempotent operation: no matter how many times you replace the contents of a resource it will have the same contents as it would have if only one request had been made. On the other hand POST is usually not idempotent because it attempts to create a new resource with every request.
  1. Kuvaus
  2. Example
The info object in OpenAPI gives basic information about your API. This basic information includes general description, API version number, and contact information. Even more importantly, it includes license information and link to your terms of service.
Instance folder is a Flask feature. It is intended for storing files that are needed when running the Flask application, but should not be in the project's code repository. Primary example of this is the prodcution configuration file which differs from installation to installation, and generally should remain unchanged when the application code is updated from the repository. The instance path can be found from the application context: app.instance_path. Flask has a reasonable default for it, but it can also be set manually when calling Flask constuctor by adding the instance_path keyword argument. The path should be written as absolute in this case.
  1. Kuvaus
  2. Serializing / Parsing
JavaScript Object Notation (JSON) is a popular document format in web development. It's a serialized representation of a data structure. Although the representation syntax originates from JavaScript, It's almost identical to Python dictionaries and lists in formatting and structure. A JSON document conists of key-value pairs (similar to Python dictionaries) and arrays (similar to Python lists). It's often used in APIs, and also in AJAX calls on web sites.
JSON schema is a JSON document that defines the validity criteria for JSON documents that fall under the schema. It defines the type of the root object, and types as well as additional constraints for attributes, and which attributes are required. JSON schemas serve two purposes in this course: clients can use them to generate requests to create/modify resources, and they can also be used on the API end to validate incoming requests.
  1. Kuvaus
  2. Common MIME types
MIME type is a standard used for indicating the type of a document.In web development context it is placed in the Content-Type header. Browsers and servers the MIME type to determine how to process the request/response content. On this course the MIME type is in most cases application/json.
Microservice is a web architecture concept where a system consists of multiple smaller components called microservices, each exposing an API for communication with other components. Typically one microservice is responsibly for just one feature of the system. The main advantage is that each microservice can be developed indepedently of each other, even with entirely different programming languages and frameworks, and one team can be responsible of one microservce. This reduces side effects from changes that often occur in monolithic systems where the entire system is one application with highly coupled components. Microservices also allow the system to be scaled up in a more granular way and be more fault tolerant. Of course nothing comes wihtout a tradeoff. For microservices it is the increased need and complexity of communication and orchestration of the services.
Database migration is a process where an existing database is updated with a new database schema. This is done in a way that does not lose data. Some changes can be migrated automatically. These include creation of new tables, removal of columns and adding nullable columns. Other changes often require a migration script that does the change in multiple steps so that old data can be transformed to fit the new schema. E.g. adding a non-nullable column usually involves adding it first as nullable, then using a piece of code to determine values for each row, and finally setting the column to non-nullable.
  1. Kuvaus
  2. Example
In ORM terminology, a model class is a program level class that represents a database table. Instances of the class represent rows in the table. Creation and modification operations are performed using the class and instances. Model classes typically share a common parent (e.g. db.Model) and table columns are defined as class attributes with special constuctors (e.g. db.Column).
  1. Kuvaus
  2. Example
In API terminology, namespace is a prefix for names used by the API that makes them unique. The namespace should be a URI, but it doesn't have to be a real address. However, usually it is convenient to place a document that described the names within the namespace into the namespace URI. For our purposes, namespace contains the custom link relations used by the API.
Object relational mapping is a way of abstracting database use. Database tables are mapped to programming language classes. These are usually called models. A model class declaration defines the table's structure. When rows from the database table are fetched, they are represented as instances of the model class with columns as attributes. Likewise new rows are created by making new instances of the model class and committing them to the database. This course uses SQLAlchemy's ORM engine.
OpenAPI (previously: Swagger) is a description language for API documentation. It can be written with either JSON or YAML. An OpenAPI document is a single nested data structure which makes it suitable to be used with various tools. For example, Swagger UI is a basic tool that renders an OpenAPI description into a browsable documentation page. Other kinds of tools include using schemas in OpenAPI description for validation, and generating OpenAPI specification from live code.
  1. Kuvaus
  2. Example
Operation object is one of the main parts of an OpenAPI specification. It describes one operation on a resource (e.g. GET). The operation object includes full details of how to perform the operation, and what kinds of responses can be expected from it. Two of its key parameters are requestBody which shows how to make the request, and responses, which is a mapping of potential responses.
With Flasgger, an operation object can be put into a view method's docstring, or a separate file, to document that particular view method.
Pagination divides a larger dataset into smaller subsets called pages. Search engine results would be the most common example. You usually get 10 or 20 first hits from you search, and then have to request the next page in order to get more. The purpose of pagination is to avoid transferring (and rendering) unnecessary data, and it is particularly useful in scenarios where the relevance of data declines rapidly (like search results where the accuracy drops the further you go). An API that offers paginated data will typically offer access to specific pages with both absolute (i.e. page number) and relative (e.g. "next", "prev", "first" etc.) URLs. These are usually implemented through query parameters.
In OpanAPI a path parameter is a variable placeholder in a path. It is the OpenAPI equivalent for URL parameters that we use in routing. Path parameter typically has a description and a schema that defines what is considered valid for its value. These parameter definitions are often placed into the components object as they will be used in multiple resources. In OpenAPI syntax path parameters in paths are marked with curly braces, e.g. /api/sensors/{sensor}/.
In database terminology primary key refers to the column in a table that's intended to be the primary way of identifying rows. Each table must have exactly one, and it needs to be unique. This is usually some kind of a unique identifier associated with objects presented by the table, or if such an identifier doesn't exist simply a running ID number (which is incremented automatically).
Profile is metadata about a resource. It's a document intended for client developers. A profile gives meaning to each word used in the resource representation be it link relation or data attribute (also known as semantic descriptors). With the help of profiles, client developers can teach machine clients to understand resource representations sent by the API. Note that profiles are not part of the API and are usually served as static HTML documents. Resource representations should always contain a link to their profile.
In database terminology, query is a command sent to the database that can fetch or alter data in the database. Queries use written with a script-like language. Most common is the structured query language (SQL). In object relational mapping, queries are abstracted behind Python method calls.
  1. Kuvaus
  2. Example
Query parameters are additional parameters that are included in a URL. You can often see these in web searches. They are the primary mechanism of passing arbitrary parameters with an HTTP request. They are separated from the actual address by ?. Each parameter is written as a key=value pair, and they are separated from each other by &. In Flask applications they can be found from request.args which works like a dictionary.
  1. Kuvaus
  2. Examples
Regular expressions are used in computing to define matching patterns for strings. In this course they are primarily used in validation of route variables, and in JSON schemas. Typical features of regular expressions are that they look like a string of garbage letters and get easily out of hand if you need to match something complex. They are also widely used in Lovelace text field exercises to match correct (and incorrect) answers.
In this course request referes to HTTP request. It's a request sent by a client to an HTTP server. It consists of the requested URL which identifies the resource the client wants to access, a method describing what it wants to do with the resource. Requests also include headers which provide further context information, and possihby a request body that can contain e.g. a file to upload.
  1. Kuvaus
  2. Accessing
In an HTTP request, the request body is the actual content of the request. For example when uploading a file, the file's contents would be contained within the request body. When working with APIs, request body usually contains a JSON document. Request body is mostly used with POST, PUT and PATCH requests.
  1. Kuvaus
  2. Getting data
Request object is related to web development frameworks. It's a programming language object representation of the HTTP request made to the server. It has attributes that contain all the information contained within the request, e.g. method, url, headers, request body. In Flask the object can be imported from Flask to make it globally available.
in RESTful API terminology, a resource is anything that is interesting enough that a client might want to access it. A resource is a representation of data that is stored in the API. While they usually represent data from the database tables it is important to understand that they do not have a one-to-one mapping to database tables. A resource can combine data from multiple tables, and there can be multiple representations of a single table. Also things like searches are seen as resources (it does, after all, return a filtered representation of data).
Resource classes are introduced in Flask-RESTful for implementing resources. They are inherited from flask_restful.Resource. A resource class has a view-like method for each HTTP method supported by the resource (method names are written in lowercase). Resources are routed through api.add_resource which routes all of the methods to the same URI (in accordance to REST principles). As a consequence, all methods must also have the same parameters.
In this course we use the term representation to emphasize that a resource is, in fact, a representation of something stored in the API server. In particular you can consider representation to mean the response sent by the API when it receives a GET request. This representation contains not only data but also hypermedia controls which describe the actions available to the client.
In this course response refers to HTTP response, the response given by an HTTP server when a request is made to it. Reponses are made of a status code, headers and (optionally) response body. Status code describes the result of the transaction (success, error, something else). Headers provide context information, and response body contains the document (e.g. HTML document) returned by the server.
Response body is the part of HTTP response that contains the actual data sent by the server. The body will be either text or binary, and this information with additional type instructions (e.g. JSON) are defined by the response's Content-type header. Only GET requests are expected to return a response body on a successful request.
Response object is the client side counterpart of request object. It is mainly used in testing: the Flask test client returns a response object when it makes a "request" to the server. The response object has various attributes that represent different parts of an actual HTTP response. Most important are usually status_code and data.
In database terminology, rollback is the cancellation of a database transaction by returning the database to a previous (stable) state. Rollbacks are generally needed if a transaction puts the database in an error state. On this course rollbacks are generally used in testing after deliberately causing errors.
  1. Kuvaus
  2. Routing in Flask
  3. Reverse routing
  4. Flask-RESTful routing
URL routing in web frameworks is the process in which the framework transforms the URL from an HTTP request into a Python function call. When routing, a URL is matched against a sequence of URL templates defined by the web application. The request is routed to the function registered for the first matching URL template. Any variables defined in the template are passed to the function as parameters.
In relational database terminology, row refers to a single member of table, i.e. one object with properties that are defined by the table's columns. Rows must be uniquely identifiable by at least one column (the table's primary key).
SQL (structured query language) is a family of languages that are used for interacting with databases. Queries typically involve selecting a range of data from one or more tables, and defining an operation to perform to it (such as retrieve the contents).
Serialization is a common term in computer science. It's a process through which data structures from a program are turned into a format that can be saved on the hard drive or sent over the network. Serialization is a reversible process - it should be possible to restore the data structure from the representation. A very common serialization method in web development is JSON.
In web applications static content refers to content that is served from static files in the web server's hard drive (or in bigger installations from a separate media server). This includes images as well as javascript files. Also HTML files that are not generated from templates are static content.
Swagger is a set of tools for making API documentation easier. In this course we use it primarily to render easily browsable online documentation from OpenAPI description source files. Swagger open source tools also allow you to run mockup servers from your API description, and there is a Swagger editor where you can easily see the results of changes to your OpenAPI description in the live preview.
In this course we use Flasgger, a Swagger Flask extension, to take render API documentation.
  1. Kuvaus
  2. Creating
System user is an operating system concept, particularly in UNIX systems, for users that exist for the sole purpose of processes taking their identity when run. Unlike normal users, they do not have a password and cannot be logged in as. Their primary purpose is to manage permissions so that each process only has access to resources it actually needs for operation, thus reducing the amount of information that an attacker can access if they are able to take control of the process.
In database terminology, a table is a collection of similar items. The attributes of those items are defined by the table's columns that are declared when the table is created. Each item in a table is contained in a row.
In software testing, test setup is a procedure that is undertaken before each test case. It prepares preconditions for the test. On this course this is done with pytest's fixtures.
In software testing, test teardown is a process that is undertaken after each test case. Generally this involves clearing up the database (e.g. dropping all tables) and closing file descriptors, socket connections etc. On this course pytest fixtures are used for this purpose.
Universal resource identifier (URI) is basically what the name says: it's a string that unambiguously identifies a resource, thereby making it addressable. In APIs everything that is interesting enough is given its own URI. URLs are URIs that specify the exact location where to find the resource which means including protocol (http) and server part (e.g. lovelace.oulu.fi) in addition to the part that identifies the resource within the server (e.g. /ohjelmoitava-web/programmable-web-project-spring-2019).
  1. Kuvaus
  2. Type converters
  3. Custom converters
URL template defines a range of possible URLs that all lead to the same view function by defining variables. While it's possible for these variables to take arbitrary values, they are more commonly used to select one object from a group of similar objects, i.e. one user's profile from all the user profiles in the web service (in Flask: /profile/<username>. If a matching object doesn't exist, the default response would be 404 Not Found. When using a web framework, variables in the URL template are usually passed to the corresponding view function as arguments.
Uniform interface is a REST principle which states that all HTTP methods, which are the verbs of the API, should always behave in the same standardized way. In summary:
  • GET - should return a representation of the resource; does not modify anything
  • POST - should create a new instance that belongs to the target collection
  • PUT - should replace the target resource with a new representation (usually only if it exists)
  • DELETE - should delete the target resource
  • PATCH - should describe a change to the resource
In database terminology, unique constraint is a what ensures the uniqueness of each row in a table. Primary key automatically creates a unique constraint, as do unique columns. A unique constraint can also be a combination of columns so that each combination of values between these columns is unique. For example, page numbers by themselves are hardly unique as each book has a first page, but a combination of book and page number is unique - you can only have one first page in a book.
  1. Kuvaus
  2. Registering
View functions are Python functions (or methods) that are used for serving HTTP requests. In web applications that often means rendering a view (i.e. a web page). View functions are invoked from URLs by routing. A view function always has application context.
  1. Kuvaus
  2. Creation
  3. Activation
A Python virtual environment (virtualenv, venv) is a system for managing packages separately from the operating system's main Python installation. They help project dependency management in multiple ways. First of all, you can install specific versions of packages per project. Second, you can easily get a list of requirements for your project without any extra packages. Third, they can placed in directories owned by non-admin users so that those users can install the packages they need without admin privileges. The venv module which is in charge of creating virtual environments comes with newer versions of Python.
Web Server Gateway Interface, WSGI (pronounced whiskey because no one wants to read that abbreviation aloud) is a Python specification that defines how web servers can communicate with Python applications so that an HTTP request gets converted into a Python function call, and the return value of the call is converted into an HTTP response. Its main purpose is to make it easy to create web applications with Python and have them work uniformly. It also has an asynchronous cousin in ASGI if you feel like your application will benefit from using an asynchronous web framework (like FastAPI).
Interface, implemented using web technologies, that exposes a functionality in a remote machine (server). By extension Web API is the exposed functionality itself.
Web server is an application that listens to HTTP and HTTPS traffic, and defines how it is responded to. Typical behaviors can include serving a static file, routing the request to a web application, or routing it to another web server. Web servers can also function as load balancers that distribute traffic to multiple server nodes in order to be able to serve a higher amount of clients simultaneously. When deploying web applications behind web servers, typically the web server takes care of handling encryption, provided the application runs on the same machine as the web server. Presently, the most common web servers are Apache and NGINX.
  1. Kuvaus
  2. Example
YAML (YAML Ain't Markup Language) is a human-readable data serialization language that uses a similar object based notation as JSON but removes a lot of the "clutter" that makes JSON hard to read. Like Python, YAML uses indentation to distinguish blocks from each other, although it also supports using braces for this purpose (which, curiously enough, makes JSON valid YAML). It also removes the use of quotation characters where possible. It is one of the options for writing OpenAPI descriptions, and the one we are using on this course.