In database terminology primary key refers to the column in a table that's intended to be the primary way of identifying rows. Each table must have exactly one, and it needs to be unique. This is usually some kind of a unique identifier associated with objects presented by the table, or if such an identifier doesn't exist simply a running ID number (which is incremented automatically).
Learning outcomes and material¶
During this exercise students will learn how to implement a RESTful API utilizing Flask Web Framework. Students will learn also how to test the API by reading the testing tutorial We expect that you follow the same process to complete the Deliverable 4.
The slides presenting the content of the lecture can be downloaded from the following link:
Implementing REST APIs with Flask¶
This exercise material covers how to implement REST APIs using Flask-RESTful, a Flask extensions that exists specifically for this purpose (in case you didn't figure that out from the name). We will also discuss how to handle
hypermedia
in your implementation in a manageable way. The material has examples for both single file applications, and applications that use the the project layout we proposed. For exercise tasks you need to submit single file applications. However for your course project we recommend following the more elaborate project structure.Introduction to Flask-Restful¶
In the first part of the exercise we'll cover how to use the RESTful extension. In the examples we are going back to the sensorhub example from the first material. The code is a bit more straightforward then the one in the API server we designed in exercise 2 so this makes for less complex examples. As a very brief recap, we had four key concepts: measurements, sensors, sensor locations and deployment configurations. We'll implement some of these into
resources
as this example goes on.Learning goals: Learn the basics of Flask-RESTful: how to define
resource classes
and implement the HTTP methods
of resources
. Learn how to define routes
for resources, and about reverse routing for building URIs
.Installing¶
Some new modules are needed for this exercise. Fire up your
virtual environment
and cast the following spells (the last one is not needed for this section but it will be for the next one):pip install flask-restful pip install Flask-SQLAlchemy pip install jsonschema
A Resourceful Class¶
Flask-RESTful defines a class called Resource. Much like Model was the base class for all
models
in our database, Resource is the base class for all our resources
. A resource class
should have a method for each HTTP method
it supports. These methods must be named the same as the corresponding HTTP method, in lowercase. For instance a collection type resource will usually have two methods: get and post. These methods are very similar in implementation to view functions
. However they do not have a route decorator - their route
is based on the resource's route instead. Let's say we want to have two resource classes for sensors: the list of sensors, and then individual sensor where we can also see its measurements. The resource class skeletons would look like this:from flask_restful import Resource
class SensorCollection(Resource):
def get(self):
pass
def post(self):
pass
class SensorItem(Resource):
def get(self, sensor):
pass
def put(self, sensor):
pass
def delete(self, sensor):
pass
We're using SensorItem for individual sensors instead of just Sensor, mostly because we already used Sensor for the
model
and this would cause conflicts if everything was placed in a single file. If you want to pursue that path, simply place these classes inside your application module that has the models. However, if you followed the project layout material, these classes should be placed in a new module (e.g. sensor.py) inside the resources subfolder (also make sure there's a file called __init__.py
in the folder - it can be empty, but must exist for Python to recognize the folder as a package).The methods themselves are just like
views
. For example, here's a post method for SensorCollection that looks very similar to the last version of the add_measurement view in exercise 1. def post(self):
if not request.json:
abort(415)
try:
sensor = Sensor(
name=request.json["name"],
model=request.json["model"],
)
db.session.add(sensor)
db.session.commit()
except KeyError:
abort(400)
except IntegrityError:
abort(409)
return "", 201
Do note that all methods must have the same parameters because they all are served from the same resource
URI
! You can, however, have different query parameters
between these methods. For example, this would be typical for resources that have some filtering or sorting support in their get method using query parameters.Resourceful Routing¶
In order for anything to work in Flask-RESTful we need to initialize an API object. This object will handle things like
routing
for us. To proceed with our example, we'll show you how to create this object, and how to use it to register routes to the two resource classes
. In a single file app the process is very simple: import Api from flask_restul, and create an instance of it:from flask import Flask
from flask_restful import Api
app = Flask(__name__)
api = Api(app)
Assuming your resource classes are in the same file, you can now add routes to them by dropping these two lines at the end of the file.
api.add_resource(SensorCollection, "/api/sensors/")
api.add_resource(SensorItem, "/api/sensors/<sensor>/")
Now you could send GET and POST to
/api/sensors/
, and likewise GET, PUT and DELETE to e.g. /api/sensors/uo-donkeysensor-4451/
. Not that they'd do much (except for POST to sensors collection which we just implemented). Extra note: When using a more elaborate project structure, resources should be routed in the
api.py
file which in turn imports the resources from their individual files. Here's the sample api.py
file, which assumes the resource classes were saved to sensor.py
in the resources folder.from flask import Blueprint
from flask_restful import Resource, Api
api_bp = Blueprint("api", __name__, url_prefix="/api")
api = Api(api_bp)
# this import must be placed after we create api to avoid issues with
# circular imports
from sensorhub.resources.sensor import SensorCollection, SensorItem
api.add_resource(SensorCollection, "/sensors/")
api.add_resource(SensorItem, "/sensors/<sensor>/")
@api_bp.route("/"):
def index():
return ""
Even More Resourceful Routing¶
When it comes to
addressability
, it only states that each resource
must be uniquely identifiable by its address. It doesn't say it can't have more than one address. Sometimes it makes sense that the same resource can be found in multiple locations in the URI
hierarchy. Consider video games for example that usually have separate developer and publisher. In that case both of these URI templates
would make equal sense:/api/publishers/{publisher}/games/{game}/
/api/developers/{developer}/games/{game}/
Both are different ways to identify the same resource. Luckily Flask-RESTful allows the definition of multiple
routes
for each resource. These would be routed as follows:api.add_resource(GameItem,
"/api/publishers/<publisher>/games/<game>/",
"/api/developers/<developer>/games/<game>/"
)
Do note that if you route like this, the resource's methods must now take into account the fact that they do not always receive the same keyword arguments: they will receive either publisher or developer. In this scenario, the GameItem resource could have a get method that starts like this:
class GameItem(Resource):
def get(game, publisher=None, developer=None):
if publisher is not None:
game_obj = Game.query.join(Publisher).filter(
Game.title == game, Publisher.name == publisher
).first()
elif developer is not None:
game_obj = Game.query.join(Developer).filter(
Game.title == game, Developer.name == developer
).first()
You can also utilize multiple routes to implement several similar resources using the same
resource class
. The MusicMeta API did this, using the same album and track resource classes for both single artist and VA albums. Actually there was no separate VA route, we just used "VA" as a special value for artist in e.g. the following routing:api.add_resource(AlbumItem, "/api/artists/<artist>/albums/<album>/")
Reverse Routing¶
One more feature that we will soon be using a lot is the ability to generate a
URI
from the routing
. In hypermedia
resource URIs will be repeated a lot and hardcoding them is asking for trouble. It's much better to do reverse routing with api.url_for
. Going back to our sensorhub example, this is how you should always retrieve the URI of a) sensors collection and b) specific sensor item:collection_uri = api.url_for(SensorCollection)
sensor_uri = api.url_for(Sensor, sensor="uo-donkeysensor-4451")
The function finds the first route that matches the
resource class
and given variables, or raises BuildError if no matching route is found. If found, the URI is returned as a string. Our two examples would generate:/api/sensors/ /api/sensors/uo-donkeysensor-4451/
Please bear in mind that if you separate resources into multiple modules, using Flask's basic routing function makes life much easier as explained here.
Generating Hypermedia¶
Moving from
view functions
to resource classes
with methods for handling actions as HTTP methods
was the more straightforward part of this exercise. Creating a hypermedia
API is a somewhat bigger effort. The key to not breaking your hypermedia responses with changes in code is to avoid hardcoding hypermedia in your resource methods. In this section we will show you some ways how to do that through examples. If you'd rather achieve the same goal in some other way, we're not gonna hold it against you - as long you as achieve it.Learning goals: How to use dictionary subclasses for creating hypermedia controls. How to include
JSON schema
in hypermedia responses.Subclass Solution¶
In Mason the root type of a hypermedia response is
JSON
object which - as we have learned - is in most ways the equivalent of a Python dictionary. However if you go and define the entire response as a dictionary in each resource method separately, the likelihood of introducing inconsistencies is quite high. Furthermore the code becomes cumbersome to maintain. For any applications that produce JSON, a good development pattern is to create a dictionary subclass that includes a number of convenience methods that automatically manage integrity of the selected JSON format. As stated before, our chosen hypermedia type for examples in this course is Mason. There are three special attributes in Mason JSON documents that we use commonly:
"@controls"
, "@namespaces"
and "@error"
. Just to give you an idea of what we're trying to avoid, here's how we would need to make a Mason document with one namespace
and control
with normal dictionaries (this example could be part of our Sensor resource's get method): body = {}
body["@namespaces"] = {
"senhub": {
"name": "/sensorhub/link-relations/#"
}
}
body["@controls"] = {
"senhub:measurements": {
"href": api.url_for(MeasurementCollection, sensor=sensor_name)
}
}
# add some data about the sensor etc
Putting stuff like this - and usually in bigger numbers - is incredibly messy. What we want to achieve is, instead, something like this:
body = MasonBuilder()
body.add_namespace("senhub", "/sensorhub/link-relations/#")
body.add_control("senhub:measurements", api.url_for(MeasurementCollection, sensor=sensor_name))
# add some data about the sensor
Without doubt this looks much cleaner. The MasonBuilder class would take care of details about how exactly to add the namespace and control into the resulting document. If something about that changed, making the change in the class would also apply the change to all resource methods. So, how does this look on the class itself? Something like this:
class MasonBuilder(dict):
def add_namespace(self, ns, uri):
if "@namespaces" not in self:
self["@namespaces"] = {}
self["@namespaces"][ns] = {
"name": uri
}
def add_control(self, ctrl_name, href, **kwargs):
if "@controls" not in self:
self["@controls"] = {}
self["@controls"][ctrl_name] = kwargs
self["@controls"][ctrl_name]["href"] = href
Observe how
MasonBuilder
extends the dict Python class, so the way of creating a MasonBuilder
is exactly the same to create a dictionary using the dict
class.Implementation detail: if you have not seen
**kwargs
used before, this is a Python feature called packing/unpacking. It's a wildcard catch for keyword arguments given to the function/method when it's called: all such arguments will be packed into the kwargs dictionary. So when we call this method with method="POST"
the kwargs will be end up like this: {"method": "POST"
}. This feature is also used in the dict __init__ method (which we inherit), you can give it keyword arguments to initialize it with a bunch of keys.Because each object should have only one
"@controls"
and "@namespaces"
attributes, it makes sense to automatically create this when the first namespace/control is added. We can add a similar method for the "@error"
attribute: def add_error(self, title, details):
self["@error"] = {
"@message": title,
"@messages": [details],
}
You can download the entire class with docstrings added from below. If you're using the more elaborate project structure, this class is something that definitely belongs into the
utils.py
file and should be imported to other modules with from sensorhub.utils import MasonBuilder
.Since items in a colletion type
resource
can have their controls, you should construct them as MasonBuilder instances instead of dictionaries. This way you can add controls
to them just as effortlessly as you can to the root object. Let's take an example of how to add the very important "self" relation
to each sensor in the sensors collection resource representation
. body = MasonBuilder(items=[])
for sensor in Sensor.query.all():
item = MasonBuilder(
name=sensor.name,
model=sensor.model
)
item.add_control("self", api.url_for(Sensor, sensor=sensor.name))
body["items"].append(item)
API Specific Subclasses¶
A generic
JSON
builder like the one we just implemented is nice, but doesn't solve the whole equation. Especially the add_control method is only truly adequate for GET methods. If we want to add a DELETE method, we would need to include more stuff into the method call - stuff that we want to avoid having to repeat:body.add_control("senhub:delete", api.url_for(Sensor, sensor=sensor_name), method="DELETE")
Granted it's not a lot of extra typing involved here, but the fact that the correct
HTTP method
for deleting is DELETE is already incorporated in the link relation
definition for "senhub:delete"
, so in a way we're typing the same information twice here. What's more desirable is probably something like this:body.add_control_delete_sensor(sensor_name)
Because all controls that delete a sensor look alike, we have now hidden all the boring details inside the method, and parametrized the only difference between each individual control: the identifier of the sensor this control should delete. Even building the
URI
should be hidden because it's always done in the same way. For implementing this method and many more alike, we should define a new class that subclasses MasonBuilder - this way our generic builder stays intact. This new class, with this one method:class SensorhubBuilder(MasonBuilder):
def add_control_delete_sensor(self, sensor):
self.add_control(
"senhub:delete",
href=api.url_for(Sensor, sensor=sensor),
method="DELETE",
"title"="Delete this sensor"
)
With this we have now ensured that every
control
to delete a sensor will always be the same. We were even able to add the optional title attribute without adding more noise to resource methods. Or if you don't want to have a method to delete each different type of resource you could make something like this for a more universal delete control:class SensorhubBuilder(MasonBuilder):
def add_control_delete(self, href):
self.add_control(
"senhub:delete",
href=href,
method="DELETE",
title="Delete this resource"
)
Now the URI needs to be built on the calling end, and our title attribute is slightly less descriptive. Still, this solution has similar ease of maintenance as the more accurate one above. You can do a similar treatment to any number of controls in your API, and you probably should do so for the majority of them.
Dynamic Schemas, Static Methods¶
In exercise 2 we sung the praises of adding
JSON schemas
to our hypermedia
controls
. Schemas do have a nasty drawback: they are awfully verbose. If a control with like three attributes was already deemed something we'd rather not repeat in our code unnecessarily, a schema that's easily over ten lines of code is definitely something that must be written in only one place. It's also worth recalling that we have two uses for them: serialize
them as parts of controls, and also to use them for validating request bodies
sent by the client. Schemas are generally needed for POST and PUT controls, and also any methods that use
query parameters
. However, the same schema is often referenced at least twice (POST and PUT) so it should not be hardcoded into any single add_control method. In fact it should not be put into a normal method at all, because sometimes we may want to retrieve the schema without having an instance of SensorhubBuilder available. Instead it should be returned by a static method, or built into a class attribute. We'll prefer static method just in the off-chance that one day some of these might need parametrization. Let's add a static method that builds sensor schema and returns it: @staticmethod
def sensor_schema():
schema = {
"type": "object",
"required": ["name", "model"]
}
props = schema["properties"] = {}
props["name"] = {
"description": "Sensor's unique name",
"type": "string"
}
props["model"] = {
"description": "Name of the sensor's model",
"type": "string"
}
return schema
Implementation detail: a static method is a method that can be called without an instance of the class and it also doesn't usually refer to any of the class attributes (that's what class methods are for). In other words it's actually a function that's just been slapped on a class to keep things more organized. It can be called as
self.sensor_schema()
from normal methods within the same class. From the outside it's called as SensorhubBuilder.sensor_schema()
.This static method could now be used in a control for creating new sensors (and similarly for modifying sensors):
def add_control_add_sensor():
self.add_control(
"senhub:add-sensor",
"/api/sensors/",
method="POST",
encoding="json",
title="Add a new sensor",
schema=self.sensor_schema()
)
Another possible location that makes sense for static methods that return a schema would be the corresponding
model class
. If you get really into it, you can even generate the schema from the model. Here's a starting point if you want to look into it, or you could just write your own. Responses and Errors¶
We briefly discussed Flask's
response object
in the Resource Locator task where we used it to set custom headers
. We have now arrived at a stage where we should actually be using it for all responses. This is largely because we need to announce the content type of our responses
, and this is done by using the mimetype keyword argument. Because we're using Mason, we need to set it to "application/vnd.mason+json"
. Since this will be repeated in every GET method, it'd be wise to make a constant of it (i.e. MASON = "application/vnd.mason+json"
). From now on a typical 200 response would look like:return Response(json.dumps(body), 200, mimetype=MASON)
We went back to using json.dumps because Response takes the
response body
as a string. We can also change all 201 and 204 responses accordingly. We already learned how to do the 201 response with Location header, and a 204 response is even simpler: return Response(status=204)
We have now resolved issues regarding responses in the 200 range (i.e. successful operations). What about errors in the 400 range? Mason also defines what errors should look like. In fact, we actually already implemented the add_error method into our MasonBuilder dictionary subclass. However, even with that, returning an error becomes a multiline effort, and we don't really want that because most of it is boilerplate and resource methods typically return errors at multiple points of their execution. Let's make a convenience function for generating errors:
def create_error_response(status_code, title, message=None):
resource_url = request.path
body = MasonBuilder(resource_url=resource_url)
body.add_error(title, message)
body.add_control("profile", href=ERROR_PROFILE)
return Response(json.dumps(body), status_code, mimetype=MASON)
This generates a Mason error messages with a title, and one message with more description about the problem. It also puts the resource URL into the response body, just in case the client forgot what it was trying to do (sometimes actually relevant, e.g. asynchronous use cases). Now instead of writing all that whenever an error is encountered in a resource method, we can just write:
return create_error_response(404, "Not found", "No sensor was found with the given name")
Static Parts of Hypermedia¶
In addition to generating
hypermedia
representations for resources
, a fully functional hypermedia API should also serve some static content. Namely: link relations
and resource profiles
. Also if you have particularly large schemas and would rather serve them separately from resource representations
, these should also be served as static content. For profiles and link relations you have two options: you can send them out as static files, or you can redirect to apiary.As Static Files¶
If your project doesn't have a static folder yet, now's the time to create one. It's also recommended to create some subfolders to keep things organized, e.g.
static ├── profiles └── schema
In order to use a static folder, it must be registered with Flask. This is done when initializing the app:
app = Flask(__name__, static_folder="static")
Static views are
routed
with @app.route
. If you are storing profiles and such locally as html files, you can implement these views quite easily by using Flask's send_from_directory
function which sends the contents of a file as the response body
- you should add it to your growing from flask import
line. With profiles you can use one route definition and view function for all the profiles, like this:@app.route("/profiles/<resource>/")
def send_profile_html(resource):
return send_from_directory(app.static_folder, "{}.html".format(resource))
The
send_from_directory
function is convenient enough that it will send a 404 response if the file is not found. Because link relation descriptions are not particularly lengthy they can be gathered into a single file, served in a similar manner:@app.route("/sensorhub/link-relations/")
def send_link_relations_html():
return send_from_directory(app.static_folder, "links-relations.html")
If you are using schema files, you can send them out in a similar manner. You will also need these URLs often in your code since almost all responses will include the
namespace
which requires the link relation URL, and likewise all resource representations have at least one profile link. Therefore you should probably at least introduce them as constants in your code, e.g.SENSOR_PROFILE = "/profiles/sensor/"
MEASUREMENT_PROFILE = "/profiles/measurement/"
LINK_RELATIONS_URL = "/sensorhub/link-relations/"
If you are using the more elaborate project structure, consider putting these into their own file, e.g.
constants.py
. As Redirects¶
You can also link to your apiary documentation by using redirect. Like
send_from_directory
, redirect
is also imported from Flask. Likewise, these should be routed as normal routes. For example, to link to link relations in Apiary:APIARY_URL = "https://yourproject.docs.apiary.io/#reference/"
@app.route("/sensorhub/link-relations/")
def redirect_to_apiary_link_rels():
return redirect(APIARY_URL + "link-relations")
This should take the viewer to the Apiary documentation, and to the Link Relations heading. A caveat of this approach is that if your profiles do not have any methods, they will not have anchorable headings. You can use the same constants as we did with static files for these URLs.
Did You GET It?¶
We have spent quite a while doing various utilities but how does all of this actually look like in
resource class
methods? Let's look at what the GET method of a single sensor would look like in this brand new world. class SensorItem(Resource):
def get(self, sensor):
db_sensor = Sensor.query.filter_by(name=sensor).first()
if db_sensor is None:
return create_error_response(404, "Not found",
"No sensor was found with the name {}".format(sensor)
)
body = SensorhubBuilder(
name=db_sensor.name,
model=db_sensor.model,
location=db_sensor.location and db_sensor.location.name
)
body.add_namespace("senhub", LINK_RELATIONS_URL)
body.add_control("self", api.url_for(SensorItem, sensor=sensor))
body.add_control("profile", SENSOR_PROFILE)
body.add_control_delete_sensor(sensor)
body.add_control_modify_sensor(sensor)
body.add_control_add_measurement(sensor)
body.add_control("senhub:measurements",
api.url_for(MeasurementCollection, sensor=sensor)
)
if db_sensor.location is not None:
body.add_control("senhub:location",
api.url_for(LocationItem, location=db_sensor.location.sensor)
)
return Response(json.dumps(body), 200, mimetype=MASON)
Implementation note: the use of and in getting the location's name circumvents the need for conditional statement to check whether the sensor has a location or not. The evaluation of this statement ends immediately if the left hand operand is equivalent to False, which avoids the AttributeError we would otherwise get from the right hand side.
This looks rather pleasant. Just this one time, let's take a look at what we have avoided doing (deliberately avoiding every single nicety we've introduced so far):
class SensorItem(Resource):
def get(self, sensor):
db_sensor = Sensor.query.filter_by(name=sensor).first()
if db_sensor is None:
error = {
"resource_url": request.path,
"@error": {
"@message": "Not found",
"@messages": ["No sensor was found with name {}".format(sensor)]
}
}
return Response(json.dumps(error), 404, mimetype="application/vnd.mason+json")
body = {
"name": db_sensor.name,
"model": db_sensor.model,
"location": db_sensor.location and db_sensor.location.name
}
body["@namespaces"] = {
"senhub": {"name": "/sensorhub/link-relations/#"}
}
body["@controls"] = {
"self": {"href": "/api/sensors/{}/".format(sensor)},
"profile": {"href": "/profiles/sensor/"},
"senhub:delete": {
"href": "/api/sensors/{}/".format(sensor),
"method": "DELETE",
"title": "Delete this sensor."
},
"edit": {
"href": "/api/sensors/{}/".format(sensor),
"method": "PUT",
"encoding": "json",
"title": "Delete this sensor.",
"schema": {
"type": "object",
"required": ["name", "model"]
"properties": {
"name": {
"description": "Sensor's unique name",
"type": "string"
},
"model": {
"description": "Name of the sensor's model",
"type": "string"
}
}
}
},
"senhub:add-measurement": {
"href": "/api/sensors/{}/measurements/".format(sensor),
"method": "POST",
"encoding": "json",
"title": "Add a new measurement for this sensor",
"schema": {
"type" "object",
"required": ["value"],
"properties": {
"value": {
"description": "Measured value.",
"type": "number"
},
"time": {
"description": "Measurement timestamp",
"type": "string",
"pattern": "^[0-9]{4}-[01][0-9]-[0-3][0-9]T[0-9]{2}:[0-5][0-9]:[0-5][0-9]Z$"
}
}
}
},
"senhub:measurements": {
"href": "/api/sensors/{}/measurements/".format(sensor),
"title": "Measurements from this sensor"
}
}
if db_sensor.location is not None:
body["@controls"]["senhub:location"] = {
"href": "/api/locations/{}/".format(db_sensor.location.name),
"title": "This sensor's location."
}
return Response(json.dumps(body), 200, mimetype="application/vnd.mason+json")
Imagine a program full of methods like this one. Just don't do it late at night, might lose your sleep.
If you want to make an even leaner GET method, you can look into generating the data part of
resource representation
automatically from your models. One library that does this is Marshmallow. You can also go for a simpler option and write a serialize
method for your model classes
. The upside of this approach is that it allows you to maintain all data related code in the same place (i.e. the model). If you need to change your underlying models, there will be much less looking around to find places that need to be modified to conform with the change.POST-it¶
Just to also show you the other side of the coin, here's the POST method for creating new sensors from the SensorCollection
resource
. Note the use of JSON Schema
for validation of the request body
. This requires an additional import as well: from jsonschema import validate, ValidationError
. One nice bonus is that jsonschema gives rather detailed validation errors, and we can conveniently return them to the client with just str(e)
. class SensorCollection(Resource):
def post(self):
if not request.json:
return create_error_response(415, "Unsupported media type",
"Requests must be JSON"
)
try:
validate(request.json, Sensor.get_schema())
except ValidationError as e:
return create_error_response(400, "Invalid JSON document", str(e))
sensor = Sensor(
name=request.json["name"],
model=request.json["model"],
)
try:
db.session.add(sensor)
db.session.commit()
except IntegrityError:
return create_error_response(409, "Already exists",
"Sensor with name '{}' already exists.".format(request.json["name"])
)
return Response(status=201, headers={
"Location": api.url_for(SensorItem, sensor=request.json["name"])
})
Anna palautetta
Kommentteja materiaalista?