Using the Data Access Object Pattern in a Microservice Architecture

Everett GriffithsSeptember 8th, 2017Last Updated: September 8th, 2017

0 52 7 minutes read

This article assumes that you have a general understanding of what microservice architecture is and why you would want to use it. The focus of the discussion here will be the Data Access Object design pattern and how it can be used to communicate with multiple data sources. Since the formal definitions of a design pattern in textbooks tend to be more academic than practical, we need to focus on something a bit more substantial than a fancy name for a class that fetches our data.

This article will demonstrate a solution using PHP and dependency injection to straddle multiple data sources in a scalable and testable way. Similar results can be obtained in other languages or using different organizational approaches.

Our Use Case: Retrieving Data from Multiple Sources

Let’s imagine that our application has outgrown its MySQL database, and now it needs to retrieve user data from a new JSON API. The catch is that some of the data still lives in MySQL so business reports and accounting can still run, but all of the ancillary user data is now housed in a user microservice app.

The legacy model class

If we are lucky, our legacy code uses some dependency injection and interfaces that look something like this:

class UserModel implements UserModelContract
{
    protected $db; 

    public function __construct(mysqli $db)
    {
        $this->db = $db;
    }
   
    public function getUser($username)
    {
        $res = $this->db->query($sql_w_joins);
        return $res->fetch_object();
    }
}

Where the $sql_w_joins is a big query that joins on various related tables to get all the relevant data needed to represent a user object.

If our luck holds, our legacy code uses a service container to bind an interface to an implementation, as does Laravel:

$this->app->bind('mysqli', function ($app) {
    new mysqli("example.com", "user", "password", "database");
});
$this->app->bind('UserModelContract', function ($app) {
    return new UserModel($app->make('mysqli'));
});

A bad way to update the model

Given legacy code like that, the challenge is to introduce the following changes:

Adjust the database lookup so it only queries the primary table.
Query the new API for supplemental user data.
Merge the data together.

Some developers may be tempted to slice open the old model class, inject another dependency, and modify the output with code like this:

// DON'T DO THIS!!!
class BadUserModel implements UserModelContract
{
    protected $db; 
    protected $client;
    
    public function __construct(mysqli $db, ApiClient $client)
    {
        $this->db = $db;
        $this->client = $client;
    }
   
    public function getUser($username)
    {
        $res = $this->db->query($sql_w_joins);
        $db_result = $res->fetch_object();
        $api_result = $this->getUserFromApi($username);
        return $this->mergeData($db_result, $api_result);
    }
    
    protected function getUserFromApi($username) {...}
        
    protected function mergeData($db_result, $api_result) {...}
}

To be fair, this approach would work. You can even inject mock classes to test your result. However, we must be a bit more philosophical and think about the core tenets of clean code. Our class now does many things — it violates the Single Responsibility Principle (SRP) because it now has many reasons to change! If we extrapolate a bit further, we realize that this approach is not scalable: what happens if we later need to retrieve even more data from even more data sources? We can’t keep injecting dependencies into a single class ad infinitum — after three dependencies or so, our class starts looking pretty unhealthy.

Let’s structure this a bit differently.

A better way to update the model: the stack

The trick we are going to do here is to chain multiple implementations of the same interface together and treat them as a whole. Instead of a single model class, we will have many classes (as many as we need) that work together to fulfill the duties of the model. Here’s how to do it.

First, let’s make a modified copy of our old UserModel — we are going to make a few small changes to it to adapt it to the new architecture:

Adjust its SQL query so it only selects the minimal data it needs (and takes the load off of MySQL).
Pass a second argument to the getUser() method to hold the $result as it is passed from class to class.
Inject a dependency for another instance of the UserModelContract interface.

Our new class looks something like this:

class UserDaoDb implements UserModelContract
{
    protected $dao;
    protected $db; 

    public function __construct(UserModelContract $dao, mysqli $db)
    {
        $this->dao = $dao;
        $this->db = $db;
    }
   
    public function getUser($username, $result = null)
    {
        $res = $this->db->query($simple_sql);
        $result = $res->fetch_object();
        return $this->dao->getUser($username, $result);
    }
}

The trick to chain together multiple instances comes with that return statement: we call the same method on the injected UserModelContract instance. What will implement that? If you guessed “the new User API,” you are correct.

Let’s look at the service container for how those are chained together. As a convention, I have named the new classes using Dao plus the name of their data source.

$this->app->bind('mysqli', function ($app) {
    return new mysqli("example.com", "user", "password", "database");
});
$this->app->bind('ApiClientContract', function ($app) {
    return new ApiClient();
});
$this->app->bind('UserModelContract', function ($app) {
    $dao = new UserDaoApi($app->make('ApiClientContract');
    return new UserDaoDb($dao, $app->make('mysqli'));
});

Take a moment to let that sink in: when we resolve the UserModelContract, we don’t get a single class as the implementation — in effect, we get a “stack” of them; the first one calls the second, etc., for as many classes as we need. The only caveat is that the method doesn’t return its data directly. It instead calls the same method on the next DAO class in the lineup.

With this approach, each class stays dedicated to performing one task on one data source. If we need to add another data source, we can create a new implementation of the UserModelContract interface and reference it in our provider. None of our previous code needs to change.

The final DAO in the stack

If you were paying really close attention, you would realize that there is a problem. The last DAO class in the stack would simply return the result. If you needed to add another data source to the stack, you would have to edit the class that came before it in the lineup and make it call the next DAO class instead of simply returning its data. But forcing edits like that would violate the open/closed principle: changing one class shouldn’t require changes in another.

A low-brow solution to this problem is to create an implementation of the interface whose job it is simply to be the last in the stack and return the result:

class UserDaoFinal implements UserModelContract
{
    public function getUser($username, $result = null)
    {
        return $result;
    }
}

Every DAO in your stack except the final one should have a dependency for another instance of the interface, and every method should call the next DAO in line instead of returning its result directly.

class UserDaoApi implements UserModelContract
{
    protected $dao;
    protected $client; 

    public function __construct(UserModelContract $dao, ApiClientContract $client)
    {
        $this->dao = $dao;
        $this->client = $client;
    }
   
    public function getUser($username, $result = null)
    {
        $apiResponse = $this->client
            ->getUserByUsername($username);
        $result = array_merge($result, $apiResponse);
        // Call the next DAO in the line...
        return $this->dao->getUser($username, $result);
    }
}

Now you can add as many additional DAO implementations as you need. As long as the last one is fixed as the UserDaoFinal class, you can freely change everything upstream. Make sure that you wire it up in your service provider something like this:

$this->app->bind('UserModelContract', function ($app) {
    $dao = new UserDaoFinal();
    $dao = new UserDaoApi($dao, $app->make('ApiClientContract');
    return new UserDaoDb($dao, $app->make('mysqli'));
});

Note that the order is important — the last DAO class that you return will be called first.

Defining a Data Transfer Object

You may have noticed that our examples still have a problem: we never really settled on what the $result variable was. What if the database query returns an array and the API lookup returns an object? How can we unify the format of this variable?

The sidekick to the DAO pattern is the Data Transfer Object (DTO). DTOs are not always mentioned in the DAO reference documents, but the idea is that we can type-hint our result and have a well-defined and consistent variable structure to pass through the stack. It is important that this DTO have no functionality (except perhaps getters and setters). Remember, it’s just a vessel.

For simplicity, we’ll define a simple class, but if you want the flexibility of using the JSON schema to define your DTO objects (eg, before returning them via an API), then you can try a PHP package I authored: dto/dto.

Here’s what our UserDto might look like:

class UserDto
{
    public $username;
    public $email;
    public $about;
    public $avatar;
    // ... etc ...
}

Now your methods in the UserModelContract implementations can type-hint this class as the result type. The code in the methods can “map” data onto this DTO object. Some properties will be filled with data from one data source, other properties will get populated with data from another data source.

Here’s what our UserDaoDb class might look like now:

public function getUser($username, UserDto $dto = null)
{
    $dto = ($dto) ? $dto : new UserDto();
    $res = $this->db->query($simple_sql);
    $obj = $res->fetch_object();  
    $dto->username = $username;
    $dto->email = $obj->email;
    // ... etc ...
    return $this->dao->getUser($username, $dto);
}

And the same method in our UserDaoApi class might look like this:

public function getUser($username, UserDto $dto = null)
{
    $dto = ($dto) ? $dto : new UserDto();
    $apiResponse = $this->client->getUserByUsername($username);
    $dto->about = $apiResponse->getAbout();
    $dto->avatar = $apiResponse->getAvatar(); 
    // ... etc ...
    return $this->dao->getUser($username, $dto);
}

You can see how the username and email properties were populated with data from the database whereas the API provided values for the about and avatar properties. All the classes in the stack can add data to the DTO because it is passed into each method. You can see why sometimes the DAO/DTO pattern is referred to as a decorator pattern (even if that may not be accurate).

Summary

This article has demonstrated one way to split code execution across multiple classes by injecting one class into another. Accessing data across various microservices is only one possible use of this trick of sequential chaining.

Drawbacks

Even though the solution presented in this article can solve the problem of aggregating data from multiple sources, it has some drawbacks:

Each class in the DAO stack requires one extra dependency to reference the next DAO in line.
Each method in the DAO requires one extra argument to pass the DTO from DAO to DAO. This might break an existing interfaces!
Any changes to the interface require edits in not just one implementation, but in multiple implementations in multiple classes. When the open/closed rule is violated here, it really blows up!
There’s a weird extra “caboose” class at the end of each stack of DAO classes.

In a future article, I hope to compare this twist on the DAO pattern with an alternative that can assist in implementing the API Gateway pattern. Stay tuned…

Reference:

Using the Data Access Object Pattern in a Microservice Architecture from our WCG partner fireproofsocks at the Codeship Blog blog.