Part 1. I think the best solution for most organizations thinking on distributing their systems, is a pattern that i define as Mini Services Architecture. This pattern may or may not use Docker containers and may or may not be orchestrated by Kubernetes. These are the key aspects of this pattern:
- Turn into a Mini Service each part of the monolith that is not going to be part of a Transaction.
- Use the same PHP Framework as template for each one of these.
- Communication between each Mini Service is through REST Api calls.
- Use DTOs (Using DTOs in PHP Symfony) as contract between each part involved to ensure data consistency and integrity.
I. As I explained in the past Article: Microservices Architecture Transaction processing is key for having a robust, highly trusted, stable system.
If we consider the example in this past article, this transaction for the purchase order:
function createPurchaseOrderForClient($params)
{
try {
$this->startTransaction();
$this->processForOrders($params);
$this->processForProducts($params);
$this->processForInventory($params);
$this->processForClientAccount($params);
$this->commitTransaction();//At this moment all executed queries are persisted
return 'Trx processed ok';
} catch (Throwable $e) {
//Something went wrong and need to roll back
$this->rollBack();//DB handler rolls back all executed queries and nothing is persisted
//...
throw new Exception($e->getMessage());
}
}We clearly see that all these Entities need to be in the same Mini Service. What I do, is not breaking apart this, but instead making modules for each:
And in each Module we organize the code in this way if in Rest Api Mode:
And in MVC Mode:
From the past transaction point of view, we see that for example:
- Users
- PDF Invoices
- Document backup storage
- Electronic Contract Agreement Signature
None of these are required for commiting this transaction. If this is true for all other types of transactions in this Mini Service, we can safely put each one of the above in its own Mini Service:
This is in my opinion the best way to have the best from both worlds:
- An application that ensures Data Integrity and Data Consistency.
- A more manageable code bases that can be deployed independently.
In terms of Persistence layer, you choose between: 1. Having a single Database Server for managing each one of the Schemas of each Mini Service, or 2. Having one Database Server for each one of the Schemas.
For option 1. we need:
- Nginx Container
- PHP Container
- SQL Proxy Container or direct DB url connection
For option 2. we need:
- Nginx Container
- PHP Container
- MySql / MariaDB Container or any other preferred DB vendor
Nginx is my preferred Web Server as if tuned correctly and paired with PHP-FPM, it can deliver very high performance even in high workloads. (more on this in this post: High performance configuration for Nginx Web Server, PHP Tuning for High Performance LEMP and some other ones).
For option 1. unless the Organization needs to deal with a very large Database and / or executing part of the processes in the DB side as stored procedures, etc.. I'd say community versions of MySql or MariaDB are more than enough. PostreSql is a very good option as well. These community versions allows you to implement DB replication as Master - Slave and even community MariaDB can be implemented as a Gallera Cluster with Master to Master replication and Sql Proxy as load balancer.
For option 2. you can easily set cron jobs for backing up the Data from each one of the DBs to a predefined Storage Bucket or implement replication for each one. Keep in mind that this option could increase complexity and it might be a better idea to implement it in a Kubernetes orchestrated environment.
II. Use the same PHP Framework as template for each one of the Mini Services you are implementing.
I strongly recommend to implement them with Symfony Framework that comes along with a vast repository of components and vendor packages to speed up development without re inventing the wheel. This way, it will be very easy and cost effective to make updates to the code bases, ensure consistency and have almost the same set of third party packages. Also, you need to write and maintain only one set of the in house modules that are common to all mini services, like the module to handle communication, data decoding and validation, security authorization and authentication, and many others. This is key to facilitate implementation, reduce development times and ensure better automation of the whole CI/CD as all code bases share the same Framework Template.
III. Communication between each Mini Service is through REST Api calls.
If you plan to have these Mini services running in docker based containers orchestrated by Kubernetes, then most likely you will have a local environment for developers to run these apps in their local machines. In local the most common orchestrator is docker-compose. In the cloud, Kubernetes allows you to have environments called namespaces: you most likely will have Dev, QA, Stage an Prod. To ensure consistency, you should have your apps to communicate to each other through REST Api calls only for that particular environment, in other words, you local services, can make internal calls to only other local services, QA services can make internal calls to other apps only living in the QA environment. This way, if developers need to implement a feature that involves 2 or more services, you can deploy them to each one of the environments, without affecting the behavior of other environments. This way, other teams can keep working uninterrupted, and of course, Prod environment must be kept isolated from any interaction from other environments.
To handle internal communications, and as I mentioned before, using the same framework allows you to have the same template class with the logic to handle REST Api calls. In my case i use guzzlehttp/guzzle package implementing something like this:
<?php
namespace App\ApiBundle\Services;
use App\ApiBundle\Constants\AppConstants;
use GuzzleHttp\Client;
use GuzzleHttp\Exception\GuzzleException;
use Symfony\Component\DependencyInjection\ParameterBag\ContainerBagInterface;
/**
* Class ApiGuzzleClient
*/
class ApiGuzzleClient
{
private const INTERNAL_PREFIX = '/internal';
/**
* @var Client
*/
private Client $client;
/**
* @var ContainerBagInterface
*/
private ContainerBagInterface $params;
/**
* @var string
*/
private string $protocol;
/**
* @var string
*/
private string $port;
/**
* ApiGuzzleClient constructor.
* @param ContainerBagInterface $params
*/
public function __construct(ContainerBagInterface $params)
{
$this->client = new Client();
$this->params = $params;//In case you need to define params
}
/**
* @return Client
*/
public function getClient(): Client
{
return $this->client;
}
/**
* @param $servicePort
*/
public function setProtocolAndPort($servicePort)
{
//In case you need to handle differnt protocols or ports for each environment
$this->protocol = 'http://';
$this->port = 80;
}
/**
* @param string $serviceName
* @return string
*/
public function getServiceDomain(string $serviceName): string
{
$this->setProtocolAndPort($serviceName.'_port');
return $this->protocol.$serviceName.'-service:'.$this->port;
}
/**
* @param string $serviceName
* @param string $uri
* @param string $method
* @param string $body
* @return string
* @throws GuzzleException
*/
public function makeInternalCall(string $serviceName, string $uri, string $method = 'GET', string $body = ''): string
{
/*
$serviceName = 'customers';
$servicePort = 'customers_port';
$uri = '/v1/some/path/as/endpoint';
$method = 'POST';
$body = 'json.encoded.string';
*/
$this->setProtocolAndPort($serviceName.'_port');
$headers = $this->setHeadersForInternalRequest();
$serviceName = $this->protocol.$serviceName.'-service:'.$this->port;
$response = $this->getClient()->request(
$method,
$serviceName.self::INTERNAL_PREFIX.$uri,
[
'headers' => $headers,
'verify' => false,
'body' => $body
]
);
return $response->getBody()->getContents();
}
/**
* @param string $serviceName
* @param string $uri
* @param array $content
* @return string
* @throws GuzzleException
*/
public function makeInternalMultiPartPost(string $serviceName, string $uri, array $content): string
{
//$content = ['headers' => $headers, 'multipart' => $multipart, 'verify' => false]
$this->setProtocolAndPort($serviceName.'_port');
$serviceName = $this->protocol.$serviceName.'-service:'.$this->port;
$request = $this->getClient()->request('POST', $serviceName.self::INTERNAL_PREFIX.$uri, $content);
return $request->getBody()->getContents();
}
/**
* @param string $accept
* @param string $contentType
* @return array
*/
private function setHeadersForInternalRequest(string $accept = 'application/json', string $contentType = 'application/json'): array
{
return [
'Accept' => $accept,
'Content-Type' => $contentType,
'Authorization' => 'Bearer ' . md5_file($this->params->get('INTERNAL_TOKEN_KEY')),
'X-Request' => 'Internal'
];
}
public function getHeadersForServiceDomain(string $accept, string $contentType): array
{
return $this->setHeadersForInternalRequest($accept, $contentType);
}
}This logic is paired with a event listener present in all services in the cluster, that handles security authentication and authorization, so to prevent possible security breaches. As many of the endpoints in these services could be called from different channels, like the Browser UI, an internal call, or maybe an external third party application, I have implemented different security firewalls to handle each one of these calls. In the case of an internal service call, the event listener checks for the presence of the 'X-Request' => 'Internal' in the headers of the request and it checks that the md5 internal token key matches with the one declared in the config settings or secrets vault.
IV. Use DTOs as contract between each part involved to ensure data consistency and integrity.
It is very important to keep data consistency and integrity, but also we need to have a blue print that allow each dev team to easy have the the data structure the called service is expecting or the response coming back. Having these DTOs set as a contract between parties, each dev team can focus on implementing its own logic without worrying on if the request or if the response will be compatible. For doing that, I have implemented along with all DTO classes, all required PHP Doc Blocks to stick to the Open Api Specification using nelmio/api-doc-bundle and zircote/swagger-php :
<?php
namespace App\CustomersModule\Controller;
use App\ApiModule\Constants\ErrorCodes;
use App\CustomersModule\Controller\CustomersDashboardFilter;
use App\CustomersModule\Api\Dtos as DTO;
use App\CustomersModule\Services\CustomersService;
use Nelmio\ApiDocBundle\Annotation\Model;
use Nelmio\ApiDocBundle\Annotation\Security;
use OpenApi\Annotations as OA;
use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\Routing\Annotation\Route;
/**
* Class CustomersController
* @OA\Tag(name="Customers")
*/
class CustomersController extends AbstractController
{
/**
* @Route("/api/v1/customer", name="createCustomerEndpoint", methods={"POST"}, defaults={"_format": "json"})
* @OA\Post(
* summary="Create a New Customer",
* description="Create a new Customer providing all neccesary fields in json format",
* operationId="createCustomerEndpoint",
* @OA\Parameter(
* name="body",
* in="query",
* description="Customer Dto object",
* required=true,
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=200,
* description="successful operation",
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=404,
* description="Not a Valid Customer"
* ),
* @OA\Response(
* response="500",
* description="There was an Internal Server Error."
* ),
* @Security(name="api_key")
* )
* @param \App\CustomersModule\Api\Dtos\Customer $request
* @param CustomersService $customersService
* @return JsonResponse
*/
public function createCustomerAction(\App\CustomersModule\Api\Dtos\Customer $request, CustomersService $customersService)
{
$customer = $customersService->upsertCustomer($request);
$result = $customersService->getCustomerById($customer->getId());
return new JsonResponse($result, ErrorCodes::HTTP_200);
}
/**
* @Route("/api/v1/customer", name="updateCustomerEndpoint", methods={"PUT"}, defaults={"_format": "json"})
* @OA\Put(
* summary="Update Customer",
* description="Update Customer providing all neccesary fields in json format",
* operationId="updateCustomerEndpoint",
* @OA\Parameter(
* name="body",
* in="query",
* description="Customer Dto object",
* required=true,
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=200,
* description="successful operation",
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=404,
* description="Not a Valid Customer"
* ),
* @OA\Response(
* response="500",
* description="There was an Internal Server Error."
* ),
* @Security(name="api_key")
* )
* @param \App\CustomersModule\Api\Dtos\Customer $request
* @param CustomersService $customersService
* @return JsonResponse
*/
public function updateCustomerAction(\App\CustomersModule\Api\Dtos\Customer $request, CustomersService $customersService)
{
$customer = $customersService->upsertCustomer($request);
$result = $customersService->getCustomerById($customer->getId());
return new JsonResponse($result, ErrorCodes::HTTP_200);
}
/**
* @Route("/api/v1/customer/{id}", name="getCustomerByIdEndpoint", methods={"GET"}, defaults={"_format": "json"})
* @OA\Get(
* summary="Get Customer by Id",
* description="This endpoint returns the full Customer Entity ",
* operationId="getCustomerByIdEndpoint",
* @OA\Response(
* response=200,
* description="successful operation",
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=404,
* description="Not Route found"
* ),
* @OA\Response(
* response="500",
* description="There was an Internal Server Error."
* ),
* ),
* @Security(name="api_key")
* )
* @param $id
* @param CustomersService $customersService
* @return JsonResponse
*/
public function customerByIdAction($id, CustomersService $customersService)
{
$result = $customersService->getCustomerById($id);
return new JsonResponse($result, ErrorCodes::HTTP_200);
}
/**
* @Route("/api/v1/customer/{id}", name="deleteCustomerByIdEndpoint", methods={"DELETE"}, defaults={"_format": "json"})
* @OA\Delete(
* summary="Delete Customer by Id",
* description="This endpoint removes a Customer Entity from DB",
* operationId="deleteCustomerByIdEndpoint",
* @OA\Response(
* response=200,
* description="successful operation",
* @Model(type=App\ApiModule\Dtos\ApiResponse::class)
* ),
* @OA\Response(
* response=404,
* description="Not Route found"
* ),
* @OA\Response(
* response="500",
* description="There was an Internal Server Error."
* ),
* ),
* @Security(name="api_key")
* )
* @param $id
* @param CustomersService $customersService
* @return JsonResponse
*/
public function deleteCustomerByIdAction($id, CustomersService $customersService)
{
$result = $customersService->deleteCustomerById($id);
return new JsonResponse($result, ErrorCodes::HTTP_200);
}
/**
* @Route("/api/v1/customer-dashboard", name="customerDashboardEndpoint", methods={"PUT"}, defaults={"_format": "json"})
* @OA\Put(
* summary="Customer Dashboard",
* description="List of Customers",
* operationId="customerDashboardEndpoint",
* @OA\Parameter(
* name="body",
* in="query",
* description="Customer Dto object",
* required=true,
* @Model(type=DTO\CustomersDashboardFilter::class)
* ),
* @OA\Response(
* response=200,
* description="successful operation",
* @Model(type=DTO\Customer::class)
* ),
* @OA\Response(
* response=404,
* description="Not a Valid Customer"
* ),
* @OA\Response(
* response="500",
* description="There was an Internal Server Error."
* ),
* @Security(name="api_key")
* )
* @param DTO\CustomersDashboardFilter $request
* @param CustomersService $customersService
* @return JsonResponse
*/
public function customerDashboardAction(DTO\CustomersDashboardFilter $request, CustomersService $customersService)
{
$result = $customersService->customerDashboard($request);
return new JsonResponse($result, ErrorCodes::HTTP_200);
}
}
With all these meta data, any developer can open the a page where all endpoints with their description along with data model for the request and for the response:
This is so far, the best way of implementing a distributed system following the Service Oriented and Micro Service Architecture Patterns.