Intro.
In the past post i ended it up mentioning the key benefits of a particular Distributed System set up gave us at that time. We had a big monolithic ERP System running in a very old computer. It was running pretty well, but it was falling very short for the current needs of the Company . Then I pushed for modernizing our System adopting a new philosophy, in this case, a System made up of smaller Applications, running in different machines.
Since then, most of those enterprise applications are fundamentally the same, only difference is that thanks to the Internet, we can interact with them in many other ways and also, we can add some more functionality.
In current times, what i have seen is that as every body can post online,there are a huge amount of posts referring to Distributed Systems directly, or naming them as "Service Oriented Architecture" or the most sticky one "Microservices".
Every body has its own opinion and point of view on how these implementations should be done, and of course, we all are bombarded by marketing strategies that their ultimate goal is selling you over complicated versions of the same thing, but at the end, much more expensive.
In your Organization, most likely you are already working in a Distributed System Environment as Accounting is running its own Application, HR its own as well or Sales or Marketing, etc. and most likely each one of those Apps are running in different machines, either physical or virtual cloud instances.
Determining the Strategy to follow to achieve Distributed System.
When the Architect has to come up with a plan to distribute the systems in an Organization,he need to take in account several factors including:
- Workload for each App
- Size of the codebase in each App
- Available Budget for this matter
- Expected grow for the next 10 years
- Size of IT department and available resources
It is a common mistake in many organizations to try to adopt technologies, architecture patterns or new tools that are advertised as the "solution for all your IT issues". Common example is Microservices with Docker and Kubernetes.
But turns out that adopting such technologies, may end in a bigger mess, so it is important to be very careful and be very sure on what has to be done.
If any of the applications were built from scratch and not bigger than 2~3 million lines of code, it is much better idea to re-write them, instead of going through the painful process of refactoring.
When you decide to do this?
1.1 If the answer is any of these:
- Codebases are a complete mess and bugs reported all the time.
- More than 10 years without an update.
- Applications response times are getting slower and slower, a typical page loads in more than 8 secs.
- Server crashes and services interrupted very often.
- Data loss or not enough Data Integrity / Consistency.
- Security vulnerabilities and high exposure to hacker attacks / Data breaches.
If one or more of the above statements are true for the Systems running in the Company, it is imperative to take action and plan for a total Systems Revamp as soon as possible, otherwise, it will be 10x more expensive to try to fix this once the Systems fall apart completely, or a Security breach cost millions in law suits.
80% of the time, performance degradation is caused by databases, so the first thing you need to evaluate is the Status of these.
Key aspects to evaluate:
- Is the DB close to the applications? Latency is a big issue and this is the first thing that need to be analyzed.
- Is one DB Server giving service to all applications? then, most likely, DB Server need more CPU Power and RAM and professional tuning, this include. If running in Oracle DB or Microsoft's SQL Server, adding more resources means increasing cost of DB operation. At this moment is important to evaluate if better migrating schemas to a more affordable DB server like MySQL / Percona / MariaDB along with a Distributed System Architecture.
- DB replication, size of the binary log, latency between replicas are important things to review and evaluate.
- Table fragmentation is another cause of performance degradation. There are some good tools available for optimizing tables and reduce fragmentation.
There is no magical recipe or nothing set in stone when talking strategies to modernize and improve performance and functionality of current Systems, but you can start from here:
- Put to work each Application in its own server. Most recommended is to use a virtualized environment either running on premises or in any cloud provider like Oracle Cloud (Try Oracle cloud for free: How to get a Free Cloud Server to run a Web Site). Amount of CPU power and RAM can be determined either by running load tests or by scaling up resources based on workload.
- Make sure your applications are running in the same local network as the DB. Latency need to be as low as possible.
After done with that, you already have a basic distributed system where each one of the applications are running in its own virtual machine.
Now its time to evaluate which applications are candidates for splitting them up into more manageable micro apps.
If you have applications that were provided by third party vendors, like Oracle JD Edwards or SAP, you need to decide if you can continue paying for their use or you start building your own from scratch.
However, these systems are big, and it will take time, most likely 3 to 6 years of paying at least for a team of 1 Architect, 3~5 Senior Developers, 4~6 Jr. to Mid Developers, 2 Business Analyst, 1 Project manager and 2~4 Q&A testers along with current employed accountants, operations, marketing, managers, etc. that would provide the requirements and use cases.
The benefits of writing your own System, is that it will be fully customized to the Organization's requirements and can be easily extended or customized for future needs.
There are many third party systems that allow custom integrations, this is some thing you should look at and decide accordingly.
But if you are in this scenario of having legacy applications that were built for the company and you have answered positive to any in 1.1 above, then you should plan for building a modern Distributed System out of them.
Next Up: Key aspects to determine how to distribute a system