Introduction
Digifloat and [Bank] worked together to design and implement a platform to help build a digital bank from scratch. This is a use case for building a small and medium-sized enterprise (SME) bank. Building a bank from scratch is no small feat. Your team first needs to learn how a bank works, then implement the logic behind it all in the cloud in a scalable way to ensure the bank meets future demand.
The bank’s team helped us understand the banking domain through training sessions and we helped them build it on the cloud.
Bank provided services
Services provided by the bank side was set of distinct APIs, when used together in a set fashion can be used to build a bank.
The details of these APIs are as follows:
Core API
Core is an expansive financial services model that encompasses thousands of banking functions exposed as API endpoints.
- All banking functions and core transaction data available via RESTful APIs
- 100% wide and deep APIs for faster and less costly integrations and customization
- RESTful JSON web services provide access to Core APIs for discrete business functions and Model APIs for CRUDL-level access to data objects
Fraud API
Fraud is a set of API endpoints which can be used to identify risks in a banking transaction. Such as alerts in a wire transfer or an incoming ACH.
FX API
FX is a set of API endpoints which can be used to get the current state of foreign exchange market, used in currency conversion in banking transactions.
Key technologies used
- Visual Studio Team Services—Share code, track work, and ship software for any language, all in a single package.
- Azure Container App—Cloud containerized apps for jobs requiring high availability.
- Azure Container Registry—Repositories for container images.
- Azure Functions—Connectors and web hooks for connecting with external services.
- Azure Application Insights—Insights through application performance management and instant analytics.
- Azure Bicep—Cloud resource management, Infrastructure as Code (IaC).
- Azure API management service—Design, develop and manage our banking APIs.
- Azure DevOps—Cloud Continuous Integration and Continuous Deployment (CICD), Pipelines, automation, and code repositories.
- Azure Notification Hubs—Sending text messages (SMS), emails, and push notifications.
- Azure Service Bus—Central point of system that communicates all the components in a manageable and safe way for exchanging messages.
- Azure key Vault—Store secrets in a safe place.
Customer Profile
Headquartered in Newyork, [bank] is a digital-first financial technology (FinTech) bank focused on small and medium-sized enterprises (SMEs). It provides a one-stop-shop combination of core banking products (business accounts, MasterCard) and integrated FinTech services. It has an international team composed of seasoned consultants and entrepreneurs with experience in digital banking, FinTech, and IT (Information Technology) architecture. Its mission is to reshape the SME banking landscape and financial services across America.Solution, steps, and delivery
The steps we took to deliver the solution directly correspond to the issues described above. They are as follows:
- Design the solution architecture
- Api mapping
- Build and release pipeline for Azure Functions and set up CICD using Azure DevOps
- Develop the IaC to manage all the infrastructure using Azure Bicep
- Use of Azure serverless technology to meet the demand and scale respectively
- End to End order tracking with Application Insights
Architecture
The above architecture was implemented to solve client’s issues. Azure Service Bus was the central point of our platform that communicates all the components in a manageable and safe way for exchanging messages between the processes. Function apps were used to implement trigger-based processes and container apps were used to implement long-running processes such as listening to the core API’s transaction logs. The Cloud Platform we built can also be referred to as a middleware, which integrates or glues all the banking APIs together to complete all the banking operations. Azure DevOps was used to implement CICD for the infrastructure and the code running on our platform.
API mapping
The banking API had several different services exposed through distinct set of APIs such as providing the banking core, analyzing transactions to know if any transaction is fraudulent or not, and case creation in case the transaction is on hold due to any reason, foreign exchange rates etc.
Each service in the banking API used different naming conventions and values required by the requests to its APIs, for them to work together on an end-to-end transaction, we had to do a mapping activity, in which we identified the requirements of each API set and how they link with other services. We had to create several sheets containing the mappings of these APIs. We referred to these sheets when we were implementing the logic in our container and function apps.
For example, the fraud API needs information about the customer, but the transaction only contained the customer id. To get the customer information we had to invoke the core banking API, which in response sends us the customer information. But the customer information the core API sends does not map 1-1 to what we must send to the fraud API, so we refer to our mapping sheet to see which part of the response needs to go where in the fraud APIs request body.
CICD – build and release pipelines
As developers, you write code and check it into the code repository, but the commits checked into the repo might not always be consistent. When multiple developers work on the same project, issues can come up with integration. Teams might run into situations where things do not work, bugs pile up, and project development gets delayed. Developers must wait until the whole software code is built and tested to check for errors, which makes the process slow and less iterative.
With continuous build and integration, developers can simplify builds and test their code by committing their changes to the source code repository and putting tests and verifications into the build environment. In this way, they are always running tests against their code. All the changes made to the source code are built continuously whenever there is a commit made to the repository. With every check-in, the continuous integration (CI) server validates and executes any test that the developer created. If the tests do not pass, the code is sent back for further changes. In this way, the developers do not break the builds that are created. They also do not have to run all the tests locally on their computers, which increases developer productivity.
Azure Pipelines
Azure Pipelines, a service in Azure DevOps, is a fully featured continuous integration and continuous delivery (CD) service that works with your preferred Git provider. It can deploy to most major cloud services, which includes Azure. You can start with your code on GitHub, GitHub Enterprise Server, GitLab, Bitbucket Cloud, or Azure Repos. Then you can automate the build, testing, and deployment of your code to Microsoft Azure, Google Cloud Platform, or Amazon Web Services (AWS).
We implemented several build and release pipelines to implement CICD using Azure Pipelines and Azure DevOps. There were two categories of code, one for the apps and the other for the infrastructure. We used Azure DevOps Repositories to manage these codes using the git version control. Similarly, there were pipelines for infrastructure CICD and the apps logic CICD.
For both, we set up the triggers in our pipelines such that whenever a code merges to staging, our staging environment in the cloud runs the newest version of the code by deploying it on merge. Similarly, when the code passes testing in staging, we merged it in the master which automatically deployed the latest functions and containers to our cloud platform.
IaC – Infrastructure as Code
Infrastructure as code (IaC) uses DevOps methodology and versioning with a descriptive model to define and deploy infrastructure, such as networks, virtual machines, load balancers, and connection topologies. Just as the same source code always generates the same binary, an IaC model generates the same environment every time it deploys.
IaC is a key DevOps practice and a component of continuous delivery. With IaC, DevOps teams can work together with a unified set of practices and tools to deliver applications and their supporting infrastructure rapidly and reliably at scale.
Azure Bicep is a domain-specific language (DSL) that uses declarative syntax to deploy Azure resources. In a Bicep file, you define the infrastructure you want to deploy to Azure, and then use that file throughout the development lifecycle to repeatedly deploy your infrastructure. Your resources are deployed in a consistent manner.
Bicep provides concise syntax, reliable type safety, and support for code reuse. Bicep offers a first-class authoring experience for your infrastructure-as-code solutions in Azure.
We developed the infrastructure code using Azure Bicep and integrated it with our CICD setup to ensure continuous integration and deployment of our infrastructure on the cloud.
Serverless Computing on the Cloud
Every application needs a back end that is responsible for data storage, business logic, and security. Managing the infrastructure to host and execute back-end code requires you to size, provision, and scale multiple servers. You must also manage OS updates and the hardware involved and apply security patches. Then you must monitor all these infrastructure components for performance, availability, and fault tolerance.
Serverless architecture comes in handy for this type of scenario because you have no servers to manage and no OS or related software or hardware updates to manage. Serverless architecture saves developer time and cost, which means faster time to market and focused energy on building applications.
Benefits of compute:
- Abstraction of servers means there is no need to worry about hosting, patching, and security, which allows you to focus solely on the code.
- Instant and efficient scaling ensures that resources are provisioned automatically or on demand at whatever scale you need.
- High availability and fault tolerance.
- Micro-billing ensures you are only billed for when your code is running.
- Code runs in the cloud written in the language of your choice.
Azure Functions
Azure Functions is an event-driven compute experience that you can use to execute your code, written in the programming language of your choice, without worrying about servers. You do not have to manage the application or the infrastructure to run it on. Functions scale on demand, and you pay only for the time your code runs. Azure functions are a great way to implement an API for a mobile application. They are easy to implement and maintain and are accessible through HTTP.
Azure Container Instances and Registry
Azure Container Instances is a great solution for any scenario that can operate in isolated containers, such as simple applications, task automation, and build jobs. Develop apps fast without managing VMs (virtual machines).
Azure Container Registry is a managed registry service based on the open-source Docker Registry 2.0. Create and maintain Azure container registries to store and manage your container images and related artifacts.
We used Azure Container instances to run long running workloads and used function apps where a certain trigger-based operation was needed. We integrated them with key vault for securing the secrets needed during the operations. We integrated them with application insights to store our operational logs and end-to-end order tracking.
End-to-end Order Tracking
Application Insights is an extension of Azure Monitor and provides application performance monitoring (APM) features, along with collecting metrics and application telemetry data, which describe application activities and health, you can use Application Insights to collect and store application trace logging data.
The log trace is associated with other telemetry to give a detailed view of the activity. Adding trace logging to existing apps only requires providing a destination for the logs. You rarely need to change the logging framework.
Application Insights supports distributed tracing, which is also known as distributed component correlation. This feature allows searching for and visualizing an end-to-end flow of a specific execution or transaction. The ability to trace activity from end to end is important for applications built as distributed components or microservices.
The Application Map allows a high-level, top-down view of the application architecture and at-a-glance visual references to component health and responsiveness.
We integrated our services with Azure Application insights for end-to-end tracking of our order using the correlation ID search-for feature of application insights. We assigned a correlation ID when a transaction began in our system, and as it passed through our system, we logged the information to application insights with the correlation ID so we can use the search feature to identify the transaction path.
A note on security
Azure Key Vault is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys. Key Vault service supports two types of containers: vaults and managed hardware security module (HSM) pools. Vaults support storing software and HSM-backed keys, secrets, and certificates. Managed HSM pools only support HSM-backed keys.
We used Azure Key Vault to safely secure our API keys and other relevant secrets.
Conclusion
We managed to build version 1.0 of the bank in the cloud, which incorporates DevOps (CICD) and a serverless, scalable working solution to the client’s issues that were identified in the beginning.
Currently, we are working on version 2.0 which will integrate some monitoring and notification services into our system.
Fawad Arshad
Sr Data Engineer