Cloud Adoption Framework – Data Migration Overview

As discussed in the Azure cloud adoption framework (CAF) overview blog, the structure of CAF is not a linear journey and is a cycle that repeats itself as cloud adoption evolves. This blog post will provide an overview of data migration to Azure.

The below  decision tree helps identify the appropriate data store(s) to use.

An Azure database services decision tree

Common Database Scenarios

ScenarioData service
I need a globally distributed, multi-model database with support for NoSQL choices.Azure Cosmos DB
I need a fully managed relational database that provisions quickly, scales on the fly, and includes built-in intelligence and security.Azure SQL Database
I need a fully managed, scalable MySQL relational database that has high availability and security built in at no extra cost.Azure Database for MySQL
I need a fully managed, scalable PostgreSQL relational database that has high availability and security built in at no extra cost.Azure Database for PostgreSQL
I plan to host enterprise SQL Server apps in the cloud and have full control over the server OS.SQL Server on Virtual Machines
I need a fully managed elastic data warehouse that has security at every level of scale at no extra cost.Azure Synapse Analytics
I need data lake storage resources that are capable of supporting Hadoop clusters or HDFS data.Azure Data Lake
I need high throughput and consistent, low-latency access for my data to support fast, scalable applications.Azure Cache for Redis
I need a fully managed, scalable MariaDB relational database that has high availability and security built in at no extra cost.Azure Database for MariaDB

Azure SQL

Azure SQL is a modern platform powered by the SQL Server engine.

Azure SQL Virtual Machine
Azure SQL virtual machine, is an infrastructure as a service (IaaS)model and is best suited for lift and shift scenarios where workloads requires an operating system access or if the database is customized.

Azure SQL Managed Instance
Azure SQL Managed Instance, is a fully managed service and platform as a service (PaaS) model, and offers the latest SQL Server (Enterprise Edition) database engine.

Azure SQL Database
Azure SQL Database is a relational database as a service (DBaaS) and falls under the platform as a service (PaaS) category, and provides a single database option and a elastic pool option.

Azure SQL Comparison Table

Azure SQL DatabaseAzure SQL Managed InstanceSQL Server on Azure VM
Supports most on-premises database-level capabilities. The most commonly used SQL Server features are available.
99.995% availability guaranteed.
Built-in backups, patching, recovery.
Latest stable Database Engine version.
Ability to assign necessary resources (CPU/storage) to individual databases.
Built-in advanced intelligence and security.
Online change of resources (CPU/storage).
Supports almost all on-premises instance-level and database-level capabilities. High compatibility with SQL Server.
99.99% availability guaranteed.
Built-in backups, patching, recovery.
Latest stable Database Engine version.
Easy migration from SQL Server.
Private IP address within Azure Virtual Network.
Built-in advanced intelligence and security.
Online change of resources (CPU/storage).
You have full control over the SQL Server engine. Supports all on-premises capabilities.
Up to 99.99% availability.
Full parity with the matching version of on-premises SQL Server.
Fixed, well-known Database Engine version.
Easy migration from SQL Server.
Private IP address within Azure Virtual Network.
You have the ability to deploy application or services on the host where SQL Server is placed.
Migration from SQL Server might be challenging.
Some SQL Server features are not available.
No guaranteed exact maintenance time (but nearly transparent).
Compatibility with the SQL Server version can be achieved only using database compatibility levels.
Private IP address support with Azure Private Link.
There is still some minimal number of SQL Server features that are not available.
No guaranteed exact maintenance time (but nearly transparent).
Compatibility with the SQL Server version can be achieved only using database compatibility levels.
You need to manage your backups and patches.
You need to implement your own High-Availability solution.
There is a downtime while changing the resources(CPU/storage)
Databases of up to 100 TB.Up to 8 TB.SQL Server instances with up to 256 TB of storage. The instance can support as many databases as needed.
On-premises application can access data in Azure SQL Database.Native virtual network implementation and connectivity to your on-premises environment using Azure Express Route or VPN Gateway.With SQL virtual machines, you can have applications that run partly in the cloud and partly on-premises. For example, you can extend your on-premises network and Active Directory Domain to the cloud via Azure Virtual Network. For more information on hybrid cloud solutions, see Extending on-premises data solutions to the cloud.

Choosing a deployment option

Courtesy of Microsoft

Migration tooling options

Data security & protection

Develop clear, simple, and well-communicated guidelines to identify, protect, and monitor the most important data assets anywhere they reside.

Identify and classify sensitive assets, and define the technologies and processes to automatically apply security controls.

Once the data you need to protect has been identified, consider how you will protect the data at rest and data in transit.

Data at rest: Data that exists statically on physical media, whether magnetic or optical disk, on premises or in the cloud.

Data in transit: Data while it is being transferred between components, locations or programs, such as over the network, across a service bus (from on-premises to cloud and vice-versa), or during an input/output process.

Data Classification

Data classification process categorizes data by sensitivity and business impact in order to identify risks. When data is classified, you can manage it in ways that protect sensitive or important data from theft or loss.

The following is a list of classifications Microsoft uses. Depending on your industry or existing security requirements, data classification standards might already exist within your organization.
You can also create custom tags with the SQL classification tools in SSMS.

  • Non-business: Data from your personal life that doesn’t belong to Microsoft.
  • Public: Business data that is freely available and approved for public consumption.
  • General: Business data that isn’t meant for a public audience.
  • Confidential: Business data that can cause harm to Microsoft if overshared.
  • Highly confidential: Business data that would cause extensive harm to Microsoft if overshared.

Data management

Azure Data Catalog
Azure Data Catalog is a fully managed cloud service that serves as a system of registration and system of discovery for enterprise data sources. Data Catalog allows users to provide their own descriptive metadata – such as descriptions and tags – to supplement the metadata extracted from the data source, and to make the data source more understandable to more people.

Azure Data Catalog also allows users to provide their own complete documentation that can describe the usage and common scenarios for the data source.

This wraps up the Cloud Adoption Framework – Data Migration Overview.

Azure Landing Zone – Networking Overview

In this blog post, we will be going through the networking overview of the Azure Landing Zone.

Networking Decision Guide

The below network decision tree should be used as a starting point to determine the network services that should be used.

 

 

 

Consider the following design elements:

Below are the design consideration for Azure networking and connectivity.

  • Planning for IP Addressing
  • Configure DNS
  • Define an Azure Networking Topology
  • Connectivity to Azure
  • Connectivity to other cloud providers

Planning for IP Addressing

IP address planning is a vital first step when designing a network in Azure especially if you have a hybrid environment to avoid overlapping IP address space across on-premises environment and Azure.

Design Considerations:

  • Azure reserves 5 IP address with each subnet so factor in the address space when sizing the virtual networks.
  • Some Azure service such as Application Gateway – WAF, Azure Firewall, Azure Bastion and VPN Gateway require dedicated subnets.
  • You can delegate subnets to some Azure service that can be injected into the virtual network.

Design Recommendation:

  • IP address space should not overlap the on-premises environment.
  • Use the non-routable, private address spaces.
    • 10.0.0.0 – 10.255.255.255 (10/8 prefix)
    • 172.16.0.0 – 172.31.255.255 (172.16/12 prefix)
    • 192.168.0.0 – 192.168.255.255 (192.168/16 prefix)
  • You cannot add the following address ranges:
    • 224.0.0.0/4 (Multicast)
    • 255.255.255.255/32 (Broadcast)
    • 127.0.0.0/8 (Loopback)
    • 169.254.0.0/16 (Link-local)
    • 168.63.129.16/32 (Internal DNS)
  • Plan for future growth since adding address space can cause an outage.
  • Public IP addresses should not be used for virtual networks.

Domain Name System (DNS)

Since DNS is a critical part of networking, some companies may use their exisiting DNS solution and other may adopt native Azure capabilities.

Design Considerations:

  • The maximum number of private DNS zone, which can be linked to a virtual network with auto-registration is one.
  • Be aware of the Azure Private DNS zone limits.

Design Recommendation:

  • Use Azure DNS zones for Azure related name space resolutions.
  • In a mix environment (Azure + on-premises environment), use exiting DNS services such as Active Directory integrated DNS.
  • If the Azure environment is running Azure Firewall, then DNS Proxy should be evaluated.

Azure Networking Topology

Azure Virtual WAN is a Microsoft managed solution that provided end to end global and dynamic transit connectivity.

Virtual WAN simplifies end to end network connectivity from on-premises to Azure and within Azure by creating a hub and spoke network architecture.

Virtual WAN network topology.

Diagram that illustrates a Virtual WAN network topology.

A traditional Azure network topology.

Diagram that illustrates a traditional Azure network topology.

Connectivity to Azure

Design Considerations:

  • Azure ExpressRoute private connectivity to Azure infrastructure since the traffic is not going through the internet.
  • Private Link can be established for connectivity to Azure platform as a service (PaaS) over ExpressRoute with private peering.

Design Recommendation:

  • ExpressRoute should be used as the primary connection for connecting an on-premises environment to Azure. Furthermore, site to site VPN can be used as a backup connectivity.
  • Using a single ExpressRoute connection is the singe point of failure so use dual ExpressRoute circuits.
  • ExpressRoute/VPN Gateway come in various SKUs so choose the right SKU based on the requirements.

Connectivity to other cloud providers

Follow the below cross-cloud connectivity flow chart for choosing an option.

Option 1 – Customer manages routing.
Option 2 – A cloud exchange provider manages routing.
Option 3 – Use site to site VPN.

Design Considerations:

  • Azure virtual network can only be connected to another cloud provider’s virtual private cloud (VPC) if the private IP addresses do not overlap.
  • Site to site VPN have lower throughput and higher latency than ExpressRoute.
  • Design Recommendation:
  • If you do not want to use public internet, then choose option 1 and 2.
  • If ExpressRoute is not available, you can use site to site VPN with traffic going through the internet for the connection between Azure and cloud provider.

This wraps up the Azure Landing Zone – Networking Overview.

Azure Landing Zone Overview

Azure landing zone, which is a subset of Microsoft Cloud Adoption Framework for Azure, help customers set up their Azure environment for scale, security, governance, networking, and identity.

Azure Subscription

Once you create an Azure tenant, the next step is to create a subscription(s). You might be thinking, ‘how many subscription(s) do I need’? This really depends on many factors so lets take a look at it.

Azure subscriptions have different limits for different resource types and this link does a great job of explaining the different service limits, quotas, and constraints.

Items to look at when designing the subscription model:
Technical Requirements
•Network Connectivity (shared or dedicated)
•Active directory requirements, clustering, identity, management tools

Security Requirements
•Who are the subscription administrators
•Least privilege model

Scalability Requirements
•Growth plans
•Allocation of limited resources
•Evolution over time (users, shared access, resource limits)

Here are some questions that should be asked before creating a subscription:
•Who will be responsible for creating subscriptions?
•What resources will be in a subscription by default?
•Are there any capacity / technical limitations?
•Do we want to ensure Separation of duties?
•Dev/Test Vs. Production?
•Different end customers?
•Different departments or business units?
•Different projects?
•What is the right naming convention to be used? Here is a great article on naming convention.

Azure Resources

Once the Azure subscription(s) are setup, the next step is to think about how to organize the Azure resources.

Azure Management group

Define the management group hierarchy on organization and environment type such as production, dev/test etc.

The root management group is for global configuration and be careful with the management level assessments as they will cascade through the hierarchy. Assign common polices and RBAC on the management group level.

The built-in RBAC roles for management group are MG contributor and MG reader.

Azure Role-Based Access Control (RBAC)

Using RBAC, you can segregate duties within your team and grant only the amount of access to users that they need to perform their jobs. Instead of giving everybody unrestricted permissions in your Azure subscription or resources, you can allow only certain actions at a particular scope.

  1. Security principal

A security principal is an object that represents a user, group, service principal, or managed identity that is requesting access to Azure resources.

– Service principal – A security identity used by applications or services to access specific Azure resources. You can think of it as a user identity (username and password or certificate) for an application. -Managed identity – An identity in Azure Active Directory that is automatically managed by Azure. You typically use managed identities when developing cloud applications to manage the credentials for authenticating to Azure services.

2. Role definition

A role definition is a collection of permissions. It’s sometimes just called a role. A role definition lists the operations that can be performed, such as read, write, and delete. Roles can be high-level, like owner, or specific, like virtual machine reader. You can create custom roles if none of the existing built-in roles don’t meet the specific needs of your organization.

3. Scope

Scope is the boundary that the access applies to. When you assign a role, you can further limit the actions allowed by defining a scope. This is helpful if you want to make someone a Website Contributor, but only for one resource group.

Azure Tags

Here is the Azure tagging decision guide. Here is the link for best practices on using Resource Tags.

Azure Policy

Azure policy can enforce real time policy and at-scale compliance assessment. Also, the policy evaluates all Azure resource and can generate events that can be used for alerting. Furthermore, Azure policy can be used to automatically remediate problem in the environment.

My recommendation is to start with audit policies, which is a safe way of understanding what a policy will do without affecting the user activity. Furthermore, deny policies should be rolled out in stages to understand the impact.

This wraps up the Azure Landing Zone Overview.