https://www.youtube.com/playlist?list=PLGjZwEtPN7j-Q59JYso3L4_yoCjj2syrM
episode 1
high availability: service up time, e.g. 99%
scalability: ability to scale up/out
elasticity: ability to scale dynamically, e.g. depending on workload
agility: ability to react quickly
fault tolerance: ability to remain up and running while some components/services fail
disaster recovery: ability to recover from disaster, e.g. using region replication
episode 2
economies of scale
as scale goes up, average cost per unit drops
episode 3
capital expenditure (CapEx)
own infrastructure
big investment initially to purchase hardware, but later on hardware cost is low
ongoing maintenance cost
Operational Expenditure (OpEx)
rent infrastructure
pay as you go
minimal maintenance
episode 4:
consumption based pricing model
pay for what you used, compute + storage + network
episode 5:
infrastructure-as-a-service (IaaS)
-networking, hardware, and virtualization
platform-as-a-service
- operating system, middleware/software (dev tools, sql, logic app), runtime (iis docker)
software-as-a-service
-data and apps, out-of-the-box applications, e.g. outlook, skype, onedrive, etc.
episode 6
deployment model
private cloud, you host everything, CapEx, need IT skills
public cloud, microsoft hosts everything, may not meet security/compliance requirements
hybird, a mixed of private and cloud, more expensive, complicated, need IT skills
episode 7
data center, a set of servers, independent power, cooling and networking
region, 1 or more data centers connected with low latency network, e.g. au east, au west
not all services are available in every region
some services (AD, DNS) are global, not specific to any region
US government region, china partnered region
Availability zone
each data center is a zone
can replicate service onto multiple availability zones to have zone redundancy
region pair
replicate to a paired region, e.g. au east-> au west to avoid natural disaster
geography
europe, asia pacific, middle east, etc.
regions are grouped into geographies
ensure dta residensy, sovereighty, compliance and compliance are met
protect from region wide failures
episode 8
resource, resource group and resource manageer
everything is a resource, with a JSON definition of all the properties
resources are grouped into resource group.
A resource group is logical and for free. It has its own region for storing information, and a resource within it can be at another region.
Many ways to group resources, type, lifecycle, billing, region, etc.
e.g. all sql servers in one resource group
all dev resources (db, network, vm, etc.) in one group
all resources from a department in one group
resource manager is a management layer for all resources and resource groups.
We can use azure portal / rest / powershell / CLI /SDKs to create and manage resources.
They all connect to the Azure Resource Manager (ARM) later using a unified language.
episode 9
'Compute' resources in Azure
virtual machine gives you good control of everything, but no autoscale, only one node. You can still add extra cpu memory though.
virtual machine scale set, autoscale by replicating your virtual machine as per load balance, one or multiple nodes
Container, is more lightweight than VM. IaaS
VM needs a layer of virtualization software to simulate hardware for each VM. Each VM has its own operating system.
Container needs a layer of container runtime to simulate operating system of the host.
Azure Container Instances can host Container Group (a VM) where multiple containers can be run. This is platform-as-a-service while VM is IaaS.
Azure Container Instances can host zero up to 20 nodes. No autoscaling. Still give you some control of the operating sys/software.
Kubernetes Services provides autoscaling for containers, 3-100 nodes as per load balance, PaaS
Azure app service
If you dont want to manage containers, Azure app service can be an option
Just build web app into a package, and deploy to app service. App Service will create nodes to run app, autoscale.
Azure function is based on Azure App service.
instead of running a web app, you can package a simple function and deploy it. Azure function creates nodes to run the function as web service. Serverless. designed for micro/nano-service. PaaS
episode 10
networking
Virtual Network
for only one region, so multiple networks if multiple regions
VNET gateway/VNET peering connects on-prem and cloud, cross VNET communication
VNET is divided into subnets
subnets is for address allocation, network security group setting
In virtual network, search for diagram, it will show the network structure
Load Balancer
for non HTTP traffic distribution onto VMs
separate load balancers for web layer, and for data layer for example
the former is called public load balancer, the later is called internal load balancer
App gateway
for HTTP traffic distribution onto VMs / App services
extra http related features, session, url routing, ssl termination, redirection, etc.
Content Delivery Network CDN
deliver web content (e.g. static image, stylesheet, java script static pages) to users
duplicate content onto many POP (points of presence) locations in different regions
users connect to nearest POP
episode 11
Azure storage account
Blob Storage (binary large object) or basically can be any file
container and files
cool, hot or archive tiers. Hot: frequent access, cool: occasional access, archive: rare access
File Storage
same as blob storage except designed for share drive only.
use file sharing protocol so it can be mounted to windows, linux as a drive.
Table Storage
semi structure data
table structure, need to specify partition key and row key for quick access.
but no schema constraint, can put in any column, any type
Designed for NoSQL storage
Queue storage
store small messages in queues
Disk storage
simulate a hard disk for virtual machine, SSD or HDD
can choose unmanaged (i.e. on blob and managed yourself), or managed (microsoft manages it)
episode 12
cosmos db, geographically distributed NoSQL database, designed for low latency
Azure SQL Server, MySQL, PostegreSQL
Some Azure SQL Server features have been cut, e.g. ssis, ssrs, ssas, you may use data factory, power bi, and analysis server instead in azure.
SQL Managed Instance, fully capability version of SQL server, but pricy
SQL Data Warehouse (Synapse), sql server for massively parallel processing, big data warehousing
SQL VM, VM for SQL server, managed by users
episode 13
marketplace
- commercial marketplace, azure, power bi, office 365, for business users
- azure, for developers and IT pros
episode 14
IOT
-IOT Hub, managed service for bi-directional communication with IOT devices, PaaS
once create a device, copy connection string and configure on the device so the device can send data to IOT hub
good for building apps from scratch
-IOT Central
built on top of IOT HUB, provides a lot of templates to avoid building apps from scratch
This is an IOT application platform, SaaS
-Azure Sphere
End to end approach for building IOT solutions
provides hardware, chips - MCUs
sphere OS based on LINUX
Azure Security service for device to cloud communication
Side notes:
Difference between IOT hub, Event Hubs and Event grid
IOT Hub focuses on connecting IoT devices to the cloud.
It connects to physical devices, and can receive and send messages, so two-way communication with devices.
Event hub, is used by IoT hub for telemetry flow path, but IoT Hub can also receive files, notifications,etc.
Event hub focues on big data streaming.
Event Grid, is for distributing events to subscribers.
An use case is using event hub to capture web site telemetry, and event grid to respond to telemetry event like item shipped.
episode 15
synapse
big data analytics platform (PaaS)
- Studio includes components
-- pipelines (data factory)
-- spark
-- synapse SQL
-easy communication with azure datalake
Azure HDInsight
Flexible multi-purpose big data platform (PaaS)
- big data clusters (support Hive, Spark, Kafka, Hbase, Storm, etc.)
Azure databricks
big data collaboration platform (PaaS)
built on top of spark
a unified workspace for notebook , cluster, data, collaboration, etc.
integrate very well with common azure data services, e.g. datalake
episode 16
Azure Machine learning
- Machine Learning Workspace, the top level resource that ties together compute, pipeline, experiment, connection, storage, deployment, everything
- Studio, web portal
- features include
-- notebooks, python/R
-- Auto ML, run multiple algorithms/params to choose the best model
-- designer is a drag and drop type of interface
-- Data and Compute, manage storage and compute resources
-- pipelines, orchestrate model training all the way to deployment, monitoring and retraining
episode 17
serverless
- Azure Funciton, Function-as-a-service, application development platform for nano-services, auto scale. Called through web service.
- Logic Apps, PaaS, no-code development of workflow orchestration /business process. many connectors available for storage, email, events, etc. It can be triggered by different events, e.g. a blob file uploaded, web, office, email, etc.
- Event Grid, event routing services. Incoming events are grouped by topics, subscribers receive events by subscribtion.
Episode 18
DevOps Services
- Azure DevOps, a collection of tools for DevOps practices, including Boards (e.g. Scrum), Pipelines (CICD continous deployment), Repo (e.g. git), Artifects (manage project deliverables), and test plans.
- DevTest labs
provide sandbox environments (VM) with templates, PaaS
quick setup of virtual machines by selecting OS, size, software, tools
can apply lab policies on quotas, sizes, etc.
episode 19
Azure portal - web portal for self-service
Azure PowerShell - run from local computer to interact with Azure, use script and command lines
Azure CLI - similar to powershell but syntax is more linux, also run from local computer
Azure Cloud shell - supports both powershell and CLI command line from azure portal
episode 20
Azure advisor
Provide recommendations and best practices for cost, security, performance, etc.
Can click on the actionable recoomendataions links to go to the pages for implementation.
Episode 21
- Network security group
like firewall setting, apply to a vnet or a subnet to filter traffic
- Application security group
a logical grouping of virtual network resources, e.g. all web servers into an group
so as to apply filtering rules on the logical groups instead of IPs.
Episode 22
Route table
- custom/user defined routes (UDR)
e.g. within a virtual network, there are 3 subsets: A, B and C
normally A can connect to C directly, but now we want to re-route the traffic from A -> C to A->B->C, where B has a VM server (virtual appliance) for traffic inspection
-- create a route table resource, associate the route table with subnet A, so it applies to all traffic from A
-- create a 'route' and select C as the address prefix so all traffic going to C will be rewritten
-- configure the next hop as a Virtual Appliance (a special VM with firewall), and put it the VM's IP as next hop address
Done, now all traffic A->C will be directed to B for inspection first and then further forwarded to C.
Episode 23
Azure firewall
PaaS, firewall as a service
Traffic filtering rules, support FQDN (fully qualified domain name) and inspect body content, etc.
This can replace the Virtual Appliance (special VM with firewall) for traffic inspection, no VM is needed.
Episode 24
Dos and DDos, denail of service, which flood the target with a lot of requests.
The basic DDos is automatically enabled for azure, e.g. app service, to block malicious traffic while allowing legitimate traffic
It also prevents the app service from unnecessary scale-up.
Standard DDos protection plan can be created separately to provide extra protection. Just need to create a DDoS plan resource and associate it with the virtual network to be protected.
episode 25
Identity, represents user, user group, application
Authentication, the process to verify an identity
Authorization, the process to verify an identity has access to something
Azure Active Directory (AD)
- provide identity management
- access management, roles, role assignment, authentication and authorization settings
- can be used by multiple platforms, azure, office 365, live.com etc.
- can sync with on-prem AD
- support multi factory authentication MFA
-- verify identity using multiple factors, password, phone, token, fingerprint, gps, etc.
episode 26
Azure security center
provide security management services
continuous security assessment and recommendation, the recommendation is also integrated in Azure Advisor
two tiers:
- free, no Azure defender, only assessment and recommendation
- paid, with defender, also security protection, scanning, just-in-time access etc.
episode 27
Key Vault,
A service for storing sensitive information
- keys, like disk encrytion keys
- secrets, username/password
- certificates
integrated with azure services, vm, logic app, data factoyr, etc.
episode 28
Role-based Access Control RBAC
Role, what you can do, read, write, owner, etc.
security principal, a user, group, service principal, managed identity
scope, where you have access to
-different levels, management group, subscription, resource group, resource
episode 29
lock is for locking resources from deletion or modification
lock applies to subscription level and down, not management group
lock supports only 'delete' or 'read only' types
only owner and admin can managed locks
episode 30
tags, key-value pairs to assigned to a resource
can't inherit tags to sub resources, except using policy to allow tag inheritence.
extra properties associated with resources for ,e.g. billing purpose, tracking by tags (department, type of resources, etc.)
episode 31
policy is an if/else check (e.g. location=AU), and return effect (accept / deny / audit, etc.)
initiative is a policy bundle
policy can applied to different levels/scopes, management group and down
for compliance, security, cost management, etc
Note, policy is about resource properties, and RBAC is about user's action
episode 32
azure blueprint
it describes a package of components (common roles, policies, arm template for resources, etc) that apply to a resource group.
The components are also called artifacts. When creating a blueprint, you need to provide those artifacts, e.g. the JSON template for a resource.
Once create a blueprint (or use an existing Azure blueprint, e.g. iso9001 blueprint) you can assign it to a resource group, and those roles policies, etc. will be applied automatically. It will also ask about the parameters of the artifacts, e.g. resource group name
episode 33
Cloud Adoption Framework is a set of tools, best practices, guidelines and documentation for moving to the cloud
It contains a few stages:
- strategy, why move and how to provide business value
- Plan, actionable plan
- ready, prepare and align users, processes and environment
- adopt, start moving by plan, migrate/innovate
- govern, compliance security standards
- manage - manage monitor and optimize environment
episode 34
Microsoft Privacy Statement, use of personal data for all MS offers, cloud, office, etc.
Online Services Terms, license terms for online products, e.g. cloud, office365
data protection addendum, appendix of online services terms to further specify processing of personal data
Trust center, one stop for reviewing security, privacy and compliance of online services
Azure compliance documentation, compliance doco just for Azure
Azure sovereign regions, us government, china, special isolated regions.
episode 35
cost affecting factors
- resource types
- services, e.g. enterprise, web direct, csp from partner, billing cycle, discount, etc
- location/region
- bandwidth - network traffic for uploading and downloading data
Episode 36
reservation, reverse resource for 1 or 3 years in advance
spot pricing, unused VM, but can lose anytime
hybrid use benefit, use existing license
Pricing calculator
TOtal Cost of Ownership TCO calculator, compare on-prem vs azure
episode 37
cost management
cost analysis, reporting dashboard
cost budget, set budget for period, e.g. month, quarter
cost alert, alert on cost
cost recommendataion
episode 38
SLA, calculated monthly
each service has its own SLA, from 99% to 99.999%
free service doesn't have SLA
composite SLA is the multiplication of every resource in an application
to improve SLA, add in redundancy/load balancing, add availability zones, improve service tiers
episode 39
service lifecycle
development, public preview, general availability
there may be private preview during development as well
public preview is for testing purpose