Skillmatrix

Terry is a seasoned Linux Ninja with 18+ years of hands-on experience and great industry track record. Experienced in architecting and implementing highly-available, scalable, high performance and secure enterprise-grade infrastructure for running mission critical workloads (containerised ;-).

3 fundamental things about me:

1 - passionate about Linux (kernel, networking, storage, security) and open source technologies

2 - love to learn, experience and build cool things

3 - love to share knowledge and help people/organisations deliver outcomes drive success


Philosophy

- KISS

- Back to basics (focus on fundamentals, don't get attached to one particular implementation)

- Take ONLY what you need (to solve problems) without introducing unnecessary complexities

- The best tools are the ones that do what you need at the best cost ;-)

- Knowledge increases by sharing, not by saving.


Sometimes the best engineering work comes before hands touch the keyboard - the whiteboard session. - Kelsey Hightower

NOTE: I don't profess mastery but I'm adept at most of the listed, and I have an aptitude towards learning what I need to ;-D (red+bold > bold > normal text)

profile for Terry Wang on Stack Exchange, a network of free, community-driven Q&A sites

Stack Overflow flair ;-)

Network / Infrastructure Architecture

cat > LOOK.OUT <<'EOM'

Look at Reference Architecture I work with on a daily basis (Citrix / Virtualization era).

In real life they are more complicated ;-)

EOM

- AWS VPC / Azure Virtual Network (network virtualization)

Connectivity model: IPsec VPN, AWS Direct Connect, VPC Peering, WireGuard (self-hosted, Tailscale), OpenVPN (least preferred for site to site), Nebula (secure & scalable overlay network)

- On-Prem (e.g. XenServer networking)

  • Segregating different types of network traffic (management, VM and IP-based storage traffic)

  • Open vSwitch as network backend (with OpenFlow control and optional DVSC)

  • VLANs

  • VXLAN (Linux kernel falvoured)

  • Overlay network in the containerization era (TCP over UDP ;-)

  • NIC bonding

  • basic OVS modes - active-active (balance-slb) and active-passive

  • LACP bond with load balancing based on IP and port of source and destination, or based on Source MAC addresses

  • multipathing (Linux DM-Multipath - does NOT work for NFS - which needs NIC bonding for better throughput)

  • Jumbo Frames (MTU=9000, NO! Not that simple, very tricky)

  • High Availability

  • Failover (network, storage, running VMs, hosts, etc.)

  • Disaster Recovery design

  • Distributed Web (IPFS, IPNS, DNSLink, etc.)


API / Web Architecture

- DDoS protection and Application Firewall

- CDN

- Load balancing

- Caching

- Web Server

- Middleware

- RDBMS


Container Orchestration, Management & Ops


Battle-tested, self-{hosting,managing} k8s since Feb 2018, k8s v1.9

K8s distributions: kubeadm, gravity (AKA telekube) flavoured, EKS Anywhere

CKA (v1.20) certified in April 2021 (finally decided to do it before a voucher expired).


Kubernetes (k8s) provisioning tooling

  • minikube

  • KinD (k8s in Docker)

  • kubeadm


K8s ecosystem technology stack and tooling

- Istio (Service Mesh framework)

- Kuma

- Knative (framework for Serverless workload on top of k8s)

- Docker CE (Dockerfile, Docker Compose)

- Podman - daemonless container runtime engine with orchestration support (work seamlessly with Docker Compose since 3.0.0), easy & flexible workflow migrating to k8s

- Buildah / Skopeo

- Container Runtime (CRI, OCI): containerd, CRI-O/runc (crictl), nerdctl

- CNI (flannel, cilium, wormhole)

- CSI

- rook (Storage orchestrator for k8s)

- OpenShift Origin (renamed to OKD - Container Application Platform) / minishift

- Managed k8s flavours: GKE, EKS, AKS

- On-premises flavoured managed K8s: Anthos (contains GKE on-prem) / AKS on Azure Stack HCI / EKS Anywhere


Linux (distro hopping journey)

My opinion towards Linux distro wars (it ended, ended up well, dominating in the Cloud, in datacenters, powering embedded and mobile devices ;-)

- All Linux distributions are the same: the Linux kernel, glibc and a bunch of GNU utils.

- "Adopting Linux like a religion is stupid." - Thomas Cameron

- "We've all been through my distro is better than yours" thing.

- "The best Linux distro is the one that does what you need at the best cost."

- Linux is NOT about the distribution but the kernel, it's about what the kernel can do.

- Distribution is a way to wrap up what the kernel can do into a more manageable way.

A battle-tested Linux Ninja who started his journey with Mandrake Linux 8.1 back in 2001, black belt level knowledge/skills, SME for the following areas

  • System Administration (Arch Linux, Fedora, Ubuntu, Debian, CentOS, RHEL, Oracle Linux, openSUSE, NixOS, Alpine Linux)

  • Storage / Volume Management

  • device mapper, md/mdadm

  • LVM

  • Btrfs (perfect for self-hosted home storage with mixed specs cheap HDDs, @btrfs maintainer, serious btrfs user since 2008)Perfect for self-hosted home storage solutions with cheap HDDs (mixed specs) using old PCs, as long as users know what to use & when to avoid raid{5,6}.

  • ZFS (OpenZFS / ZFS on Linux)
    There is a learning curve and cost involved to get ZFS right

  • iSCSI (tgt/STGT, Linux-IO Target, SCST, open-iscsi/iscsiadm CLI utility)

  • Open FCoE
    Fibre Channel (HBA) SAN

  • Multipathing (DM Multipath)

  • Ceph (object storage and file system)

  • Troubleshooting (strace/ltrace, kernel crash dump analysis using crash, core dump analysis using gdb, etc...)

  • Performance Metrics (FlameGraph, perf-tools, bcc)

  • Tuning (kernel, filesystem, networking etc...)

  • Monitoring

  • Security Best Practice (very high standard for security and privacy)

  • High Availability (Ksplice, kpatch, DRBD)

  • Automation & Configuration Management (Terraform, Ansible)

  • Virtualization in the kernel mainline (Xen, KVM, LXC/LXD)


Linux Kernel

Extensive knowledge of Linux kernel, have been closely following kernel development since kernel 2.6.32, many thanks to Linux Kernel Newbies.

Specialized in Kernel Crash Dump analysis, capable of tracing back to kernel source code and do RCA (love Linux Cross Reference powered by LXR!).

Real world experience in Live (Dynamic) Kernel Patching tools

    • Ksplice uptrack (Acquired by Oracle, free for Ubuntu and Fedora) - zero-downtime (reboot-less) kernel patching

    • Hands-on since July 2011

  • kGraft (by SUSE, main developer - Jiri Slaby) - source code -> kGraft

    • kpatch (by Red Hat) -> kpatch@GitHub

    • NOTE: casual kpatch contributor, mainly worked on adding distribution support (Ubuntu 14.04 LTS, Oracle Linux 7 and CentOS 7 so far) and improved its documentation. Allegedly the 1st person to get kpatch running on Ubuntu ;-D


Package Managers

  • apt / aptitude / dpkg

    • pacman / yaourt / yay

    • dnf / yum / rpm since Fedora 21 (introduced into Fedora in 18 beta) / rpm

  • zypper / rpm

  • nix (NixOS)

  • homebrew / brew equivalent for Linux, especially non-rolling LTS releases (like CentOS, Ubuntu LTS, Debian stable), perfectly complements distro's native package manager

  • apk (Alpine)

  • opkg

  • AppImage

  • for desktop (GUI) app distribution and management (build a simple and nice workflow focusing on drag & drop, double click)

  • AppImageLauncher

  • > NOTE: personally I don't like flatpak or snap as I am happy with nix + AppImage


Filesysten and Volume Management

  • ext{2,3,4}

    • Btrfs (early adopter)

  • ZFS (on Solaris 10/11.1 and FreeNAS ZFS v5.0 Storage Pool v28, ZFS on Linux now)

  • XFS

  • LVM (LVM2)

  • Stratis

  • LVM + XFS, or backed by any block devices

  • eCryptfs (Enterprise cryptographic filesystem for Linux, filesystem level encryption)

  • dm-crypt / LUKS (disk encryption subsystem in Linux Kernel, part of the device mapper infrastructure, used by Android full disk encryption)

  • FAT16/FAT32/exFAT

  • NFS {v3,v4}

  • SMB / CIFS

  • GlusterFS


Data Recovery

    • testdisk & photorec

    • Powerful data recovery software, opensource. The best in breed I have ever used!

    • DiskGenius

    • As its name indicates. It does a decent job.




Networking

Linux Networking ;-D

NIC Bonding

  • Active-active (balance-slb in XenServer provided by Open vSwitch)

  • Active-backup

  • balance-alb

  • balance-tlb

  • LACP Bond with load balancing based on IP and port of source and destination, or based on source MAC addresses

Jumbo Frame for IP-based Storage Traffic (iSCSI, NFS).

net-tools (legacy the collection of base networking utilities for Linux, deprecated)

iproute2 (a collection of utilities for controlling TCP/IP networking and traffic control in Linux)

Netfilter / iptables / conntrack{,d} - power user

NOTE: nftables (successor of iptables in kernel mainline since 3.13)

shorewall - (iptables made easy) gateway/firewall configuration tool for Linux

Linux Bridge (bridge-utils)

Open vSwitch - OVS (network backend of XenServer, virtual switch providing NIC Bonding, VLANs, QoS and etc..)

Floodlight - OpenFlow controller and Citrix XenServer DVSC (Nicira)

SD-WAN

k8s CNI add-ons / Overlay Network (VXLAN)

- Flannel

- Cilium + Hubble (eBPF based, in-kernel ;-)

- Weave Net

- Open vSwitch (okd, OpenShift default)

- Calico (Canal)

Global Overlay Network (VPN site-to-site, host-to-site, host-to-host)

- Nebula


Networking Protocols

SSH (OpenSSH implementation), HTTP (including HTTP/2), TCP/IP, SSL/TLS, IPsec, IKE, DHCP, SMTP/ESMTP, DNS, NFS, SMB, ICMeP, BGP.

Switching and Routing basics, NAT, etc.

Commonly used CLI utilities

  • nmap

  • tcpdump (network traffic dumper in CLI)

  • Wireshark (AKA Ethereal) / tshark command line utility

  • ntop / netstat / ss / mtr / iftop / iptraf / route / ethtool / ethstatus / slurm / nethogs / bwm-ng / sar (sysstat) / lsof / dig / nslookup / host / ngrep ...

  • curl / wget / wput / aria2

  • iperf (iperf 2.x and iperf3, TCP/UDP/SCTP Bandwidth Measurement Tool - love it!)

  • netcat (nc)

  • netstat / ss

  • tc (traffic control)

  • OpenSSH (ssh, sshd, ssh-add, ssh-agent, ssh-copy-id, ssh-keygen, ssh-keyscan, scp, sftp)

  • openssl

    • gnupg (GnuPG - GNU's OpenPGP implementation)

    • mitmproxy / mitmdump / mitmweb

  • ...


DevOps / Automation / Configuration Management / Infra as Code

Chef

Chef Solo, Chef Client (knife, knife-solo), Chef Server.

Bootstrap (shell) scripts: chef-solo-bootstrap

chef-solo-skeleton project (to be added to GitHub)

Chef related tools

  • Berkshelf to manage cookbooks and their dependencies

  • Foodcritics - lint tool

Vagrant - Tool for building and distributing development environment (perfect with Chef Solo)

A list of base boxes for vagrant => vagrantbox.es Oracle Linux 7 x86_64, Oracle Linux 6 x86_64 & Oracle Linux 5 x86_64 base boxes were contributed by me;-)

NOTE: I answer Vagrant, VirtualBox, XenServer, Networking and Linux related questions on Stack Overflow ;-)

Terraform (Infrastructure as Code, Infrastructure Automation) - Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

  • Infrastructure as code

  • Execution Plans

  • Resource Graph

  • Change Automation

Ansible ad-hoc parallel execution tasks, playbook for orchestrated deployment and host configuration management

Packer - similar to Veewee but does not rely on Ruby, more light-weight and flexible (JSON + Shell scripts).

Container Runtimes

- Docker

- containerd

- CRI-O/runc

Docker Compose

LXC / LXD

Kubernetes (Production grade container scheduling and management)

minikube - run k8s cluster locally

kubeadm - DIY build k8s clusters

Fission - FaaS (Function as a Service) for Kubernetes

minishift - run okd / openshift origin locally

Cloud Foundry / Heroku

Veewee - Amazing tool to automate the building of custom Vagrant base boxes. Create definitions (shell scripts + definitions in Ruby) and build! The Oracle Linux {5,6,7} templates were contributed by me ;-)

Jenkins - A Hudson fork, generic automation tool (often considered the de facto CICD tool)

GitHub Actions - (GitHub's CI/CD solution - workflows are just YAML files, with the power of the community)

Argo CD - Declarative GitOps CD for k8s (kubectl diff & kubectl apply with Web UI)

GoCD - Open Source, cloud native, end-to-end visualisation

Tekton - k8s native pipelines

Crypto

Encryption is easy. Key management is HARD.

Key exchange: Diffie-Hellman (DH) key exchange, ECHDE (using Curve25519), X25519 (Elliptic Curve DH over Curve25519).

Ciphers: AES-CBC, AES-GCM, AES-XTS, CHACHA20-POLY1305

Public Key algorithms: DSA, RSA, ECDSA (P-256), ed25519 (used by OpenSSH for signing) - it is Edwards-curve DSA over Curve25519, a EdDSA variation

MAC: MD5, SHA-512, POLY1305-AES

AEAD (cipher + MAC)

Forward Secrecy

Log Management Solutions

Log Collection / Parsing / Search & Analytics / Visualisation

    • Elastic Stack

    • Elasticsearch, Kibana, Beats, Logstash

    • Managed Elasticsearch - Amazon OpenSearch

  • rsyslog / syslog

  • syslog-ng


Knowledge Management (Wiki) & Issue Tracker

All self-hosted

    • Confluence (started from 2.0, all the way up to 5.1.3)

  • gollum (Git + Markdown + Sinatra)

    • Jekyll / Octopress (Vim + Git + Markdown)

    • JIRA as issue tracker (hands-on administration skills from 3.12.1 to 4.2)

  • Mediawiki (fuck it)

  • vimwiki

  • wiki.js


SCM (Source Code Management) / Version Control

  • Git - git power user

  • Best version control - the most important invention by Linus Torvalds ) / GitHub (social coding;-)

  • GitLab => Self hosted Git Management Software (since version 4.0)

  • SVN - basic working knowledge

  • CVS (self hosted CVS Windows using CVSNT which is no longer free now. Forget about CVS, move on!)


Programming / Scripting

Shell Scripting (Bash)

- GNU Bash

- fish shell user

Python (read, modify, debugging) - proficient working knowledge

NOTE: XenServer core XenAPI and utilities are heavy user of shell/python scripts. Some core components like sm (storage manager) are written in Python (83.1%) and Shell (13.1%).

Ruby (user)

RubyGems, RVM, rbenv + ruby-build, bundler

Beginner, started to learn Ruby when coming across Homebrew (package manager for OS X), OpsCode Chef and Jekyll + Octopress , Sinatra (web application framework/lib, DSL, similar to the well known rails).

Go

Working knowledge with k8s / istio source down to source code. Experienced in maintaining xxxctl CLI utility written in Go.

Rust

rustc / cargo / rustup

Working knowledge, picking up ;-)


Editors / IDE

  • Vim (vim-plug Vundle.vim as plugin manager, moving to neovim soon)

    • Visual Studio Code

  • Sublime Text 3

  • RubyMine 4/5/6 (Ruby IDE)

diff / merge tool

    • vimdiff

    • meld (GUI)

JavaScript, JSON (used by Chef Cookbooks, Chef Server, MongoDB)

Markdown - heavy user for technical writing / documenting. Pandoc as universal document converter.

C

Capable of reading (crash dump analysis work - tracing down to kernel source code for RCA) , core dump related, strace (Linux system call related stuff), modifying and compiling C codes using GCC (LLVM + CLANG on OS X).

Java SE / Core Java (Swing/Socket//Networking/JDBC etc...)

Java SE and JVM Troubleshooting and Performance tuning (Sun JVM and JRockit Mission Control - now a part of Java SE 8).

Tools - Profiler/Performance Tuning Tools

JProfiler, YourKit Java Profiler, Eclipse Memory Analyzer (MAT, formerly SAP Memory Analyzer), IBM HeapAnalyzer, Samurai (Thread Dump viewer), ThreadLogic (Oracle), jconsole, jvisualvm.

Java IDEs

  • Eclipse (Window Builder, egit)

  • JBuilder

  • Oracle Enterprise Pack for Eclipse (OEPE) 11g


Cloud (IaaS) stuff

  • AWS

    • EC2, EBS, EFS, S3, VPC, VPN, Direct Connect, ELB, RDS, SQS, IAM, Route 53 (DNS), etc.

    • Azure & Google Cloud Platform equivalents

  • Citrix Accelerite CloudPlatform / Apache CloudStack

  • Cloud Foundry (with its micro VM by VMware)

  • Heroku (PaaS)

  • Google Apps (SaaS)

    • Iaas/VPS - RAM Host (OpenVZ) / BuyVM (OpenVZ) / 123Systems (Xen and KVM) / Digital Ocean (KVM)

Virtualization

  • VirtualBox {2,3,4,5,6}

  • VMware Workstation {3,4,5,6,7,8,9,10} (since version 3, 2001)

    • VMware OVF Tool (CLI)

  • VMware Server {1,2} (formerly GSX Server, now freeware)

  • VMware ESXi Server 4.0+

  • LXC/LXD

  • Docker

  • Xen Hypervisor 4.x

  • Citrix XenServer {6.0,6.0.2,6.1,6.2,6.5,7.0} => Black Belt

  • vGPU with NVIDIA GRID K1/K2, PCI Pass-through, GPU Pass-through, USB Pass-through (with both xl pci-* XAPI and libvirt + virsh).

  • KVM/QEMU (with libvirt & virt-manager GUI)

  • CoreOS (Linux Kernel + systemd + LXC + Docker + Btrfs)

Parallels

Microsoft Virtual PC/Virtual Server/Hyper-V


Proxy / Cache

  • Varnish Cache (HTTP Accelerator, Reverse Caching Proxy, Server Side cache/proxy)

  • Nginx as reverse proxy and cache server

  • Squid (Proxy Server, Forward Proxy)

  • Dante (SOCKS Proxy)

  • Memcached (in-memory key-value store)

  • Redis (in-memory key-value data store, support more types)

CDN - Content Delivery Network

CloudFlare / AWS CloudFront / GCP Cloud CDN


Web Server / Application Server

    • Apache HTTP Server (httpd) {1.3.x,2.0.x,2.2.x,2.4.x} + mod_pagespeed

    • Nginx 0.7.x - 1.22.x (optional ngx_pagespeed) tracking stable and mainline

    • Personal favourite web server, have been using it to serve personal web site and various applications since it's 0.7.0 release.

    • Hands-on performance tuning, security hardening (securing traffic using SSL/TLS certificate, HSTS, etc.) experience.

    • NOTE: My personal web sites are rated A+ by SSL Lab's SSL Server Test ;-D

  • Caddy (powerful, enterprise ready open source web server written in Go)

  • Microsoft IIS {5.0,6.0,7.0}

    • Apache Tomcat {3,4,5,6,7,8}

  • Websphere Application Server 5/6

  • Oracle Web Tier (HTTP Server + Web Cache + OPMN - Oracle Process Manager and Notification Server)

    • Varnish (Web Cache, HTTP Accelerator, Reverse Proxy, Server side cache/proxy)

    • Jetty (light-weight HTTP web server and Java Servlet container)

  • Cherokee

    • HTTP File Server (HFS)


Load Balancing

  • HAProxy - Layer 7 (HTTP / application layer) and Layer 4 (TCP/UDP) Load Balancer

  • User space, slower but more flexible, easier to customize and tweak.

  • LVS (Linux Virtual Server) / IPVS - Layer 4 (IP packets and UDP datagrams)

  • Works in kernel space, fast, high performance low latency but not very flexible.

  • Nginx - all purpose layer 7 and layer 4 load balancing

  • keepalived (VRRP implementation for virtual router failover and load balancing)


Infrastructure

  • OpenSSH

    • Have been closely following OpenSSH since 6.7, 8 years+ experience dealing with OpenSSH.

    • Safe and secure with Forward Secrecy (PFS) provided by Diffie-Hellman key exchange, which the NSA hates.

    • Make sure you check this example ~/.ssh/config [Gist] NOTE: sshd configuration required for it to work.

    • Powered by OpenSSH Manuals (RTFM), and Linux Networking Cookbook and SSH (The Secure Shell) - The Definitive Guide.

    • Linux Ninja cannot live without this thing. Not only a secured communication channel, but also port forwarding (including X11 Forwarding), dynamic (application level) port forwarding (creates a SOCKS proxy), can be used to work around firewalls (including GFW, that when I started digging into SSH).

  • Jumpserver (4A Open Source SSH Bastion, MFA support, also supports RDP, VNC, telnet, etc.) web UI / CLI

  • MTA {Postfix,Exim}, Mail Server Solution - Zimbra Collaboration Server,


IP-based file system

  • NFS {v3,v4}

  • SMB / CIFS using Samba (re-implementation on Linux)


VPN

  • PPTP - PPP between hosts via GRE. MPPE encryption and MPPC compression, MS-CHAP v2 security. Solution for Linux: pptpd + iptables (do NAT).

  • L2TP/IPsec - xl2tpd (establish the tunnel) + openswan/libreswan for encryption

  • IPsec VPN - strongSwan, some of my early >>>strongSwan (IPsec VPN) instances<<< ;-D
    NOTE: capable of Install (build from source) and configure strongSwan on Linux from scratch for remote access, Site-to-Site and Host-to-Host scenarios. Client configurations on most modern operating systems, solid knowledge of IKEv1/IKEv2 and IPsec. Also familiar with ipsec-tools (Racoon IKE daemon).

  • OpenVPN (SSL VPN, use OpenSSL for encryption, fast, highly configurable)

  • WireGuard (in kernel since 5.6, simply, fast, secure, modern VPN that utilizes state-of-art crypto, Linus Torvalds called it `work of art`)

  • FTP proftpd/vsftpd (Linux) | Serv-U FTP Server/IIS (Windows) | Forget FTP, please use SFTP instead!!!

  • DNS (bind/named, dnsmasq, CoreDNS, Amazon Route 53)

  • Working knowledge of SPF, DNSSEC, DKIM, DMARC, DNS over HTTPS, DNS over TLS and DNSCrypt

  • LDAP

  • DHCP

  • NTP (now bloody systemd takes care of NTP...)

  • TFTP (not useful any more as iPXE supports HTTP, NFS, FCoE, iSCSI etc, if TFTP is really needed for legacy PXE, dnsmasq does the job)


Firewall / Netfilter (iptables / nftables)

iptables power user and practitioner (thanks to the GFW!?) ;-D


LDAP/Active Directory/Identity Management


  • Apache Directory Server / Studio

  • OpenLDAP 2.x

  • Oracle Internet Directory 11g

  • Sun Java System Directory Server 5.2 (AKA iPlanet DS, Netscape DS, now Oracle Directory Server EE)

  • Novell eDirectory 8.8 SP5

  • Microsoft Active Directory 2000/2003

Administration Tools: Apache Directory Studio, OpenLDAP client CLI -> ldap{search,add,delete,modify,passwd}

Single Sign-on/SSO

  • Windows Integrated Authentication (WNA, WIA)

  • Oracle Access Manager 11g (OHS + WebGate + WebLogic Server 11g + WebCenter 11g + OAM11g + OID 11g)

High Availability / Clustering / Disaster Recovery

WebLogic Server Cluster

WebLogic Portal Cluster Architecture (Zero downtime Architecture)

WebCenter 11g High Availability (clustering, failover)

Oracle Web Tier (OHS) Runtime Clustering

DRBD - (Network based RAID 1) - typically used for MySQL (DRBD + Pacemaker + Corosync) replicating over network, block device level replication.

keepalived (VRRP)

XenSever High Availability / DR

Apache Cassandra Cluster (Datastax OpsCenter as management tool)

Database


Oracle Database 10gR1/10gR2

Oracle Database 11gR1/11gR2 (RAC)


MySQL {4.x,5.x} (I've been managing LAMP environment since Fedora Core 1 era)DBA, backup/restore, replication, High Availability using DRBD + Pacemaker + Corosync.


PostgreSQL


SQLite 3 / SQLite CLI


PointBase (embedded in WebLogic Server version, 10.3.2.0 and earlier. Since 10.3.3.0 it is replaced by Derby)


Apache Cassandra (Install / Configure / Clustering / Monitoring)

Hands on experience. Familiar with monitoring tools like DataStax OpsCenter Community Edition (free), jconsole and jvisualvm (cassandra-webconsole crap...).


MongoDB


Time series database

InfluxDB

TSDB (Prometheus)


Distributed Key-value database

    • etcd (v2 and v3)

    • hands-on with v3, gravitational (gravity shipped), kubeadm flavoured k8s cluster (static pod manifest way).


Monitoring / Metrics / Measuring / Management Tools

Traditional monitoring tools / stack

Cockpit - modern Linux admin interface (easy to use, integrated, glanceable, and open web-based interface for servers.

monit

supervisord / supervisor

goaccess (simple, real-time web log analyzer and visualizer)

monitorix

cacti

nagios

MRTGe

Munin

StatsD (node.js powered) + Graphite

Ganglia

ntop

webmin

weblizer

gnuplot

container / k8s native monitoring stack

  • Heapster + Grafana + InfluxDB + Kapacitor

  • Promethus (node-exporter + kube-state-metrics) + Grafana + Alertmanager + Satellite

healthchecks (self-hosted) cron monitoring tool with Web UI

Application Performance Management - APM

New Relic

Server Density

Free Open Source Software/Solutions

LAMP (Linux+ Apache + MySQL + PHP)

LEMP (Linux + Nginx + MySQL + PHP - php-fpm)

Jumpserver (4A Open Source SSH Bastion solution)

teleport (gravitational) - secure remote access gateway

  • Cockpit

  • OpenSSL

  • GnuPG / OpenPGP

  • Keybase

  • keys (keys.pub)

  • Ksplice / kpatch

  • FreeIPA

  • Jellyfin (Plex like)

  • Scrutiny (S.M.A.R.T. Monitoring)

  • Bitwarden

  • mod_pagespeed for Apache (by Google)

  • XAMPP

  • Apache JMeter (load test)

  • Apache Tomcat

  • Discourse

  • Discuz! (php)

  • WordPress

  • Octopress

  • Jekyll / Hugo / Hexo

  • Duplicati 2

  • dupGuru

  • rclone

  • rbenv + ruby-build

  • homebrew

  • MovableType

  • phpMyAdmin / phpPgadmin

  • phpsysinfo

  • AWstats

  • Gregarius

  • ......

PDF processing

Ghostscript

img2pdf + ocrmypdf


Gravitational / gravity flavoured k8s stack

Package Kubernetes clusters as images and run them anywhere, from cloud to ground (on-prem - that's why it is named gravity?)

Skill set required to keep k8s + gravitational stack up and running

    • Linux (kernel, sysadmin, networking, security, storage, etc.) skills, at least intermediate (6 years worth of hands-on problem-solving experience)

    • Infrastructure architecture design knowledge and experience: architecting, implementing and supporting highly available, scalable and high performance and secure enterprise grade infrastructure for running mission critical workloads

    • iptables / IPVS (or LVS) in k8s v1.11.0+ (kube-proxy modes)
      - Understanding iptables' limitations and disadvantages in large clusters
      - experience with eBPF based Cillium + Hubble is a great plus (replacing kube-proxy)

    • Storage: LVM2, btrfs, ZFS, iSCSI, FCoE, NFS, SMB, DRBD, Gluster FS, etc.
      - Rook fundamentals (storage operators for k8s, storage orchestration)

    • DNS (dnsmasq, CoreDNS), Service Discovery, Service Registration

    • Shell scripting (Bash)

    • Kubernetes (k8s) - Container Orchestration
      - kubeadm
      - other flavours (gravity)

    • Istio / Kuma (Service Mesh) & Knative (Serverless) for future

    • k8s Deployment (API Object) Strategies
      How to do Rolling (Ramped), Blue/Green, Canary, etc.?

    • Gravitational (Telekube) / gravity CLI

    • OpenShift - container application platform / okd (AKA OpenShift Origin) - great plus

    • Container Runtime: Docker CE, containerd (crictl, ctr, nerdctl, moby buildkit), CRI-O/runc

    • Dockerfile / Docker Compose

    • Podman (podman-compose) / Buildah / Skopeo

    • KISS, next generation container toolchain

    • AWS / Azure / Google Cloud Platform / Oracle Cloud / Alibaba Cloud (multi-cloud)

    • Hands-on experience with mainstream managed k8s offerings (GKE, EKS, AKS)

    • Nginx / HAproxy

    • OpenSSH

    • OpenLDAP (or Apache Directory Server, Active Directory, or equivalent)

    • OpenSSL / PKI

    • OpenPGP, GnuPG (good to have)

    • curl / HTTP / TLS / REST APIs / gRPC

    • DB: MySQL, PostgreSQL, Cassandra, InfluxDB (time series database)

    • DevOps knowledge & practices, e.g. Git, GitOps, CI/CD (Jenkins, GitHub Actions) / TnRP

    • Monitoring & Alerting (kube-state-metrics, Prometheus, Prometheus Operator experience is preferred)

    • Elastic Stack

    • Fluent Bit (Logs and Metrics processor and forwarder)

    • Some Java, JVM, Tomcat or similar application server knowledge

    • Some Python, Go

    • Heroku

    • etc.


Middleware (Portals/Content/SOA/BPM)

Why do you even read this? ;-)

Oracle Fusion Middleware 11gR1 (11.1.1.x)

Repository Creation Utility

WebLogic Server

WebCenter Portal

WebCenter Content (formerly UCM/ECM, AKA Stellent) 10gR3/11g

SOA Suite (BPM, Service Bus)

Identity Management (OID/OVD, Access Manager - SSO)

Oracle Enterprise Pack for Eclipse (OEPE, AKA WebLogic Workshop or Workshop for WebLogic)

Oracle WebCenter Interaction {5.x,6.0.x,6.1.x,6.5.x,10.3.0.x,10.3.3.x} (AquaLogic User Interaction Suite, ALUI, Plumtree, WCI)

WebLogic Server 6, 7, 8.1.x, 9.x, 10.x, 10gR3 (10.3.0.0), 11gR1 (10.3.2.0 up to 10.3.6.0), 12c (12.1.1)

Oracle Certified Associate - WebLogic Server 11g System Administration (1Z0-102)

WebLogic Portal {10,10gR3,10.3.2,10.3.4,10.3.5}

Oracle Secure Enterprise Search {10gR1,11.1.2,11.1.2.2.0}

Oracle BPM (Fuego BPM, ALBPM) 5.5/5.7.x/6.0.x/10gR3 (I have no experience with 11g as it is JDeveloper based now-_-)

Oracle JRockit Mission Control 4.0 (JRockit JVM)