Search this site
Embedded Files
Skip to main content
Skip to navigation
BEN CHEN's Homepage
Good Day
wiki
Big Data / Cloud
AWS
Setup SSH access on EC2
Start with S3
Lambda
Athena
Glue Studio
DynamoDB get started
DynamoDB connection Boto3 and Python
AWS RDS vs Redshift
AWS notes
Boto3 list AWS S3 bucket objectse
Azure
Data factory
Event Hubs
Databricks - send to & receive from event hubs
data lake notes
managed identity notes
DevOps deploy repo to databricks
Data factory self-hosted integration runtime
service principal
data factory - load to datalake and databricks
Data factory - incremental load - high watermark
Connect to Datalake from Python
Data factory - small things
AZ900 fundamentals from Adam
Data factory - partition files in dataflow
Azure App Service in Python
synapse mount data lake
synapse spark streaming
synapse quick notes
stream analytics job
Azure function
ARM template / Infrastructure as code
Microsoft Graph API
Microsoft Dataverse API via python
Kubernetes
k8s setup a cluster
k8s dashboard
k8s notes
hadoop
Install Hive on Hadoop
Install HBase
Install Hive with MySQL metadata store
install hadoop
firewall for hadoop on ec2
spark
learn pyspark
Streaming late arriving data
small things
Read data files from multiple sub-folders
Split string column in Spark
spark infer and update schema
structured streaming triggers
Selectively overwrite table partitions
spark - add file name to dataframe
spark timestamp add interval
Create Checkpoint on delta table
Databricks
Schedule by CRON expression
REST API
SQL endpooint
Connect to cluster from PowerBI
Secrets and CLI
Query cluster or sql endpoint
Read files in repos
small things
Range query join hint
range join using pandas merge_asof
Databricks spark structure streaming
databricks database and table
delta table
databricks late arriving data
Delta Live Table
databricks notes
databricks functions mis
delta table merge / upsert
databricks parameters
Install jar file in Compute libraries
git on databricks
Download file from DBFS
Create Date Dimension
Mount Fabric Lakehouse / onelake / blob storage
Databricks cluster init script
Unity Catalog notes
clustering, partitioning, z-order
replace table / schema
Delta Lake
Delta lake vs standard parquet file
VACUUM OPIMIZE and CHECKPOINT
snowflake
snowflake get started
assorted tools
MongoDB
DataIku
DataIku WebApp CSS
enconomy
2008 financial crisis
货币起源
ATO super contribution notes
李白
Finance Account Dimensions
Accounting 101
profit & loss, balanced sheet
mt4 basic
linux mis
commands
docker img creation and run
docker intro
Dockerfile example
gunicorn py3
install docker rhel
install flask & gunicorn
Install Flask,Gunicorn & Nginx
install sklearn & xgboost
k8s notes
lvm resize logical volume
move docker default directory
nexus and docker
setup ms odbc
systemctl
Update Internet Interface in Ubuntu
Virtual Machine Network
Windows Subsystem for Linux WSL
install dash on WSL ubuntu
wsl - systemd service
linux commands
Read share folder from Linux / Python
awk extract fields from ls command
grep with regex
programming
codes
CNN dropout
generate gt for rbox
lstm
Neural network dropout
ocr mser
ocr pytesseract
ocr_cv2
text classification
Query Twitter API in Python
template - logistic regression
template - support vector machine svm
template - xgboost
template - ROC PR and R2
template - CNN model with pytorch
example - transformer - existing torch module
example - transformer - decoder only - gpt style
example - decoder-only transformer with training loop
typescript
typescript environment setup
typescript datatypes
typescript function and object
typescript arrow function
typescript class
c#
C# Rest Service Call
C# Rest Service Host
C# Soap RDA
C# Thread
C# Thread Safe
Convert String to Bytes
EDIT EXCEL C#
Query AD without the 1000 limit
R
SAS
Data Step
DecisionTree
DTREE
gchart
gmap
Graph Style
Load data file
logit
model output
ODBC & SQL
Procedures
SGPLOT
react
react helloworld
react building components
Python
bytes image cv2
compile python to exe
Conda
Config
ctc loss
CUDA gpu
cv2 fillpoly
Decorator
Edge Recognition
Flask image post
flask render template
google search
GPS from photo
Heatmap
heatmap2 (meshgrid)
hello pytorch
Install Package
jupyter
jupyter dark theme
jupyter auto scroll
miscellaneous
miscellaneous plots
plot major minor range as ticks
matplotlib format timestamp axis
oracle conn
pandas
pandas update values by condition
pandas plot
pandas join by time range
pandas custom groupby
Open excel spreadsheet
pandas pivot and unpivot
pandas data correlation
pandas groupby and get list
plot img and polygon
Python web and WSGI
PyTorch
Scatter Plot
string similarity
virtual env
word cloud
selenium xpath
multi processing
subfigure / subplot
Reactive Extensions (RX)
reflection / reflective programming
Unix / Posix / Epoch time
matplotlib table
write data to ms sql through pyodbc
Get normal distribution from data
find peak and bottom values from a series
Python OLEDB query
dash
dash helloworld tutorial
dash show static plot figure
multi-page dash app
Ping server and refresh page on changes
dash plot figure performance
ALL and MATCH pattern matching callback
Dagster
Dagster - code structure for jobs
dagster job and op config
Bulk insert rows to SQL
interpolate grid data
SQL Server windows authentication linux python
Log log file to avoid re-running
Kernel density estimate KDE
geopandas
unzip tar file
SVM / SVC in sklearn
numpy
interpolate nan values
add numbers to different dimensions
airflow
airflow install
airflow dag and task
matrix rotation in python
regex
FastAPI
Call API with OAuth
flask website with authentication
flask api with authentication
Package management tools
python async await
miscellaneous
Cisco Data Virtulization
compress pdf
Git
Google Analytics API
Google Vision API
html css
html loading wheel on click
jenkins
Json schema
LightSwitch
OBIEE RPD study notes
oracle regex
Oracle XML Query
Read multi-page TIFF
Silverpop
Tableau quick look
VBA query SQL Server
vlookup
oracle KEEP FIRST/LAST
jQuery
jQuery progress bar
C# async & await & Task
3rd normal form
conceptual logical & physical models
View wifi password on windows
Anaconda installation
read pdf text in python
OSI PI - query data
read write parquet in python
Json read, write and pretty print
VS Code for C# winform
NSSM
Azure Git Repo credential
move window from outside of screen
mysql
Task scheduler runs python script
osi pi asset framework quick overview
OSI PI AF SDK query
InfluxDB
dimensional modeling
power bi
power bi training notes
Power BI python setup
power bi DAX
power bi direct query and performance analyzer
power bi cool visuals
parameterize powerbi data source
powerbi role and row-level security
powerbi semantic model
Enable service principal to access Fabric / onelake
git rebase
ML and Math
Adam & Gradient Descent
ARIMA
auto encoder
bag of words
batch normalization
bayes and maximum likelihood
Canny Edge Detection
Chi-Square Test
connected components
convolution network design
convolution notes
Cross Validation and Grid Search
ctc loss
ctpn
Derivative table
Eigenvector Eigenvalue
embedding
encoder-decoder
Evaluation Metric
Generalized linear models
Goodness-of-fit Test
hinge loss
Hypothesis tests def
kNN
L1 & L2 Penalty
linear algebra
Linear Regression
Logistic Regression Python
logit logloss
Loss function and Eval metric
LSTM
MaximumWeightedMatching
Model measure
naive bayes
nan loss / gradient
negative log loss and cross entropy in pytorch
neural network
non-maximum supression
pack variable-length sequences
position info encoded in CNN
RBOX
Rotate vector
Sampling - Margin of Error
SMOTE
stacking model
SVM
TF IDF
training data different sizes
xgboost
特征工程
Generic Attention & Self Attention
Binomial, Multinomial, Gamma, Beta and Dirichlet
Intuition behind Beta Distribution
LDA
PCA
transformer
east
Fitting exponential decay curve
Why minimax >= maximin
Vector Auto regression VAR
Gelu
Kalman Filter - a simple start
Kalman Filter - a detailed tutorial
R2 Score
Generative AI
Generative Adversarial Network GAN
GPT introduction
RAG
unstructured layout detection notes
SQL Server
Backup and Restore
build & publish sql project
CDC
Change server-level collation
Check DB Lock
Custom Aggregate
Database Encryption TDE
exec job from script
Geometry
partitioned table
Replication
Script index definition
split_string
SQL 2014 In-Memory Table
SQL2012_Exec_Package
SSIS 2014 Deployment
string agg
TDE encryption
TFS for VS2008
Query sql server user permissions
SQL Serv Change Tracking
mining
open cut - truck cycle / load & haul cycle
Underground coal mine on google maps
CHPP process
asset maintenance
Bayer process - bauxite to alumina
Hall-Héroult process - from alumina to aluminium
LC-MS
optimization
Linear Programming
Mixed Integer Programming - A complex example
Optimization tools and Gurobi
Gurobi sample script
Gurobi non-linear example
Charnes-Cooper transformation
chance-constrained / scenario-based optimization
mood
2025-03-14 flying scooter
BEN CHEN's Homepage
data lake notes
data lake notes pdf
Google Sites
Report abuse
Page details
Page updated
Google Sites
Report abuse