NOTE
ここでは、休日のコーディングの際に作成したツール等の覚書きを記述します。
[全般]
特定のプログラミング言語を習得しようとするとき、その言語の下記をおさえればなんとかなる。 (※)
・プログラミング言語のバージョン管理システム。(e.g., Python の pyenv)
・ロギングの仕組み。 (デバッグに使う。) (e.g. Python の logging)
・単体テストツール。 (e.g., Python の unitttest)
・ビルドシステム。(e.g., Java の Ant)
・ファイル入出力の関数。
※ 以下のような問いを考えるのもよい。
観念的な問い: 上記の項目をおさえてもなんとかならなそうなプログラミング言語習得に必要なものは何か。
[R]
ARMA モデルの時系列を生成 (e.g., n: 100, 自己回帰係数 (ar): 0.5, 移動平均係数 (ma): 0.5)
ワンライナー:
N=100;AR=0.5;MA=0.5;Rscript−e"options(max.print=999999);arima.sim(n=N=100;AR=0.5;MA=0.5;Rscript−e"options(max.print=999999);arima.sim(n={N},list(ar=AR,ma=AR,ma={MA}))" | sed -n 5,\p | sed -e 's/ /\n/g' | grep "[0-9]" | grep -v '\[' > arma_p | sed -e 's/ /\n/g' | grep "[0-9]" | grep -v '\[' > arma_{N}_{AR}_{AR}_{MA}
R のバッチ処理コマンド Rscript を用いて、 ARMA モデルの時系列を生成します。
処理の流れ:
入力: N, AR, MA
出力: ARMAモデルの時系列を保存したファイル
Step 1. 関数 arima.sim で時系列を生成。
Step 2. sed -n 5,\$p で必要な行を抽出。
Step 3. sed -e 's/ /\n/g' で空白を改行に置換。
Step 4. grep "[0-9]" で数値がある行のみ抽出。
Step 5. grep -v '\[' で行数を示す表示の行を除外。
Pandas の行ラベル、列ラベル、行番号、及び列番号を一度に理解するための例
import pandas as pddf = pd.DataFrame({ 'a': [1,2,3], 'b': [4,5,6], 'c': [7,8,9] })
# row label and column label[ [ print(df.at[index, column]) for index in df.index ] for column in df.columns]
# row number and column number[ [ print(df.iloc[i, j]) for i in range(len(df)) ] for j in range(len(df.columns))]
角度データの取り扱い (極座標の散布図)
import numpy as npimport matplotlib.pyplot as pltfrom scipy.special import ivr = np.arange(0, 2, 0.01)theta = 2 * np.pi * rfig = plt.figure(figsize=(10, 10))
ax = plt.subplot(111, projection='polar')theta_1 = [2 / (i + 1) * np.pi for i in range(100)]theta_2 = [2 / (i + 1) * np.pi for i in range(100)]r_1 = np.ones(100)r_2 = np.ones(100) * 1.5ax.scatter(theta_1, r_1, color='red')ax.scatter(theta_2, r_2, color='blue')ax.set_rmax(2)ax.set_rticks([0.5, 1, 1.5, 2])ax.set_rlabel_position(-22.5)ax.grid(True)plt.show()
参考 URL, https://matplotlib.org/examples/pylab_examples/polar_demo.html
PairPlot
from matplotlib import pyplot as pltimport numpy as npimport pandas as pdfrom numpy.random import * def create_hist(df, col, fig, num_numeric_cols, subplot_num): plt.rcParams['font.size'] = 5 ax = fig.add_subplot(num_numeric_cols, num_numeric_cols, subplot_num) ax.set_title('histgram {0}'.format(col), size=10) ax.text(0.6, max(df[col])/ 2, df[col].describe(), fontsize=8) ax.set_xlabel(col) ax.set_ylabel('frequency') ax.hist(df[col], bins=50, alpha=0.5) def create_scatter(df, col_x, col_y, fig, num_numeric_cols, sub_plot_num): plt.rcParams['font.size'] = 5 ax = fig.add_subplot(num_numeric_cols, num_numeric_cols, sub_plot_num) ax.set_xlabel(col_x) ax.set_ylabel(col_y) ax.set_title('scatter plot {0} v.s. {1}'.format(col_x, col_y), size=10) ax.scatter(df[col_x], df[col_y], alpha=0.3, s=5) def create_pairplot(df): col_list = df.columns len_df_columns = len(df.columns) fig = plt.figure(figsize=(10, 10)) sub_plot_num_diag = 1 for col in df.columns: print(col) create_hist(df, col, fig, len_df_columns, sub_plot_num_diag) sub_plot_num_diag = sub_plot_num_diag + len_df_columns + 1 sub_plot_num_i = 1 for col_i in df.columns: sub_plot_num_j = 1 for col_j in df.columns: if col_i != col_j: sub_plot_num_i_j = len_df_columns * (sub_plot_num_i - 1) + sub_plot_num_j create_scatter(df, col_i, col_j, fig, len_df_columns, sub_plot_num_i_j) sub_plot_num_j += 1 sub_plot_num_i += 1 df = pd.DataFrame( { 'a': randn(100), 'b': randn(100), 'c': randn(100) }) create_pairplot(df)Categorical Data axis and 3DPlot
import matplotlib.pyplot as pltimport numpy as npimport pandas as pdfrom mpl_toolkits.mplot3d import Axes3Dchars_list = ['a','b','c']regions_x = pd.factorize(pd.Series(chars_list))regions_num_x = regions_x[0]regions_y = pd.factorize(pd.Series(chars_list))regions_num_y = regions_y[0]regions_z = pd.factorize(pd.Series(chars_list))regions_num_z = regions_z[0]
fig = plt.figure()ax = fig.add_subplot(111,projection='3d')ax.scatter(xs=regions_num_x, ys=regions_num_y, zs=regions_num_z)
ax.axes.set_xticklabels(chars_list)ax.axes.set_yticklabels(chars_list)ax.axes.set_zticklabels(chars_list)
ax.axes.set_xticks(regions_num_x)ax.axes.set_yticks(regions_num_y)ax.axes.set_zticks(regions_num_z)
plt.show()
参考 URL, https://stackoverflow.com/questions/24509503/3d-plot-with-categorical-axis-python-matplotlib
Categorical Data and Datetime axis and 3DPlot
import matplotlib.pyplot as pltimport numpy as npimport pandas as pdfrom mpl_toolkits.mplot3d import Axes3Dimport matplotlib as mplimport datetimedatetime_data_ = [ datetime.datetime(2019, 4, 12, 0, 0, 0), datetime.datetime(2019, 4, 12, 0, 0, 1), datetime.datetime(2019, 4, 12, 0, 0, 5)]
datetime_data = [ mpl.dates.date2num(t) for t in np.array(datetime_data_)]
chars_list = ['a','b','c']regions_x = pd.factorize(pd.Series(datetime_data))regions_num_x = regions_x[0]regions_y = pd.factorize(pd.Series(chars_list))regions_num_y = regions_y[0]regions_z = pd.factorize(pd.Series(chars_list))regions_num_z = regions_z[0]
fig = plt.figure()ax = fig.add_subplot(111,projection='3d')ax.scatter(xs=datetime_data, ys=regions_num_y, zs=regions_num_z)
ax.axes.set_xticklabels(datetime_data_)ax.axes.set_yticklabels(chars_list)ax.axes.set_zticklabels(chars_list)
ax.axes.set_xticks(datetime_data)ax.axes.set_yticks(regions_num_y)ax.axes.set_zticks(regions_num_z)
plt.show()
参考 URL: https://stackoverflow.com/questions/47913108/how-can-multiple-scatter-points-be-connected-by-a-line-in-a-3d-plot
zsh の設定 (.zshrc)
source /home/tadachikaoki/antigen.zsh
# history
# refference: https://masutaka.net/chalow/2014-05-18-2.html
HISTFILE=$HOME/.zhistory
HISTSIZE=1000000
SAVEHIST=1000000
setopt extended_history
alias hall="history -E -i 1"
autoload history-search-end
zle -N history-beginning-search-backward-end history-search-end
zle -N history-beginning-search-forward-end history-search-end
bindkey "^p" history-beginning-search-backward-end
bindkey "^n" history-beginning-search-forward-end
# Load the oh-my-zsh's library.
antigen use oh-my-zsh
# Bundles from the default repo (robbyrussell's oh-my-zsh).
antigen bundle git
antigen bundle heroku
antigen bundle pip
antigen bundle lein
antigen bundle command-not-found
# Syntax highlighting bundle.
antigen bundle zsh-users/zsh-syntax-highlighting
# Load the theme.
antigen theme robbyrussell
# auto completion
antigen bundle zsh-users/zsh-completion
# Tell Antigen that you're done.
antigen apply