字詞頻度表與網路圖

字詞頻度與關係：

import os import sys from operator import itemgetter import subprocess class NewsAnalytics(): dataPath = None def __init__(self, dataPath): self.dataPath = dataPath def NewsTermFreq(self, dataFileName): termDict = dict() dataFile = self.dataPath + "/" + dataFileName + ".txt" if not os.path.isfile(dataFile): return(False) with open(dataFile, "r") as fileOp: sourceData = fileOp.read() v = sourceData.split() for i in range(0, len(v) - 1): key = v[i] if termDict.get(key) is None: termDict[key] = 1 else: termDict[key] += 1 fileOp.closed dataFile = self.dataPath + "/" + dataFileName + "Freq.csv" with open(dataFile, "w") as fileOp: fileOp.writelines("TERM, WGHT, SEQ\n") i = 0 for term in sorted(termDict.items(), key=itemgetter(1), reverse=True): i += 1 fileOp.writelines(u"{0:s}, {1:d}, {2:d}\n".format(term[0], term[1], i)) fileOp.closed return(True) def NewsTermNetwork(self, dataFileName): linkDict = dict() termDict = dict() dataFile = self.dataPath + "/" + dataFileName + ".txt" if not os.path.isfile(dataFile): return(False) with open(dataFile, "r") as fileOp: sourceData = fileOp.read() v = sourceData.split() for i in range(0, len(v) - 1): key = v[i] if termDict.get(key) is None: termDict[key] = i + 1 for i in range(0, len(v) - 2): t1 = v[i] t2 = v[i+1] key = "%s,%s" % (t1, t2) if linkDict.get(key) is None: linkDict[key] = 1 else: linkDict[key] += 1 fileOp.closed dataFile = self.dataPath + "/" + dataFileName + "Net.csv" with open(dataFile, "w") as fileOp: fileOp.writelines("N1, N2, WGHT, TERM1, TERM2\n") for term in sorted(linkDict.items(), key=itemgetter(1), reverse=True): keyPair = term[0].split(",") fileOp.writelines(u"{0:d}, {1:d}, {2:d}, {3:s}, {4:s}\n".format( \ termDict[keyPair[0]], termDict[keyPair[1]], term[1], \ keyPair[0], keyPair[1])) fileOp.closed return(True) if __name__ == "__main__": dataPath = "." dataNews = "ChinaTimes" if len(sys.argv) > 1: dataNews = sys.argv[1] if len(sys.argv) > 2: dataPath = sys.argv[1] dataNews = sys.argv[2] newsAnalytics = NewsAnalytics(dataPath) print("字詞頻度: ", end="") if newsAnalytics.NewsTermFreq(dataNews): print("OK") else: print(" FALSE") print("字詞關係: ", end="") if newsAnalytics.NewsTermNetwork(dataNews): print("OK") else: print(" FALSE")

網路分析：

library(igraph) rm(list=ls()) dataPath <- 'C:/Users/Rich Lee/Documents/News' dataName <- 'NewsNet' dataFile <- paste(dataName, 'csv', sep='.') setwd(dataPath) LoadData <- function(dataPath, dataFile) { dataEdgeList <- read.csv(paste(dataPath, dataFile, sep='/')) dataEdgeList[, 2] <- as.character(dataEdgeList[, 2]) dataEdgeList <- as.matrix(dataEdgeList) dataGraph <- graph.edgelist(dataEdgeList[, 1:2]) E(dataGraph)$weight <- as.numeric(dataEdgeList[, 3]) V(dataGraph)$size <- degree(dataGraph) dataGraph } dataGraph <- LoadData(dataPath, dataFile) closeGraphList <- sort(closeness(dataGraph), decreasing=TRUE) betweenGraphList <- sort(betweenness(dataGraph), decreasing=TRUE) degreeGraph <- centralization.degree(dataGraph)$centralization closeGraph <- centralization.closeness(dataGraph, mode="all")$centralization evcentGraph <- centralization.evcent(dataGraph, directed=FALSE)$centralization print("Closeness:") print(closeGraphList) print("Betweenness:") print(betweenGraphList) figureGraph <- data.frame(varible=c("Degree", "Closeness", "Eigen"), figure=c(degreeGraph, closeGraph, evcentGraph)) bmpFile <- paste(dataName, 'bmp', sep='.') bmp(paste(dataPath, bmpFile, sep='/'), width=2048, height=2048) plot(dataGraph, layout=layout.fruchterman.reingold, vertex.color="yellow", vertex.label.font=24, vertex.label=V(dataGraph)$name, edge.width=E(dataGraph)$weight) dev.off()

Closeness：

司法院研議人民專業司法正義行政院會疏離法官 0.3012048 0.3003003 0.2985075 0.2857143 0.2659574 0.2652520 0.2570694 0.2557545 0.2531646 精密法律公平抗議無罪推定病人刑事現代社會可行性 0.2487562 0.2475248 0.2469136 0.2364066 0.2358491 0.2352941 0.2331002 0.2325581 0.2257336 倚賴華爾街主張價值觀證據主義現代義憤台灣陌生 0.2202643 0.2127660 0.2114165 0.2109705 0.2096436 0.2074689 0.2024291 0.1996008 0.1996008 阿拉伯之春醫生基本普遍司改關鍵兩面之詞僵局白玫瑰運動 0.1934236 0.1919386 0.1908397 0.1845018 0.1845018 0.1838235 0.1824818 0.1824818 0.1814882 民眾基本理念醫病關係靜聽確實熱情判斷調查論壇 0.1801802 0.1703578 0.1694915 0.1680672 0.1677852 0.1672241 0.1666667 0.1658375 0.1587302 僵硬雙方位置高度法定蘋果日報倨傲罪刑誤解 0.1567398 0.1560062 0.1557632 0.1550388 0.1550388 0.1485884 0.1457726 0.1449275 0.1396648 看法關注媒體立法院試行條例 0.1317523 0.1246883 0.1183432 0.1126126 0.1074114

Betweenness：

人民專業台灣民眾司法陌生疏離義憤調查 2232.8333 1529.8333 820.6667 712.0000 671.8333 632.0000 630.0000 560.0000 501.0000 試行條例立法院媒體關注看法誤解蘋果日報白玫瑰運動論壇 477.0000 475.0000 473.0000 471.0000 469.0000 467.0000 465.0000 463.0000 463.0000 基本理念關鍵無罪推定法官正義普遍高度熱情阿拉伯之春 461.0000 459.0000 343.6667 341.0000 335.3333 332.0000 320.0000 318.0000 288.3333 華爾街抗議倨傲僵硬醫病關係司改證據主義罪刑法定 286.3333 284.3333 180.0000 178.0000 176.0000 174.0000 166.0000 161.0000 159.0000 判斷基本價值觀位置僵局靜聽醫生兩面之詞主張 157.0000 125.6667 123.6667 115.5000 114.6667 113.5000 113.0000 111.5000 111.0000 病人雙方確實現代可行性研議倚賴刑事精密 109.0000 93.0000 91.0000 57.0000 57.0000 57.0000 57.0000 55.0000 55.0000 行政院會司法院現代社會法律公平 55.0000 0.0000 0.0000 0.0000 0.0000

varible figure 1 Degree 0.09839477 2 Closeness 0.28996402 3 Eigen 0.89711812