High Performance Computing

Handling big data with high-performance computing (HPC) in R involves leveraging specialized packages and techniques to process large datasets efficiently, often using parallel processing and distributed systems. Below is a concise overview of key approaches, tools, and techniques for implementing big-data solutions in R with HPC, tailored to your query context.

READ MORE

Table of Content

Introduction

Big Data with High-Performance Computing (HPC)

Handling Big Datasets in R

• Introduction

• Big Data import/export in R

• Large data wrangling with {data.table}

• Using {disk.frame} for large datasets

• Data processing with DuckDB in R

• Working with PostgreSQL in R

Parallel Data Processing and Analysis

• Introduction to parallel data processing and analysis

• Data processing and analysis with {parallel}

• Data processing and analysis with {future}

• Data processing and analysis with {snow}

• Big data processing and analysis with {doParallel}

Distributed Computing for HPC Clusters

• Introduction to distributed computing systems in R

• Distributed computing with {SparkR}

• Distributed computing with {sparklyr}