GPU and Tensor

1. Check your CPU architecture

import platform

platform.platform()

2. Check what is your GPU device

import torch

torch.has_mps (or torch.has_cuda or others)

3. Set your GPU device

device=torch.device('mps')

device # check the device you just set

4. In your code, send the computation to the GPU device and send the results back to CPU.

  model = GPT2LMHeadModel.from_pretrained('gpt2')

  model_gpu = llm.eval().to(device)

  tokenize_input = tokenizer.encode(sentence)

  tensor_input = torch.tensor([tokenize_input]).to(device)

  loss= llm_gpu(tensor_input, labels=tensor_input)[0]

  loss_cpu = np.exp(loss.detach().cpu().numpy())