Lumerical Palmetto

Drafted by Michelle, modified on 06/13/23

Given the challenges associated with opening an Ansys GUI on Palmetto, I propose creating the file on your personal computer and subsequently uploading it to Palmetto. Consequently, I will break this process down into three separate segments:

1. The procedure to construct a project locally.

2. Steps to upload the file to Palmetto.

3. Guidelines for running the file on Palmetto.

• Taking the ring resonator as an example: Click the New project following the varFDTD bar and create the project.

https://optics.ansys.com/hc/en-us/articles/360042800293-Ring-resonator-getting-started-Design-and-initial-simulation

The first step is setting up the model (create a ring resonator):

1. Body

2. Structure

Press on arrow on the STRUCTURES button, and select a RECTANGLE from the pull-down menu. 

In 'Geometry', define the value of x, y, and z to decide the center location of the rectangle. x span, y span, and z span decide the size of the rectangle. Other parameters depend on the special simulation.

In 'Material',  we can choose the index from the library that Lumerical provides, or custom our own material.

For the other two options,  we can change the value depending on our requirements. If not necessary, it is better to keep the default value.

The second step is setting up the simulation (FDTD solution):

Lumerical provides different kinds of solvers and in the most condition, we will use 'Mode' and 'FDTD'.  

• Mode (https://optics.ansys.com/hc/en-us/articles/360020687354): It has three kinds of solvers at a very high probability to be used. 

  2.5D varFDTD solver:  https://optics.ansys.com/hc/en-us/articles/360034917213-varFDTD

  Finite Difference Eigenmode (FDE) solver: https://optics.ansys.com/hc/en-us/articles/360034917233-FDE

          EigenMode Expansion (EME) solver: https://optics.ansys.com/hc/en-us/articles/360034396614-EME 

• FDTD (3D)

Finite Difference Time Domain (FDTD) solver: https://optics.ansys.com/hc/en-us/articles/360034914633-Finite-Difference-Time-Domain-FDTD-solver-introduction

The third step is setting up the source:

• Plane-wave and beam source: Plane-wave sources are used to inject laterally-uniform electromagnetic energy from one side of the source region. In two-dimensional simulations, the plane wave source injects along a line, while in three-dimensional simulations the plane wave source injects along a plane.  https://optics.ansys.com/hc/en-us/articles/360034382854-Plane-wave-and-beam-source-Simulation-object.

• TFSF source: The TFSF source is often used to study scattering from small particles illuminated by a plane wave. https://optics.ansys.com/hc/en-us/articles/360034902093-Total-Field-Scattered-Field-TFSF-source-Simulation-object

• Mode source(*):  The mode source is used to inject a guided mode into the simulation region. In three-dimensional simulations, the modes are computed across a plane, while in two-dimensions they are computed across a line. From a list of possible modes, a single mode is selected for injection into the simulation region. https://optics.ansys.com/hc/en-us/articles/360034902153-Mode-source-Simulation-object

• Dipole source: Oscillating dipoles act as sources in Maxwell's equation to produce electromagnetic fields. Dipoles are used to simulate point source radiators, such as radiation from a fluorescent molecule. https://optics.ansys.com/hc/en-us/articles/360034382794-Dipole-source-Simulation-object

• Import source: The import source allows the user to specify a custom spatial field profile for the source injection plane. The custom field profile can be calculated from an analytic formula, imported from another FDTD simulation, or from other simulation tools. https://optics.ansys.com/hc/en-us/articles/360034383014-Import-source-Simulation-object

The fourth step is setting up the monitor:

• Refractive index monitor: Index monitors records the n and k value as a function of frequency/wavelength in a simulation. In the future, the index monitor will be able to capture the time evolution of the physical properties profile for nonlinear media. https://optics.ansys.com/hc/en-us/articles/360034383134-Refractive-index-monitor-Simulation-object

• Field time monitor: These monitors provide time-domain information for field components over the course of the simulation. Time-domain monitors can consist of point, line, or area monitors to capture this information over different spatial extents within the FDTD and varFDTD simulation regions. https://optics.ansys.com/hc/en-us/articles/360034902353-Field-time-monitor-Simulation-object

• Movie monitor: Movie monitors capture a desired field component over the region spanned by the monitor for the duration of the simulation. Movie monitors are only available in the two-dimensional variety (and only z-normal for propagator simulations). The resultant movies are saved with the same name as the monitor in the current working directory. https://optics.ansys.com/hc/en-us/articles/360034902373-Movie-monitor-Simulation-object

• Frequency-domain Profile and Power monitor: Frequency-domain field monitors collect the field profile in the frequency domain from simulation results across some spatial region within the simulation in the FDTD, varFDTD solvers. https://optics.ansys.com/hc/en-us/articles/360034902393-Frequency-domain-Profile-and-Power-monitor-Simulation-object

• Mode expansion monitor: Mode Expansion Monitors use overlap analysis to calculate the forward/backward propagating components of any mode of a waveguide or fiber at an arbitrary location in the simulation region. https://optics.ansys.com/hc/en-us/articles/360034902413-Mode-expansion-monitor-Simulation-object

• Port: Ports can act as a combination of mode source, field monitor, and mode expansion monitor. Ports can be used alone or in conjunction with the S-parameter sweep tool to extract the S-parameters for a device by using the mode expansion method. https://optics.ansys.com/hc/en-us/articles/360034382554-Ports-FDTD-Simulation-Object

The Fifth step is clicking the run button:

Before this operation, we can config the simulation resources. Changing the value of process and thread to accelerate the simulation job. 

   https://optics.ansys.com/hc/en-us/articles/360058790674-Resource-configuration-elements-and-controls

The sixth step is saving the project:

Then we will get the <file-name>.fsp file, we can load this file into the Palmetto and run it.

2. Steps to upload the file to Palmetto.

log in to the Palmetto and scp the local file into Palmetto.

For example,

>> ssh miaoxiy@login.palmetto.clemson.edu

>> Enter the password

>> cd /project/twei2/home/michelle

>> scp <file folder path>  ./

3. Guidelines for running the file on Palmetto.

Open Google, search 'Palmetto Documentation', and enter this website.  Then click the arrow at the right of 'Connecting', click 'Open OnDemand', click 'Introduction', and click a link 'openod.palmetto.clemson.edu'.  

In the upper bar, click 'Interactive Apps' and choose 'Palemtto Desktop'. Then we need to apply for the access. 

In the Resolution, it is upto the user.

Number of resource chunks means the number of PC is needed.

CPU cores per chunk(ncpus) and amount of memory per chunk is depending on the requirements.

Note: it is better to choose any in the Interconnect option, otherwise you will cann't find the PC.

Then click 'Launch'.

•  I found that if we apply for 128 cores per chunks, it will be very hard to success.

After this,  we will see the following figure, click 'Launch Palmetto Desktop'.  Open a terminal and input:

>> ping -c 4 license4.clemson.edu

>> module load lumerical/2021.2

>> scp /project/twei2/home/michelle/<file>.fsp ./

>> fdtd-solutions <file-name>.fsp  (i.e fdtd-solutions ring_resonator.fsp)

Then opening the Ansys GUI, we can click the run button directly.