Perl might seem like a daunting language. However, it's very powerful in its own area.
Professionally, I'm using it to parse content from an XML file and generate Java code from this input.
I've been long time looking for a way to decently debug Perl scripts and I think I found a way to do it: use JetBrain's (community) IntelliJ to create and debug Perl files!
Some interesting web sites:
Before you can start using IntelliJ to write and debug Perl scripts, the following has to be done:
This seems obvious, but make sure you have a Perl installation on your Windows machine. I've chosen for Strawberry Perl since this is the most user-friendly Perl installer you can find for Windows.
As of this writing, the Perl version installed by Strawberry Perl is 5.38.0.
Next, make sure you have the community edition of IntelliJ. IntelliJ is by far the most powerful editor in the universe to edit files. It's indirectly used in Android Studio (or should I say Android Studio is built on top of IntelliJ) and I'm using this (Java) IDE for my daily job. So, logically I'm trying to continue to use that powerful editor for other than Java files!
IntelliJ, however, is very expensive. The paid version is using an annually renewable subscription that can cost you hundreds of Euro's a year. For a non-professional usage, that's way too much.
Luckily, they also have community versions that costs nothing. You don't have all the whistles and bells of the paid version but in 99% of the cases you don't even need those... things and bells. So, go ahead and grab your version of the community edition!
As of this writing, the latest version is 2023.2.5.
Next, you have to install a Perl plugin. Therefor, go to File -> Settings -> Plugins and search for 'Perl' in the search box. You should end up finding the Perl plugin written by Alexandr Evstigneev.
Once the Perl plugin is installed, you can create a new project: File -> New -> Project. Make sure you select "Perl" in the left side of the New Project view!
Then click Next and fill in the Perl5 Interpreter: input field. If necessary, click the cog wheel on the right and select Add System Perl... to point to the Perl executable of the Perl installation you've done (remember, Strawberry Perl is by far the most user-friendly installation!).
Once the Perl5 interpreter is know, click Next again.
Then fill in the regular things like project name, project directory and so on in the next dialogue screen. When finished, click the Create button.
If IntelliJ is already running, it will ask you where the new project should be created:
In the current window: your active project will be closed and the new project will be taking in its place
In a new window: a new IntelliJ session will be opened and you can continue to work in your current or new project.
Et voila, your first Perl project is created!
Once IntelliJ is set up with the Perl plugin and you've created your first Perl project, you will see there's not yet a Perl file created.
Next to this, if you expand the External Libraries section in the left part of the project view, you will see 4 lib directories + a MSWin32 multithread thing popping up. Those are the libraries installed and owned by Strawberry Perl when you installed that package (see before).
You will also see the location of the Perl executable and you will notice it's the one taken from the Strawberry Perl installation. Chances are you have more than one perl.exe on your PC so it's important you refer to the one that is installed by Strawberry Perl. That should be the case since you selected the Strawberry Perl executable as Perl5 Interpreter during the setup phase of the project.
To create a new file in the project, select File -> New -> Perl5 File. A new, small dialogue box will open, titled New Perl5 File.
Type in the name of the file in the Name box (no extension needed, see next paragraph).
Next, you have to choose which file you want to open:
A package (Perl module, extension .pm)
A script (regular file, extension .pl)
A test (no idea yet what this would contain)
A POD file (no idea what that means... yet...)
Since we want to create a new "script", select the Script option. Since Script is selected, the extension of the filename will be .pl. If we would have chosen Package then the file extension would have been .pm.
The file will already have some "imports" (to use the Java jargon):
use strict;
use warnings FATAL => 'all';
This file can already be compiled (but won't do anything yet, of course): press the green arrow in the icon bar in IntelliJ and you should see a compilation phase starting in the Run tab at the bottom. As said, nothing will be happening but the exit code of the process must be 0, indicating both the compilation as well as the execution went well
As a first example, let's create a small Perl script to read out the content of given directory.
For this to work we need the following code (obviously, the below example is only one of the many ways to achieve the same):
#!/usr/bin/perl
use 5.010;
use strict;
use warnings FATAL => 'all';
use Path::Tiny;
my $files_dir = path('E:\AppData\<*********>\<**********>\2020');
my $iter = $files_dir ->iterator;
my $extension = ".xlsm";
say("");
say "*" x 80;
say("Print all files with the extension [$extension]:");
say "*" x 80;
say("");
while (my $local_file = $iter->()) {
next if $local_file ->is_dir();
print "$local_file \n";
}
As you can see, there are 2 extra "imports" added compared to the initial file content set up by the framework itself:
use 5.010; => this is needed if you want to use say() instead of print() in Perl. say has the small advantage over print that it's adding a \n implicitly
use Path::Tiny; => this is a module that provides the user with useful file and path manipulations. You can for instance check if an item detected is a file or a directory, to give only one example.
What happens here is that for a given directory all the files are printed. If the directory would also contain other (sub)directories, those would be skipped by the line
next if $local_file ->is_dir();
This is one of the many subroutines of the package Path::Tiny.
There's also an iterator created to get all the files. You could also use the glob("*.*") method to fetch all the files in the directory and return them into an array. Then, you could apply the foreach() mechanism to "run" over the array content.
But as said before, there's more than one way to achieve the same.
This is already a bit more sophisticated and elaborated. We will need another Perl module to read the content of a spreadsheet: Spreadsheet::Read.
The nice(st) thing of the IntelliJ IDE is that it will detect this package is not installed yet and they will propose to install it for you.
If you accept this, IntelliJ will open a CPAN session and start fetching, compiling and installing the Perl module for you. If that's not a magnificent service! No more hassle to get yourself the missing packages: IntelliJ is doing it for you! No wonder people getting lazy... ;-)
Below is the code (again, multiple solutions are possible):
#!/usr/bin/perl
use strict;
use 5.010; # for say
use warnings FATAL => 'all';
use Path::Tiny;
use Spreadsheet::Read;
use constant SHEET_1 => 1;
use constant SHEET_2 => 2;
use constant SHEET_3 => 3;
use constant SHEET_4 => 4;
use constant SHEET_5 => 5;
my $excel_dir = path('\E:\AppData\<*********>\<**********>\2020');
my $regex = join( "|", "xlsm");
my @files = glob($excel_dir . "/" . "*.xlsm");
my $iter = $excel_dir->iterator;
my $extension = ".xlsm";
say("");
say "*" x 80;
say("Print all files in the directory using an iterator:");
say "*" x 80;
say("");
while (my $local_file = $iter->()) {
next if $local_file ->is_dir();
print "$local_file \n";
}
say("");
say "*" x 80;
say("Print all files with the extension [$extension], using a foreach() loop:");
say "*" x 80;
say("");
foreach my $local_file (@files) {
next if ( -d $local_file );
print "$local_file \n";
}
say("");
say "*" x 80;
say("Print content of cell {A3} of sheet one for a given XLSM file:");
say "*" x 80;
say("");
my $workbook = ReadData($excel_dir . '\20200102-7026349161.xlsm');
print $workbook->[SHEET_1]{A3} . "\n";
First, all files in a given directory are printed. Next, all files with a given file extension in that directory are printed. Note that we here make use of the glob() Perl method to do the filtering.
Finally, the content of one cell for a given sheet in an Excel workbook is printed.
Note that we also make use of constants here to avoid "magic numbers" in the code.
To start debugging a Perl script, first put one or more breakpoints in the code. Then, click on the bug symbol in the icon bar of IntelliJ to start the debug session.
The Debug tab will be selected in the bottom area of IntelliJ and the Console section will be shown by default, giving you the output.
The moment a breakpoint is hit, the code stops. The Console section will still be shown but you can select the Debugger section on the left of the Console section to see the content of the variables that are in scope the moment the breakpoint is hit.
Pressing F8 jumps from one line of code to the other, not jumping to methods in between: step over is the action executed by the debugger.
If you call a subroutine from a package (Perl module) and you hit F7 (step inside) you will enter the code of the Perl module itself! An example in the above section is when the is_dir() subroutine is called from the package Path::Tiny. The moment you press F7 on that line the debugger will jump to the external package were you again can see the status of the different variables. Really beautiful!!!
Next challenge for me was to create a very basic graphic application in Perl (on Windows 10) and try to debug it also in IntelliJ.
This is the link to the minimalistic GUI application: https://tkdocs.com/tutorial/firstexample.html
Complex UI? Don't think so...
Difficult to understand? Don't think so...
Not doable? Don't think so...
Useful application? Don't think so...
Can I learn a lot from this example? Sure do!!!
But... Little did I know that it would be such a difficult path to get there. However, I'm a "persevereer" an in the end I was able to achieve my goal!!!
How did it all started:
I created a small test app with only the following lines in it:
#!/usr/bin/perl
use 5.010; # for 'say'
use strict;
use warnings FATAL => 'all';
use Tkx;
say("This is a Tkx test app");
Absolutely no rocket science, agree? (hopefully, you do)
IntelliJ was highlighting Tkx and was telling me Unable to find a package file for Tkx and proposed me to install it using cpanminus (never heard of this before...). See the image at the end of this section.
So, I went for that option. Result?
tclConfig.sh=D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh
tcl_library=D:/WinApps/Programming/Git/mingw64/lib/tcl8.6
tcl_version=8.6
Using config data in D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh
incpath -I/mingw64/include/tcl8.6 from your tclconfig D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh does not provide tcl.h at Makefile.PL line 36.
N/A
! Configure failed for Tcl-1.27. See C:\Users\Geert\.cpanm\work\1701350275.13752\build.log for details.
! Installing the dependencies failed: Module 'Tcl' is not installed
! Bailing out the installation for Tkx-1.10.
Process finished with exit code 1
The red line was puzzling me and I couldn't find a way to solve this. Also when I just used the command line cpan from Strawberry Perl I got similar failure messages:
tclsh=D:/WinApps/Programming/Git/mingw64/bin/tclsh.exe
tclConfig.sh=D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh
tcl_library=D:/WinApps/Programming/Git/mingw64/lib/tcl8.6
tcl_version=8.6
Using config data in D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh
incpath -I/mingw64/include/tcl8.6 from your tclconfig D:/WinApps/Programming/Git/mingw64/lib/tclConfig.sh does not provide tcl.h at Makefile.PL line 36.
No 'Makefile' created VKON/Tcl-1.27.tar.gz
D:\WinApps\Programming\StrawberryPerl\perl\bin\perl.exe Makefile.PL -- NOT OK
Stopping: 'install' failed for 'Tcl'.
Failed during this command:
VKON/Tcl-1.27.tar.gz : writemakefile NO -- No 'Makefile' created
After searching for ages on the internet, I finally found this clue: https://stackoverflow.com/a/54396365/1252050
Next to this, I also realised that I had to install ActiveState Tcl (read it many times in other threads but my penny didn't drop on time...).
So, what I did was this:
Install ActiveState Tcl (can be downloaded for free from this web page) and made sure the ActiveState Tcl path was added during installation of the package (is an option, but it's default checked).
Due to this, the .\ActiveTcl\bin directory will be the first in the PATH environment variable and the script tclsh.exe (or tclsh86t.exe for a 32-bit OS) will be found immediately, when called.
As of this writing, the ActiveState Tcl version is 8.6.13 and it's for MSWin32-x64.
VERY IMPORTANT!!!
- Open the file tclConfig.h in .\ActiveTcl\lib\
- Search for the line starting with TCL_LIB_SPEC
- Change it into:
TCL_LIB_SPEC='-T<path_to_ActiveTcl_installation>\ActiveTcl\lib\tcl86t.lib'
- Search for the line starting with TCL_INCLUDE_SPEC (should be in the neighbourhood)
- Change it into:
TCL_INCLUDE_SPEC='-I<path_to_ActiveTcl_installation>\ActiveTcl\include'
Opened a new DOS command window in the Strawberry Perl bin directory (.\Strawberry\perl\bin) and ran the cpan.bat file inside that directory.
First installed Tcl (could be that it was indirectly installed too when trying to install Tkx directly but I didn't try that option): install Tcl
Then installed Tkx: install Tkx
During this installation some dialogue box will pop up to test the widgets generated.
Finally, all was installed. The install directories can be found in <path_to_ActiveTcl_installation>\StrawberryPerl\cpan\build and the versions of Tcl and Tkx are as of this writing:
* Tcl-1.27
* Tkx-1.10
Going back to IntelliJ I didn't see the use Tkx; highlighted anymore, so it was able to find the Perl module.
By the way: both Tcl.pm as well as Tkx.pm are installed in <path_to_ActiveTcl_installation>\StrawberryPerl\perl\site\lib.
After going through the muddy waters to get Tcl as well as Tkx installed I could finally put my attention to building my first graphical Perl application using Tkx.
As mentioned before, I found a nice try-out example here. The only change I had to make was to declare the variables used in the code first (feet and meters). This is because I'm using the use strict; pragma while it was not used in the example code. So, here's my complete Perl code:
#!/usr/bin/perl
use 5.010; # for 'say'
use strict;
use warnings FATAL => 'all';
use Tkx;
my $feet;
my $meters;
Tkx::wm_title(".", "Feet to Meters");
Tkx::ttk__frame(".c", -padding => "3 3 12 12");
Tkx::grid( ".c", -column => 0, -row => 0, -sticky => "nwes");
Tkx::grid_columnconfigure( ".", 0, -weight => 1);
Tkx::grid_rowconfigure(".", 0, -weight => 1);
Tkx::ttk__entry(".c.feet", -width => 7, -textvariable => \$feet);
Tkx::grid(".c.feet", -column => 2, -row => 1, -sticky => "we");
Tkx::ttk__label(".c.meters", -textvariable => \$meters);
Tkx::grid(".c.meters", -column => 2, -row => 2, -sticky => "we");
Tkx::ttk__button(".c.calc", -text => "Calculate", -command => sub {calculate();});
Tkx::grid(".c.calc", -column => 3, -row => 3, -sticky => "w");
Tkx::grid( Tkx::ttk__label(".c.flbl", -text => "feet"), -column => 3, -row => 1, -sticky => "w");
Tkx::grid( Tkx::ttk__label(".c.islbl", -text => "is equivalent to"), -column => 1, -row => 2, -sticky => "e");
Tkx::grid( Tkx::ttk__label(".c.mlbl", -text => "meters"), -column => 3, -row => 2, -sticky => "w");
foreach (Tkx::SplitList(Tkx::winfo_children(".c"))) {
Tkx::grid_configure($_, -padx => 5, -pady => 5);
}
Tkx::focus(".c.feet");
Tkx::bind(".", "<Return>", sub {calculate();});
sub calculate {
$meters = int(0.3048*$feet*10000.0+.5)/10000.0 || '';
}
Tkx::MainLoop();
Finally! It's working and I can step through the code!!!
Here's the end result:
This site has a very detailed explanation about all methods Tk(x) is using.
Next to Tkx, I wanted to try out another Perl GUI but this one was not using Tkx but Tk instead.
Same story: IntelliJ didn't find the Tk.pm file, so I allowed IntelliJ to install the package.
During compilation I stumbled again upon compilation errors. What else did you expect, huh?
Issue was residing in a file X.h, part of the X11 environment. More precisely in .\Tk-804.036-0\pTk\mTk\xlib\X11\X.h.
I found a solution on this site and applied it to the file X.h.
Running cpan install Tk again was no option since it would install another session of Tk (would be named Tk-804.036-1 on my harddisk). So, I went for the manual installation after I modified X.h according the guidelines given in the link above.
Those are the steps I took:
Go into the directory of the Perl Tk package: in my case this was .\cpan\build\Tk-804.036-0
Run the command perl Makefile.PL
Run the command make (this will take a long time)
Run the command make test (this failed for a reason unknown to me, but I simply ignored this result)
Run the command make install
However, when creating a small test application with Tk, it doesn't work.
So far, I have no idea why, so I left Tk for what it is and I keep on using Tkx instead. It's any how advised on many places to use Tkx instead of Tk (although I've seen opponents of this statement too)
The Linux distro comes with what is called a "system perl" version. That is the version of perl that is used system wide and might be used by packages installed.
However, this does not necessarily is the latest version of perl. For example: when installing Ubuntu 22.04 LTS the installed perl version is 5.34.0 while the latest perl version currently is 5.38.2 (as of this writing: Dec. 29, 2023).
You can't just install/move to the latest version of perl using sudo apt install perl since that one will again take the version that is currently distributed by the Linux distro.
However, there's a possibility to use whatever version of perl you want in a local, isolated environment. This can be done through an application called perlbrew. perlbrew gives you the possibility to show a list of available perl versions that can be used and let you chose one of these to install in your home directory.
To get perlbrew installed, run the command curl -L https://install.perlbrew.pl | bash. After a short while, perlbrew will be installed in a directory called ~/perl5/perlbrew/bin.
You still have to do the following when perlbrew is installed:
add the following to the end of the ~/.profile file: source ~/perl5/perlbrew/etc/bashrc
open a new terminal
perlbrew should now be available
Below is a list of some of the more useful perlbrew commands:
perlbrew info
Shows an overview of the current perl used. Shows if the current system perl is used or a perl version installed through perlbrew.
perlbrew list
Shows a list of locally installed perl versions through perlbrew (will not show the system perl version in this list)
perlbrew available
Shows a list of available perl installations
perlbrew install
Allows you to install a certain perl version. Example: perlbrew install perl-5.34.1
After you ran this command perlbrew will download that specific perl version from CPAN and will start the build process which might take a long time.
To follow the build process, you can open another terminal and run the command tail -f ~/perl5/perlbrew/build.perl-5.38.2.log to see the progress. But... be patient.
perlbrew use
Use a certain version of perl in the current shell only.
perlbrew off
Turn off perlbrew in the current shell.
perlbrew switch
Use a certain version of perl permanently.
perlbrew switch-off
Permanently switch off the perl version activated through perlbrew and revert back to the system perl.
More information: https://perlbrew.pl.