Some useful research tools
In course of the research, one encounters several similar-sounding problems which appear to be simple enough in nature that their solutions need not be developed from ab initio but complicated enough that there might be more than a bunch of good, bad, and ugly solutions to them. Such issues could include saving figures in programmable-cum-editable-vector form (to be more ambitious, one could throw in OS dependencies in the mix too), or creating keyboard shortcuts for quick Google scholar search. Every researcher ends up employing their own strategy (pure or mixed) to a slew of such problems. In this section, I will provide some of these tools and t'wicks with the hope of avoiding reinventing the wheel. The relevant topic for each pointer is marked in bold,
Programming.
Speed up:
Vectorize your program.
Here is a minimal working example (in MATLAB) with a neat plot showcasing an improvement in run-time of two orders of magnitude. Pleading innocent for the suspected plagiarism in the file name.
Mathworks documentation may be a good place to start. Loren has also written about this and more ideas to speed up the program in great depth with good examples.
If a memory- or time-intensive piece of the program is possibly repeated in the program, consider memoization. It is a simple technique of storing processed input and outputs (say, as two matrices, one each for inputs and outputs) and looking through the list before a new input is processed.
Visual studio code (VS code)
VS Code has several useful extensions. "Diff_Folders" shows all differences between the two locations. "TODO highlight" adds characters for focus # TODO:, # FIXME:, etc. "Make hidden" adds context menu to hide certain files types, e.g., .bak files.
Define maximize window shortcut by searching for open keyboard shortcuts and editing keybindings.json as follows (I used Ctrl + Shift + F):
{
"key": "ctrl+shift+f",
"command": "workbench.action.toggleEditorWidths"
}
Connect GitHub with R. Elaborate instructions for existing/new projects.
Here is an electric fast crash course: With 1. git installed, 2. Github profile setup, 3. Rstudio installed, 4. Two-factor authenticator app for github setup, 5. Save github PAT (personal access token) using https://happygitwithr.com/https-pat, 6. Connect Rstudio with github using Existing project, GitHub first
Important note: do NOT sync GitHub repo on Google Drive. Consider "My solution for using git with any cloud-synced folders (one drive, google drive, dropbox, etc) is to put the local repo elsewhere, but put a bare clone on the cloud folder and link it as a remote repo, then do your push/pull against the bare clone. Anyone who has access to the cloud folder can do their push or pull (depending on the permissions) against it as well. This is more aligned with how git is supposed to operate anyway. "
Efficient use of a computer (shell/scripts/shortcuts/hotKeys/symbols/smart typing/composite repeatable actions).
FancyZones (learn and install) by Microsoft PowerToys is a great tool for custom window-tiling across different desktops (And please configure multiple desktops on Windows).
Spend some time quickly learning AutoHotKey. If you ever do a series of repetitive tasks (e.g., typing super-/sub-scripts, copy-paste-google/scholar), you will become an instant fan. Must use those shortcuts for them to grow on you. To get you started, my script file as of today (Oct 26, 2023) is here. The script is heavily commented. Two HOT examples follow:
To generate bibTeX file from doi from say a pdf or a website, I wrote a script that
assigns the shortcut Ctrl + Shift + B to copy the entire bib entry on the clipboard from simply selecting the doi and pressing the shortcut,
uses Crossref API, and
cleans the bib entry to remove '%2F' or weird entries to journal's names, e.g., a journal name with ampersand in it.
⁰ through ⁹ can be typed on text editors/emails, etc., with '\sup0' through '\sup9' without the quotes. The space bar after the keyword will convert the shortcut.
Trouble finding symbols in Excel? Watch this video. The same definitions work in MS Word too.
Trouble finding symbols in LaTeX? Try detoxify. Fond of Unicode characters with their codes. Try shapecatcher.
Remember times when you needed to download dozens of files from a website, and you ended up right-clicking and saving each one of them individually? You don't have to do that! Check this simple Matlab script.
Multi-line editing is often a necessity in long programs or type editing. Often the use of the Alt key in combination with Shift, Ctrl, arrow keys, or mouse does the trick. Some examples from different programming or type-editing applications follow.
Notepad++ Shift + Alt + Down-arrow key (and then Right-arrow, if required). Perhaps the most versatile vertical or multi-line editor.
Windows cmd/MATLAB editor (2021b onward). Hold down Alt and drag over multiple lines with the mouse.
Overleaf. Quite supportive of multi-line editing, Hold down Ctrl and click with the mouse on different lines. Non-contiguous editing is supported.
Microsoft Word does not allow editing. But copying and formatting in a rectangle are permitted using Shift+ Alt + mouse/arrow keys.
Notepad++
Smart Highlighting: highlight all occurrences of a selected word with its location in the scroll bar. Install the plugin jN Notepad++ plugin.
Adding surrounding characters ($, #, quotes, brackets, asterisks, dashes) around the selected text is a useful feature that is present by default in a bunch of programs, e.g., TeX Studio, Overleaf. Add the plugin SmartSelection to get this feature.
Compare two scripts/functions graphically by installing the ComparePlus plugin. Note that this is an improvement over the Compare plugin.
On Windows 10 onward, activating clipboard history (just press Windows + v) may be the best favor you will do to yourself today.
In powerpoint, F4 does not seem to work, which is a pain. So, if you want to repeat some actions, say change the font of a selected box and assign a shortcut to it, here are two steps to that.
Write a sub as follows:
Sub changeFont_psb()
Dim oShape As Shape
Set oShape = ActiveWindow.Selection.ShapeRange(1)
With oShape.TextFrame.TextRange.Font
.Name = "Times New Roman"
End With
End Sub
Assign a shortcut to the sub by adding macro to the quick access toolbar (the thing at the top of the top of powerpoint window).
Home -> Options -> Quick Access Toolbar -> Your Macro Name -> Add -> Move to the top -> Alt + 1
Here is a wiki with collections on shortcuts for different programs. https://shortcutworld.com/Word/win/Microsoft-Word_Shortcuts
Windows powershell, WinSCP, and PuTTY are easy-to-use tools for ssh'ing. If Ctrl+V does not work, you may need to activate it from settings, and then Ctrl+Shift+V works for pasting from the clipboard.
Spreadsheet. Excel.
If you deal with large spreadsheets with multiple sheets. The New Window feature can be helpful. View -> New Window. It lets you open multiple copies of the same spreadsheet. Different sheets can be seen at the same time (more helpful when working on multiple screens). Note that if the file is saved while open in multiple windows (names as fileName.xlsx:2, etc.), it will also open as such. To overcome this, close other instances before saving.
Note: For individual sheets with a large number of entries, splitting the view is perhaps more suitable than the new window. View -> Split. Select a row/column to split before along rows/columns or a cell to split along both. Double-click on the split demarcations to remove them.
Generating modules with input and output arguments (possibly, more than one) in Excel (without VBA). Using Excel for operations that generally require a Function.
Data Table in Excel is a powerful tool to carry out a set of repeated calculations that give an output by changing one or more cells without actually feeding the changing variable manually. Here is a self-explanatory spreadsheet for implementing a TABLE in Excel.
The problem appears to be unmanageable when there is more than one input variable. Here is a spreadsheet (not yet uploaded) with an elegant solution to this. The solution uses suppressing several argument inputs to one variable and using it as a single input argument.
AutoFit Merged Cells: "Avoid merged cells," is a pearl of oft-heard Excel wisdom. They are messy to deal with while sorting, extending formulas, referring to cells, etc. But there are times when you just want them there, especially in the title sheets/cover pages, etc. Unsurprisingly, you cannot use the AutoFit feature for rows or columns that contain merged cells in Excel. Here is a way around by amazing TheSmallman. This solution requires changing the target cell every time you want to implement an autofit for a merged cell. I tweaked their solution (a) to avoid having to change the VBA code every time and (b) to assign a keyboard shortcut so that it always works in the background seamlessly. To get this working, (a) download the edited code from here, (b) copy the code and paste it into a module of an add-in file (read this for the starters on user-defined add-ins), make sure the add-in file (.xla/.xlam) is selected under File -> Option -> Add-ins -> Manage -> Excel add-ins, (c) close Excel and start it again. Select the merged-wrapped cell that needs autoFitting, and press Ctrl+Shift+M. Voilà!
Adding figures as tool-tip on cells: Insert comment -> (rClick on its boundary) Format comment -> Colors and Lines -> Color dropdown -> Fill Effects -> Select Picture. Did I forget to emphasize the boundary of the comment? If you click in the center, you'd only get text-formatting options.
Report templates. Unofficial templates for annual progress seminar reports and dissertations. I either created these from scratch or by putting bits and pieces together from here and there. Though I have paid attention, there is no assurance of meeting all guidelines.
Report preparation. Mostly LaTeX.
Trouble finding symbols in LaTeX? Try detexify.
Most of the citation styles are available in the public domain on BibTex or natbib package (under LaTeX) or on Zotero (under Word), but on the off chance that some style is too weird to be exactly the same as one of the standard citation styles (I have figured this to be often the case with conferences), a new citation style can be created as follows:
The website http://editor.citationstyles.org/visualEditor/ is an outstanding easy way of creating new styles. Read on for the step-by-step process.
Use Search By example to find the closest match to the required bibliography style from http://editor.citationstyles.org/searchByExample/
Once the search is complete, either click on "Install" when there is a 100% match or click on Edit. This opens a "Visual Editor"
Change both fields---"Title" and "ID"---under Info tab. If these are not changed, the new style will not work.
Make changes as required in the graphical mode.
Once the changes are complete, save the Style. and Download it.
In Zotero Preferences, Under Cite, add this style by using the + button.
You can choose to keep the created citation style in the public domain with an appropriate name on the same website.
Natbib is an amazing bibliographical management package. It has powerful features to change citation styles on the fly. Read its documentation here. Here, I will show an example for creating your own bibliographical style and integrating it with Overleaf.
Objective: changing plainnat to show last name first.
Download plainnat.bst file from CTAN.
Rename the bst file to say, plainnat-psb.bst. You shouldn't edit it without creating a new file. Edit the function format.names from
{ s nameptr "{ff~}{vv~}{ll}{, jj}" format.name$ 't := % first name first
to
{ s nameptr "{vv~}{ll}{, f.}{, jj}" format.name$ 't := % last name first
Include the reference file bibliography.bib with the new bst style
\bibliographystyle{plainnat-psb}
\bibliography{bibliography.bib}
Tired of multi-clicking before Google Scholar lets you have your way with the BibTex. Try this script that creates a link for each search. You would need something like TamperMonkey and install this javascript. Mix that with the clipboard history (windows + v), and your bibliography life can be much nicer.
Better BibTex is an alternative to harmonize Zotero with LaTex citations.
Including .svg files can be a pain. The following steps are tried and tested (source).
\usepackage{svg} in the preamble.
\includesvg[width = 0.9\textwidth]{figNameWithoutExtension} in the document.
Install Inkscape.
Make sure that Inkscape's .exe is in the path under env variables.
Build LaTeX using --shell-escape option, which can be done for individual files by adding the following at the top of a document and clicking allow for this document for the first run.
% !TeX TXS-program:compile = txs:///pdflatex/[--shell-escape]
Show “only number” in the caption cross-reference in the MS Word document.
This may be useful in a variety of situations. I needed it for a journal that had Figure captions starting with "FIGURE n". The in-text references should be "Figure n".
(1/2) The easier solution is to edit the field code of the cross-referencing of the captions directly. Solution based on this. The steps follow:
Insert the reference as usual.
Right-click on reference -> Toggle field codes
Change the following:
{ REF _Ref218925266 \h }
to
{ REF _Ref218925266 \# 0 \h }
Toggle field. Update field (Select All and F9 refreshes all field codes).
This solution fails with some pdf converters and rendered the pdf with "Figure FIGURE n".
(2/2) The robust solution is to create a sequence and use bookmarks for cross-referencing. Solution based on this. The steps are as follows:
Create a sequence of numbers.
Ctrl + F9 to insert a field.
Between curly braces, { SEQ FigNum \r 1 \* MERGEFORMAT }, for the first appearance of figure numbers and { SEQ FigNum \n \* MERGEFORMAT } for the subsequent figures. The switch r is for reset to 1, n is for next number.
Use different sequence style for tables, e.g., { SEQ TabNum \r 1 \* MERGEFORMAT }
Select and F9 to see the rendered sequence.
Assign bookmarks to the field.
Select the field.
Click on Insert -> Add bookmarks
Assign an easily recognizable name, e.g., F1, F2, T1, T2.
Nope, there is no way to automate this. Word (Unline, LaTeX) treats sequence objects and bookmarks differently.
Cross-refer to these bookmarks in the regular fashion by selecting Bookmarks in the dropdown for cross-reference.
Generate LaTeX code for tables from spreadsheet. Download the add-in Excel2LaTeX from CTAN. Extract and save the add-in file in an easily accessible location.
Open Excel -> Options -> Add-ins -> Manage Excel Add-ins -> Go -> Browse -> Navigate and select the add-in.
Check the Excel2LaTeX add-in box.
Select all cells of the table for which LaTeX code is required. Click on Add-ins -> Convert Table to LaTeX -> Copy to Clipboard.
Play with options for booktabs package and converting $, \, etc.
(Re)presentations. Tips on figures, flowcharts, diagrams:
(MATLAB) Save figures in multiple formats in a single stroke. Change the list of extensions to save more or less files.
exportName = sprintf('myFig');
extensions = {'fig', 'epsc', 'png', 'jpeg', 'emf'};
for k = 1:length(extensions)
saveas(gcf, exportName, extensions{k})
end
A nice little website for flowcharts and line diagrams is draw.io They also have a desktop application. Some advantages are: (a) Capacity to export as pdf, SVG, and XML files, (b) save the cropped pdf so you don't have to do so in LaTeX, (c) seamless synchronization with different cloud services, (d) useful shortcuts (e.g., for subscript Ctrl + , particularly useful when playing with the fonts and subscript needs to be un-unchecked), (e) add arbitrary polyshapes (turn Basic Shapes on by clicking on + More Shapes in the left pane -> add a Polyline -> right-click, Edit Style -> change normalized coordinates in the relevant HTML code. Note that node indices go counterclockwise and the vertical axis coordinates are first), (f) add images to the diagram by clicking on Insert (+ sign on the top horizontal toolbar) -> Image, (g) select objects intersecting a selection window by pressing Alt while drawing the window, (h) options to add new fonts, use Google Fonts if you like. If your eyes, like mine, don't feel comfortable without Knuth's modern computer font, add it by clicking on Fonts -> Custom -> Web Fonts and copying the address https://cdn.rawgit.com/dreampulse/computer-modern-web-font/master/fonts.css
Frustrated with finding icons/symbols/cartoons/images for representations in your ppt. Check out the noun project. Most images are downloadable (even if in a low resolution) after a free sign-up. Another (better but less community-driven) alternative is iconfinder.
Color blind check: To check the suitability of figures for color-blindness check, download Color Oracle. It is a simple and effective tool (standalone exe, no installation required).
Combine different pdf. Adobe pro can be effective but costly and heavy. Light open-source solutions exist.
Use LaTeX to combine pdf's as follows (source):
\documentclass{article}% or something else
\usepackage{pdfpages}
\begin{document}
\includepdf[pages=...]{paper1}
\includepdf[pages=...]{paper2}
\end{document}
Open-source pdftk offers a universal command-line tool and a gui-based windows app. pdftk is extremely light, easy to use, and has a bunch of options to play with pdf's. An easy command line example is to burst a pdf into individual pages.
pdftk ExampleFile.pdf burst output Individual_File_P%d.pdf
Collaboration. Tools used for research collaboration: GitHub, GitHub Desktop, Overleaf.
Writing: In order to have explosive productivity in writing, you need to find your drug. Writing on site. Having an accountability partner(s). If there doesn't exist one, make it. One cool website for synchronized timer is https://cuckoo.team
Miscellaneous open-source software
Python code visualization: https://pythontutor.com/visualize.html#mode=edit
Mind map: FreeMind
Scam/fake/spam/fraud website diagnosis:
Look up on ICANN.
Get the address and ip from who.is/ This should give enough red flags already. Further, take the IP and go to arin.net/whois to find out where they actually have their website parked.
GeoGebra: A free online tool that lets you draw rectangles and circles with given dimensions. A bit like CAD but not a full blown tool.
LAN messenger/file sharing: on github. Also downloadable from SourceForge.