How to debug - A general guide
Debugging Applications
Debugging applications can be a challenging task; it takes programming abilities, understanding of the code flow and specific bug finding abilities, on top of these there can be multiple complications, environment, closed libraries, different browsers and servers.
I write this document in the hope that it will help you improve your debugging abilities, so you can minimize your debugging time and also code better to prevent bugs!
* This tutorial uses many techniques from PHP Tequila framework but can be easily used for other languages / frameworks.
In this document:
Editor selection
The best starting point is the editor selection; use an editor that helps with indentation, function identification, and common syntax highlighting. Line numbers is an absolute must!
For PHP I strongly recommend: Zend / Eclipse, for general purposes Notepad++ (be sure to add/enable the function list plug in).
Enable error reporting in your environment
I often find systems with dozens of errors simply because developers turn off reporting, and as PHP allow Notices and some errors to go by they simply don't care. This can hide big coding problems and it also looks really bad on a developer / company. This MUST never be done, turn on your debugging!
To turn on error reporting:
Permanently (A MUST for all developers and managers):
Open PHP.ini
Set error_reporting = E_ALL;
* After PHP 5 you can use E_STRICT; to get notice of deprecated functions
Runtime
At the beginning of your code add:
error_reporting(E_ALL);
* When you set tequila to debug mode, error reporting is turned on
Disable error suppression
PHP allows @ symbol to suppress errors, this is very useful in some cases but at debugging time must be removed to be able to see the errors.
Debugging
In order to learn debugging we must understand:
Types of bugs
The debugging process
Complex applications (Ajax, RIA, etc.) behavior
Complex applications moments of interest
Tools
Types of bugs
Syntax/parse errors
Fatal errors, warnings and notices
Database errors
Logical errors
Syntax Errors
This is the most common error at development time, luckily most can be avoided using a good editor; the most common causes are:
Missing ; at the end of the line (The error normally displays on the next line)
Missing or extra parentheses
Unclosed braces
Type errors
Incorrect string concatenation
Fatal Errors
Fatal errors normally occur in development time and stop PHP from running. Error messages are normally clear and indicate the precise line that causes the error.
Common causes are:
Extra } in classes
Calls to non existing classes or functions
Warnings
Errors that doesn't stop PHP but will normally cause problems. Some common causes:
Incorrect number of parameters
Not found index in array
Cannot modify header information
Session_start - headers already sent
All warnings must be fixed, normally PHP will display which line of code is making the incorrect function call, this is the place to start checking.
If you need to trace the function calls (stack) to detect where the problem occurs you can use debug_print_backtrace() in PHP; in Tequila you can call showcallstack();
Notices
Notices are lighter errors in PHP, sometimes code appear to work properly if you ignore them, but this might just mean code shows a standard behavior but doesn't always mean the developer intended behavior. Good code must not generate a single Notice, common 'Notice' causes are:
Undeclared variables
Missing quotes
i.e. $array[key];
// if you mean:
$array['key'];
//it will work as PHP believes it's a constant with value 'key'
// but if you meant
$array[$key];
//it might work but logic is completely wrong, an empty key might be returned
Database errors
In this category I will comment about database design, SQL errors and Data validation errors which are very common ones.
It's also important to notice that not all errors are bugs or failures but sometimes incorrect use of data and/or resources.
What makes use of data 'correct'?
Minimal exchange of information
Delimited rows
Delimited columns
Don't retrieve BLOB/TEXT until required
Standard SQL queries
Non specific db-brand queries
Proper indexed and accessed information
Proper keys definition
Reuse of objects and connection objects
In Tequila by using DAO classes and generated pages these errors are somehow minimized as the page provide simple JS validation and the SQL is formed by the DAO class, unfortunately DAO cannot determine when to eliminate BLOB's/TEXT fields from the query, nor workaround poor DB libraries.
As with any code introduction new places for bugs/issues are created, let's examine some common ones.
DAO errors
There are some common misuses of DAO classes:
Using DAO instead of DAO_unique for tables with no autonumeric field (auto increment)
Using $key instead of $keys for DAO_unique
Not setting $nullable array for tables that doesn't allow nulls and use ADODB library
Abuse of data
Some errors are very hard to notice as they don't create errors but just affect performance
Not filtering results
Loading Text and BLOB fields when they are not required, specially on multiple records
Loading a full set of data to display just a section (pagination, see LIMIT clause for more info)
* To avoid these problems in DAO classes you should create a special method in the DAO child that return just the fields you need.
SQL Errors
A common source of failures are incorrect SQL statements; at development time all SQL queries must be tested out of the system first (and then in); some SQL statements might work correctly with a simple test query, but will fail when using real data, some common mistakes are:
Filter criteria is incorrectly joined
One side of a JOIN can include NULL values, You need to use a left/right join (this often becomes evident, when the system uses real data)
Null values in the data
Database design
Database design is an speciality in itself, please consult with your manager or project leader. When creating tables don't forget to use the prefixes T_ for tables and I_ for fields.
Out of Tequila I still recommend this or a similar notation, specially if you consider a database migration can take place in the future.
Issues with Naming, Advanced functions & DB versions
Please be aware that some common functions might not be supported by specific databases, or will fail even if supported,
Here's a list of some of issues we have discovered in different databases:
MySql before 4.1
Lack of support for subselects (Use joins instead)
MySql
Regex support is faulty. We have found mysql crashed several times due to use of REGEX to find tags (others worked well)
MSSQL
SQL PHP library is really deficient, therefore Tequila uses the ADODB COM object (You can monitor status of new library here)
MSSQL using ADODB
Support is still incredibly bad, with lack of paging support and other critical stuff.
Until the release of the new Microsoft library this issues are unsolvable and Tequila/PHP uses workarounds for them.
ORACLE
Long names. Oracle doesn't support very long field names
Oracle use UPPER CASE
CASE sensitivity
A common cause for bugs on system migration is case sensitivity, be aware of the following:
MySQL doesn't respect Case in table names in windows systems
MySQL respect case in *nix systems
PHP is case sensitive (always)
Oracle forces UPPERCASE*
*Tequila has a modified Oracle library that allow mixed case to be used in queries (to allow db migration) however this comes with an overhead.
Implications breakdown
Coming down to code this means that when you change the OS or even the web server your application might completely crash. Possible reasons are:
The Script to create the DB was exported from the window system
The target database uses different case
Example:
We assume system was developed on win and SQL is dumped from this system
// Table is exported from Win system
$mSql = "SELECT UserName from Users WHERE idUser = 3";
...
$username = $rst['UserName'];
// Will succeed in windows
// Will fail in *nix as Users table doesn't exist; Users != users
// Will fail in Oracle as field it's called $rst['USERNAME']
Recommended practices
In your code use always UPPER CASE for table names and fields
Define your database using UPPER CASE for tables
Verify your export database script 'case' is correct before recreating.
Database mobility
Coding for proper database mobility is a challenge in itself. In Tequila/BB we have standardized a set of practices to favor mobility:
Prefer use DAO over SQL to have a single control point
Have single entry points for DB querys (functions execSQL and insertSQL)
Have a single function for data retrieval (getData)
Use only standard SQL
Avoid / Minimize use of stored procedures
Avoid / Minimize use of temp tables
Prefix all tables with T_
Prefix all field names with I_
* These practices are critical, not following them might result in your code being rejected, some critical cases like the use of 'name' field will result in database reserved word conflicts
Logical errors
The most difficult kind of error to debug, it normally doesn't relate with a mistake in the language but with the problem solving.
This errors will normally be noticed because the results of an operation doesn't match the expected ones in all or some of the cases.
To narrow down this errors, we often need to analyze what's happening at execution time, sometimes we are able to line by line debugging if we have a proper set up environment, still many times, we need to debug in online servers, or other environments without an integrated debugger.
In order to debug this errors we normally need to 'see' what's happening inside the application, for this your best friends in PHP are:
echo; print_r; debug_print_backtrace()
This functions however doesn't allow us to debug in a real environment or to inspect specific cases, Tequila implements a set of debugging functions that works conditionally allowing you to implement them in a real environment and have the ability to inspect what's happening.
Tequila equivalent functions.
Tequila offers the possibility of conditionally activate debugging, please check Tequila documentation for a complete set of functions and the ways to activate in different scenarios
Example of a logical case
Let's imagine an example where $user is the publisher of a document, and the document $author field appear empty
The strategy for tackling this bug would be something like:
Try to point out the place where you believe data / execution is going wrong (Controller perhaps?, model?)
Add debugging code, i.e. myecho ("executing suspecting function, Value of user = $user");
Add debug code (as in 2) in several spots until you can trace the place where the value of $user change
Try function / class entry points
Try before and after calling other functions that you believe can affect this value
Remove all the debugging entries that show correct values
Drill down the function where the value was altered
Repeat until you can pin point an specific line or section
The idea behind is to narrow the problem to an specific function or segment of the code.
Notes
Remember:
Try to be logical, common functions used by everyone in the same way are normally not the cause of the problem
Don't blame it because you cannot understand it
SQL
RegEx
Packages
Libraries
Complex strings
Most times libraries and packages are safe to use, it's more common that your own code is faulty and not the team /public code. (This doesn't mean they cannot contain bugs/issues that no one discovered before, but you shouldn't blame this parts from the beginning)
Dates
Dates are a small challenge in application development, verify that your application date data is congruent. Tequila favors the canonical format:
YYYYMMDD HH:MM:SS
As this is the only universally understood format, while you can use many formats to display dates, we recommend this to store and pass date data.
Be aware of locales with different years like Thai, for this cases we recommend to store normal data and just replace the year on view, as the calendar doesn't match the year it mentions but the current one.
Example:
Thai year: 2552, does not match the calendar of standard year 2552, but matches the calendar of 2009.
Other errors
There are other kinds of possible errors that go beyond debugging, you can read Tequila good practices or other similar documents to better understand this, among this errors:
Bad architecture design
Improper code separation
Non independent functions
Bad OO coding
Line by line debugging
The best and easiest way to debug in any language is to use a line by line run time debugger in the style of visual studio, where you can see how the application is running and the values are changing, unfortunately this option is not often available in PHP.
You can try installing the debugger in Zend or one of the many options available. (Check IBM link for more info)
Debugging in Online servers without access to Core files
In some case you will find that it will be useful to get debugging information from Tequila core files in an environment you don't have access to modify, you have 2 options for this scenario:
Files that are inheritable
Most Tequila classes are descendant of other Tequila classes, i.e. Dao, application_controller, application_view. In this case you will normally have access to the child class (part of the application) but not to the parent file (part of core)
Case: Debug person_DAO() method save
Open DAO.php from your local copy of Core
Open person_DAO
Copy all methods you are interested in debugging from (1) to (2)
Add Tequila debugging information
Execute locally to verify your new code doesn't introduce new errors and that debugging information is valuable
Upload to testing/real environment
Execute using by case activation(Check debugging page if you need help with this)
You will now have full view of what's happening without needing to have access to DAO or any other Core class
Files that are not inheritable
Other times you will need to have access to framework files, here the recommendation splits again:
Systems under development
Long time running systems
For systems under development, I recommend to add and keep a core with basic debugging functions (production copy has all debugging info commented). Only once the system is released the debugging information is removed
For long time running systems, if there's really no option, the debugging file should be prepared and replaced on server so per instance conditional debugging can be run.
Another last option for servers with many systems will be to keep on server 2 copies of the framework, one with debugging instructions and a production one, it's very easy to update the configuration of a single system to use one or the other.
Debugging advanced applications
Debugging have become increasingly complex in web applications as there are too many elements taking part.
Common components
Let's start by inspecting the common components:
Database server
Web server
PHP
Framework
Application
Browser
JS
CSS
XML
The real problem with Rich Clients is that they hide information from us, the data comes in one of many formats "behind the curtains" making access to data very complicated.
It's important to notice that to debug these applications many times you will need good understanding of HTML, CSS, JS, PHP, XML and AJAX. If you don't feel confident about your abilities on some, request help from your peers or your project manager after you have pinpointed the problem.
Tools to debug applications
Firefox
Firebug
Web developer (optional)
Common errors in rich applications
Let's review some common errors in rich applications:
Missing files / resources
Missing updates
CSS Class conflict
Data related problems
Incorrect data
Server errors make the data dirty
UTF with BOM files
Cache results don't let you see the results
Incorrect JS
1. Missing files / resources
One of the most common issues we find in applications is that developers forget to download or deploy some files, CSS files, support images, etc..
Images
This mistakes are subtle and sometimes hard to notice as some browsers simply don't show anything, unfortunately other browser will show a massive X in place of the missing image.
CSS
You will normally notice you are missing a CSS file because the screen looks weird (layout), or the font colors, backgrounds have changed
JS
Javascript errors are caused many times by not loaded JS files.
Debugging
You can easily find which files are missing using firebug,
Open firebug
Go to Net tab
Select All, or the specific type you suspect
All missing files are written in red
You can mouse over them to see the complete path of the file and then verify if the resource exist on server or not
You can click the + symbol, to check the requesting Headers, the Response header and the HTML
Paths are correct, resources still missing
In some environments, resources will keep failing to download, this is specially true when you load a lot of files. Tequila offers some strategies to prevent this problem. To enable them: file minimization, joining and monitored downloading, you simply:
Open includes/config.php (server side)
Find the js_includes configuration section (package it's included in core)
Modify these lines to enable:
$js_safeloading = true;
$js_blocktillcomplete = true;
$js_retry = 3;
$js_cachedir = 'temp';
This will minimize, join all JS files, cache results for faster execution, monitor downloading, retry up to $js_retry times and optionally ($js_blocktillcomplete) Block the screen use until the files are loaded successfully.
* Notice. Minimization require all your JS files to be very well done (at least all closing brackets and ending semicolons). Tequila uses a standard minimization library and there are many tools online to check your library is valid, try this very strict validator JSLINT
2. Missing updates
Another common reason of involuntary bug induction is incorrect application updating, as files and resources are split across the application it's quite common that partial updates are made. i.e. Template files are not updated, or JS or CSS or...
Solution
Use a file comparison tool, or use mercurial to upgrade your application.
3. CSS Class conflict
CSS is awesome, it allow many levels of definition that allow precise change of look and layout, unfortunately some times CSS definitions can crash. i.e. A YUI component / JS component define the look of <div> and so your application does. The final result is a mix of both definitions.
Debugging
Find out the source of the conflict
Right click on the conflicting element
Inspect element
Verify on the left panel you are on the right HTML tag
Check the list of styles on the right panel, to see which one is defining the tag
Each entry also mentions the file and line number
Solution
Once you have identified the conflicting resources you can:
Decide to remove one of them (this removal can be specific to one page, i.e. a complex one-screen)
Modify one of the CSS to be more specific
You can use CSS syntaxis to affect just the children of one node, i.e. #mainlayer div
You can modify the HTML to apply an specific class to the affected element
You can add a class to parent to differentiate, (i.e. HTML: <div class="special_case"> CSS: .special_case div )
4. Data related problems
Modern applications transmit a lot of data using ajax, this data is hard to debug as it's normally not visible, so the 1st step is to make data visible.
Viewing exchanged data
Open firebug using the icon on the right bottom corner
Click on console
You will see a list of all AJAX calls
Open them by clicking on the + icon
Params tab: You will be able to see the information sent by JS
Headers tab: You can see the headers sent by server
Response tab: You can view here the reply from server
Common problems
Based on this data you can start moving towards a solution, some common problems you can find are
Server errors make the data dirty
Incorrect data
UTF with BOM files
1. Server errors
This is the easiest error to debug, simply inspect the "response tab" like a normal web page and work over the PHP in server to clean up all the errors.
* If you are sending all parameters using GET (querystring). You can also right click on the Console entry and select Open in New Tab
This will allow you to inspect data in detail, you can switch server modes from XML to cd, to get rid of the headers and be able to read the complete errors. (Simply replace rrt=xml -> rrt=nh or cd; read RRT section on Tequila manuals for more info on Requested Response Types)
2. Incorrect data
Incorrect data might point to a logic problem in the server code, use your normal debugging techniques to solve this issue.
Another possible problem is that server and client versions are out of sync. Verify which data is correct before starting modifications. Then work on the server or JS files as necessary.
3. UTF with BOM files
Sometimes you will discover everything is correct, data is perfect, JS is in place and still you keep getting errors from your ajax callback function (this happens specially with XML data).
There's a subtle bug with files saved as UTF with BOM, the problem is that when they are included, PHP immediately start the output making impossible to set the correct XML headers.
Solution
Open every file included in the task with Notepad++ or another editor that support this setting and re save them WITHOUT BOM (optionally you can enable output cache but this is only recommended for testing, cache code is commented in index.php file)
5. Cache results don't let you see the results
This is a common issue, sometimes you own browser cache keeps showing images, css or data from previous calls. You can clear the cache or install the "web developer" plug in mentioned above, simply:
Click on Disable button > Disable Cache.
This will allow you to have a clean environment all the time.
Server cache
We have found on some online servers that PHP pages are cached and the changes made on the PHP are not reflected, try adding some extra info, an extra echo or some visual element that let you know your code is being executed. You might even be connected to the wrong server :)
If required, duplicate the file you are testing, and use this copy for debugging / developing. Update the real file after results are satisfactory, normally the server will start using the updated version after some hours.
6. Incorrect JS
Once you have determined that the server is sending the right responses and all resources are loaded, it's time to move debugging to the client side, You can work on this debugging in many ways.
Echo, echo echo.
Using the exact same technique as in PHP, you can use alert() at some places to inspect execution values.
debug.js
Tequila includes a very basic library to inspect values at running time, you must include it at server side to be able to use it on client, a couple of useful functions are:
function showobj(myObj, title, complete)
function showScript (content, safehtml, title)
Firebug!
And your best friend again: line by line debugging. Before you get crazy following each line, you should inspect your code in an external editor and determine the function you think might be the problem, i.e. the callback function, in ajax
To start debugging in firebug
Open firebug (icon on bottom right side of the screen)
Click on script tab
Under script you will find 2 selectors, the 2d one indicates the name of the library being displayed
Open this selector and select the library you want to debug
Move the left pane to the line of code you want to inspect
Add a breakpoint by clicking on the left hand side of the numbers, a red spot will appear
Keep the window open
Trigger in the normal window the action you want to inspect
Execution will stop in the breakpoint you added, you can then
Control execution advance using the icons on top of the code, right side
Watch the value of all variables including objects on the right pane
You can also add custom watches on the right pane
Useful links on debugging:
Good luck and happy debugging!
Itzco