How to debug - A general guide

Debugging Applications

Debugging applications can be a challenging task; it takes programming abilities, understanding of the code flow and specific bug finding abilities, on top of these there can be multiple complications, environment, closed libraries, different browsers and servers.

I write this document in the hope that it will help you improve your debugging abilities, so you can minimize your debugging time and also code better to prevent bugs!

* This tutorial uses many techniques from PHP Tequila framework but can be easily used for other languages / frameworks.

In this document:

Editor selection

The best starting point is the editor selection; use an editor that helps with indentation, function identification, and common syntax highlighting. Line numbers is an absolute must!

For PHP I strongly recommend: Zend / Eclipse, for general purposes Notepad++ (be sure to add/enable the function list plug in).

Enable error reporting in your environment

I often find systems with dozens of errors simply because developers turn off reporting, and as PHP allow Notices and some errors to go by they simply don't care. This can hide big coding problems and it also looks really bad on a developer / company. This MUST never be done, turn on your debugging!

To turn on error reporting:

Permanently (A MUST for all developers and managers):

• 1. Open PHP.ini

  2. Set error_reporting = E_ALL;

* After PHP 5 you can use E_STRICT; to get notice of deprecated functions

Runtime

At the beginning of your code add:

error_reporting(E_ALL);

* When you set tequila to debug mode, error reporting is turned on

Disable error suppression

PHP allows @ symbol to suppress errors, this is very useful in some cases but at debugging time must be removed to be able to see the errors.

Debugging

In order to learn debugging we must understand:

• • Types of bugs

  • The debugging process

  • Complex applications (Ajax, RIA, etc.) behavior

  • Complex applications moments of interest

  • Tools

Types of bugs

• • Syntax/parse errors

  • Fatal errors, warnings and notices

  • Database errors

  • Logical errors

Syntax Errors

This is the most common error at development time, luckily most can be avoided using a good editor; the most common causes are:

• • Missing ; at the end of the line (The error normally displays on the next line)

  • Missing or extra parentheses

  • Unclosed braces

  • Type errors

  • Incorrect string concatenation

Fatal Errors

Fatal errors normally occur in development time and stop PHP from running. Error messages are normally clear and indicate the precise line that causes the error.

Common causes are:

• • Extra } in classes

  • Calls to non existing classes or functions

Warnings

Errors that doesn't stop PHP but will normally cause problems. Some common causes:

• • Incorrect number of parameters

  • Not found index in array

  • Cannot modify header information

  • Session_start - headers already sent

All warnings must be fixed, normally PHP will display which line of code is making the incorrect function call, this is the place to start checking.

If you need to trace the function calls (stack) to detect where the problem occurs you can use debug_print_backtrace() in PHP; in Tequila you can call showcallstack();

Notices

Notices are lighter errors in PHP, sometimes code appear to work properly if you ignore them, but this might just mean code shows a standard behavior but doesn't always mean the developer intended behavior. Good code must not generate a single Notice, common 'Notice' causes are:

• • Undeclared variables

  • Missing quotes

i.e. $array[key];

// if you mean:

$array['key'];

//it will work as PHP believes it's a constant with value 'key'

// but if you meant

$array[$key];

//it might work but logic is completely wrong, an empty key might be returned

Database errors

In this category I will comment about database design, SQL errors and Data validation errors which are very common ones.

It's also important to notice that not all errors are bugs or failures but sometimes incorrect use of data and/or resources.

What makes use of data 'correct'?

• • Minimal exchange of information

    • Delimited rows

    • Delimited columns

    • Don't retrieve BLOB/TEXT until required

  • Standard SQL queries

  • Non specific db-brand queries

  • Proper indexed and accessed information

  • Proper keys definition

  • Reuse of objects and connection objects

In Tequila by using DAO classes and generated pages these errors are somehow minimized as the page provide simple JS validation and the SQL is formed by the DAO class, unfortunately DAO cannot determine when to eliminate BLOB's/TEXT fields from the query, nor workaround poor DB libraries.

As with any code introduction new places for bugs/issues are created, let's examine some common ones.

DAO errors

There are some common misuses of DAO classes:

• • Using DAO instead of DAO_unique for tables with no autonumeric field (auto increment)

  • Using $key instead of $keys for DAO_unique

  • Not setting $nullable array for tables that doesn't allow nulls and use ADODB library

Abuse of data

Some errors are very hard to notice as they don't create errors but just affect performance

• • Not filtering results

  • Loading Text and BLOB fields when they are not required, specially on multiple records

  • Loading a full set of data to display just a section (pagination, see LIMIT clause for more info)

* To avoid these problems in DAO classes you should create a special method in the DAO child that return just the fields you need.

SQL Errors

A common source of failures are incorrect SQL statements; at development time all SQL queries must be tested out of the system first (and then in); some SQL statements might work correctly with a simple test query, but will fail when using real data, some common mistakes are:

• • Filter criteria is incorrectly joined

  • One side of a JOIN can include NULL values, You need to use a left/right join (this often becomes evident, when the system uses real data)

  • Null values in the data

Database design

Database design is an speciality in itself, please consult with your manager or project leader. When creating tables don't forget to use the prefixes T_ for tables and I_ for fields.

Out of Tequila I still recommend this or a similar notation, specially if you consider a database migration can take place in the future.

Issues with Naming, Advanced functions & DB versions

Please be aware that some common functions might not be supported by specific databases, or will fail even if supported,

Here's a list of some of issues we have discovered in different databases:

• • MySql before 4.1

    • Lack of support for subselects (Use joins instead)

  • MySql

    • Regex support is faulty. We have found mysql crashed several times due to use of REGEX to find tags (others worked well)

  • MSSQL

    • SQL PHP library is really deficient, therefore Tequila uses the ADODB COM object (You can monitor status of new library here)

  • MSSQL using ADODB

    • Support is still incredibly bad, with lack of paging support and other critical stuff.

    • Until the release of the new Microsoft library this issues are unsolvable and Tequila/PHP uses workarounds for them.

  • ORACLE

    • Long names. Oracle doesn't support very long field names

    • Oracle use UPPER CASE

CASE sensitivity

A common cause for bugs on system migration is case sensitivity, be aware of the following:

• MySQL doesn't respect Case in table names in windows systems

• MySQL respect case in *nix systems

• PHP is case sensitive (always)

• Oracle forces UPPERCASE*

*Tequila has a modified Oracle library that allow mixed case to be used in queries (to allow db migration) however this comes with an overhead.

Implications breakdown

Coming down to code this means that when you change the OS or even the web server your application might completely crash. Possible reasons are:

• The Script to create the DB was exported from the window system

• The target database uses different case

Example:

We assume system was developed on win and SQL is dumped from this system

// Table is exported from Win system

$mSql = "SELECT UserName from Users WHERE idUser = 3";

...

$username = $rst['UserName'];

// Will succeed in windows

// Will fail in *nix as Users table doesn't exist; Users != users

// Will fail in Oracle as field it's called $rst['USERNAME']

Recommended practices

• In your code use always UPPER CASE for table names and fields

• Define your database using UPPER CASE for tables

• Verify your export database script 'case' is correct before recreating.

Database mobility

Coding for proper database mobility is a challenge in itself. In Tequila/BB we have standardized a set of practices to favor mobility:

• • Prefer use DAO over SQL to have a single control point

  • Have single entry points for DB querys (functions execSQL and insertSQL)

  • Have a single function for data retrieval (getData)

  • Use only standard SQL

  • Avoid / Minimize use of stored procedures

  • Avoid / Minimize use of temp tables

  • Prefix all tables with T_

  • Prefix all field names with I_

* These practices are critical, not following them might result in your code being rejected, some critical cases like the use of 'name' field will result in database reserved word conflicts

Logical errors

The most difficult kind of error to debug, it normally doesn't relate with a mistake in the language but with the problem solving.

This errors will normally be noticed because the results of an operation doesn't match the expected ones in all or some of the cases.

To narrow down this errors, we often need to analyze what's happening at execution time, sometimes we are able to line by line debugging if we have a proper set up environment, still many times, we need to debug in online servers, or other environments without an integrated debugger.

In order to debug this errors we normally need to 'see' what's happening inside the application, for this your best friends in PHP are:

echo; print_r; debug_print_backtrace()

This functions however doesn't allow us to debug in a real environment or to inspect specific cases, Tequila implements a set of debugging functions that works conditionally allowing you to implement them in a real environment and have the ability to inspect what's happening.

Tequila equivalent functions.

Tequila offers the possibility of conditionally activate debugging, please check Tequila documentation for a complete set of functions and the ways to activate in different scenarios

Example of a logical case

Let's imagine an example where $user is the publisher of a document, and the document $author field appear empty

The strategy for tackling this bug would be something like:

• 1. Try to point out the place where you believe data / execution is going wrong (Controller perhaps?, model?)

  2. Add debugging code, i.e. myecho ("executing suspecting function, Value of user = $user");

  3. Add debug code (as in 2) in several spots until you can trace the place where the value of $user change

     1. Try function / class entry points

     2. Try before and after calling other functions that you believe can affect this value

  4. Remove all the debugging entries that show correct values

  5. Drill down the function where the value was altered

  6. Repeat until you can pin point an specific line or section

The idea behind is to narrow the problem to an specific function or segment of the code.

Notes

Remember:

• • Try to be logical, common functions used by everyone in the same way are normally not the cause of the problem

  • Don't blame it because you cannot understand it

    • SQL

    • RegEx

    • Packages

    • Libraries

    • Complex strings

  • Most times libraries and packages are safe to use, it's more common that your own code is faulty and not the team /public code. (This doesn't mean they cannot contain bugs/issues that no one discovered before, but you shouldn't blame this parts from the beginning)

Dates

Dates are a small challenge in application development, verify that your application date data is congruent. Tequila favors the canonical format:

YYYYMMDD HH:MM:SS

As this is the only universally understood format, while you can use many formats to display dates, we recommend this to store and pass date data.

Be aware of locales with different years like Thai, for this cases we recommend to store normal data and just replace the year on view, as the calendar doesn't match the year it mentions but the current one.

Example:

Thai year: 2552, does not match the calendar of standard year 2552, but matches the calendar of 2009.

Other errors

There are other kinds of possible errors that go beyond debugging, you can read Tequila good practices or other similar documents to better understand this, among this errors:

• • Bad architecture design

  • Improper code separation

  • Non independent functions

  • Bad OO coding

Line by line debugging

The best and easiest way to debug in any language is to use a line by line run time debugger in the style of visual studio, where you can see how the application is running and the values are changing, unfortunately this option is not often available in PHP.

You can try installing the debugger in Zend or one of the many options available. (Check IBM link for more info)

Debugging in Online servers without access to Core files

In some case you will find that it will be useful to get debugging information from Tequila core files in an environment you don't have access to modify, you have 2 options for this scenario:

Files that are inheritable

Most Tequila classes are descendant of other Tequila classes, i.e. Dao, application_controller, application_view. In this case you will normally have access to the child class (part of the application) but not to the parent file (part of core)

Case: Debug person_DAO() method save

• 1. Open DAO.php from your local copy of Core

  2. Open person_DAO

  3. Copy all methods you are interested in debugging from (1) to (2)

  4. Add Tequila debugging information

  5. Execute locally to verify your new code doesn't introduce new errors and that debugging information is valuable

  6. Upload to testing/real environment

  7. Execute using by case activation(Check debugging page if you need help with this)

You will now have full view of what's happening without needing to have access to DAO or any other Core class

Files that are not inheritable

Other times you will need to have access to framework files, here the recommendation splits again:

• 1. Systems under development

  2. Long time running systems

For systems under development, I recommend to add and keep a core with basic debugging functions (production copy has all debugging info commented). Only once the system is released the debugging information is removed

For long time running systems, if there's really no option, the debugging file should be prepared and replaced on server so per instance conditional debugging can be run.

Another last option for servers with many systems will be to keep on server 2 copies of the framework, one with debugging instructions and a production one, it's very easy to update the configuration of a single system to use one or the other.

Debugging advanced applications

Debugging have become increasingly complex in web applications as there are too many elements taking part.

Common components

Let's start by inspecting the common components:

• 1. Database server

  2. Web server

  3. PHP

  4. Framework

  5. Application

  6. Browser

  7. JS

  8. CSS

  9. XML

The real problem with Rich Clients is that they hide information from us, the data comes in one of many formats "behind the curtains" making access to data very complicated.

It's important to notice that to debug these applications many times you will need good understanding of HTML, CSS, JS, PHP, XML and AJAX. If you don't feel confident about your abilities on some, request help from your peers or your project manager after you have pinpointed the problem.

Tools to debug applications

• 1. Firefox

  2. Firebug

  3. Web developer (optional)

Common errors in rich applications

Let's review some common errors in rich applications:

• 1. Missing files / resources

  2. Missing updates

  3. CSS Class conflict

  4. Data related problems

     1. Incorrect data

     2. Server errors make the data dirty

     3. UTF with BOM files

  5. Cache results don't let you see the results

  6. Incorrect JS

1. Missing files / resources

One of the most common issues we find in applications is that developers forget to download or deploy some files, CSS files, support images, etc..

Images

This mistakes are subtle and sometimes hard to notice as some browsers simply don't show anything, unfortunately other browser will show a massive X in place of the missing image.

CSS

You will normally notice you are missing a CSS file because the screen looks weird (layout), or the font colors, backgrounds have changed

JS

Javascript errors are caused many times by not loaded JS files.

Debugging

You can easily find which files are missing using firebug,

• 1. Open firebug

  2. Go to Net tab

  3. Select All, or the specific type you suspect

  4. All missing files are written in red

     1. You can mouse over them to see the complete path of the file and then verify if the resource exist on server or not

     2. You can click the + symbol, to check the requesting Headers, the Response header and the HTML

Paths are correct, resources still missing

In some environments, resources will keep failing to download, this is specially true when you load a lot of files. Tequila offers some strategies to prevent this problem. To enable them: file minimization, joining and monitored downloading, you simply:

• 1. Open includes/config.php (server side)

  2. Find the js_includes configuration section (package it's included in core)

  3. Modify these lines to enable:

$js_safeloading = true;

$js_blocktillcomplete = true;

$js_retry = 3;

$js_cachedir = 'temp';

This will minimize, join all JS files, cache results for faster execution, monitor downloading, retry up to $js_retry times and optionally ($js_blocktillcomplete) Block the screen use until the files are loaded successfully.

* Notice. Minimization require all your JS files to be very well done (at least all closing brackets and ending semicolons). Tequila uses a standard minimization library and there are many tools online to check your library is valid, try this very strict validator JSLINT

2. Missing updates

Another common reason of involuntary bug induction is incorrect application updating, as files and resources are split across the application it's quite common that partial updates are made. i.e. Template files are not updated, or JS or CSS or...

Solution

Use a file comparison tool, or use mercurial to upgrade your application.

3. CSS Class conflict

CSS is awesome, it allow many levels of definition that allow precise change of look and layout, unfortunately some times CSS definitions can crash. i.e. A YUI component / JS component define the look of <div> and so your application does. The final result is a mix of both definitions.

Debugging

• 1. Find out the source of the conflict

     1. Right click on the conflicting element

     2. Inspect element

     3. Verify on the left panel you are on the right HTML tag

     4. Check the list of styles on the right panel, to see which one is defining the tag

        1. Each entry also mentions the file and line number

Solution

Once you have identified the conflicting resources you can:

• 1. Decide to remove one of them (this removal can be specific to one page, i.e. a complex one-screen)

  2. Modify one of the CSS to be more specific

     1. You can use CSS syntaxis to affect just the children of one node, i.e. #mainlayer div

     2. You can modify the HTML to apply an specific class to the affected element

     3. You can add a class to parent to differentiate, (i.e. HTML: <div class="special_case"> CSS: .special_case div )

4. Data related problems

Modern applications transmit a lot of data using ajax, this data is hard to debug as it's normally not visible, so the 1st step is to make data visible.

Viewing exchanged data

• 1. Open firebug using the icon on the right bottom corner

  2. Click on console

     1. You will see a list of all AJAX calls

     2. Open them by clicking on the + icon

     3. Params tab: You will be able to see the information sent by JS

     4. Headers tab: You can see the headers sent by server

     5. Response tab: You can view here the reply from server

Common problems

Based on this data you can start moving towards a solution, some common problems you can find are

• 1. Server errors make the data dirty

  2. Incorrect data

  3. UTF with BOM files

1. Server errors

This is the easiest error to debug, simply inspect the "response tab" like a normal web page and work over the PHP in server to clean up all the errors.

* If you are sending all parameters using GET (querystring). You can also right click on the Console entry and select Open in New Tab

This will allow you to inspect data in detail, you can switch server modes from XML to cd, to get rid of the headers and be able to read the complete errors. (Simply replace rrt=xml -> rrt=nh or cd; read RRT section on Tequila manuals for more info on Requested Response Types)

2. Incorrect data

Incorrect data might point to a logic problem in the server code, use your normal debugging techniques to solve this issue.

Another possible problem is that server and client versions are out of sync. Verify which data is correct before starting modifications. Then work on the server or JS files as necessary.

3. UTF with BOM files

Sometimes you will discover everything is correct, data is perfect, JS is in place and still you keep getting errors from your ajax callback function (this happens specially with XML data).

There's a subtle bug with files saved as UTF with BOM, the problem is that when they are included, PHP immediately start the output making impossible to set the correct XML headers.

Solution

Open every file included in the task with Notepad++ or another editor that support this setting and re save them WITHOUT BOM (optionally you can enable output cache but this is only recommended for testing, cache code is commented in index.php file)

5. Cache results don't let you see the results

This is a common issue, sometimes you own browser cache keeps showing images, css or data from previous calls. You can clear the cache or install the "web developer" plug in mentioned above, simply:

Click on Disable button > Disable Cache.

This will allow you to have a clean environment all the time.

Server cache

We have found on some online servers that PHP pages are cached and the changes made on the PHP are not reflected, try adding some extra info, an extra echo or some visual element that let you know your code is being executed. You might even be connected to the wrong server :)

If required, duplicate the file you are testing, and use this copy for debugging / developing. Update the real file after results are satisfactory, normally the server will start using the updated version after some hours.

6. Incorrect JS

Once you have determined that the server is sending the right responses and all resources are loaded, it's time to move debugging to the client side, You can work on this debugging in many ways.

Echo, echo echo.

Using the exact same technique as in PHP, you can use alert() at some places to inspect execution values.

debug.js

Tequila includes a very basic library to inspect values at running time, you must include it at server side to be able to use it on client, a couple of useful functions are:

function showobj(myObj, title, complete)

function showScript (content, safehtml, title)

Firebug!

And your best friend again: line by line debugging. Before you get crazy following each line, you should inspect your code in an external editor and determine the function you think might be the problem, i.e. the callback function, in ajax

To start debugging in firebug

• 1. Open firebug (icon on bottom right side of the screen)

  2. Click on script tab

  3. Under script you will find 2 selectors, the 2d one indicates the name of the library being displayed

  4. Open this selector and select the library you want to debug

  5. Move the left pane to the line of code you want to inspect

  6. Add a breakpoint by clicking on the left hand side of the numbers, a red spot will appear

  7. Keep the window open

  8. Trigger in the normal window the action you want to inspect

  9. Execution will stop in the breakpoint you added, you can then

     1. Control execution advance using the icons on top of the code, right side

     2. Watch the value of all variables including objects on the right pane

     3. You can also add custom watches on the right pane

Useful links on debugging:

• http://www.ibm.com/developerworks/library/os-debug/

• http://www.phpfreaks.com/tutorial/debugging-a-beginners-guide

• http://www.php-debugger.com/dbg/installation.php

• http://xdebug.org/

Good luck and happy debugging!

Itzco