Web Service Toolkit for FPC & Lazarus

Inoussa OUEDRAOGO's Home page 


 

Web Service Toolkit” is a web services package for FPC, Lazarus and Delphi; “Web Service Toolkit” is meant to ease web services consumption and Creation by FPC, Lazarus and Delphi users.

Current Version is 0.7

Content( wiki page at Lazarus wiki site )

  • Client Side ( Service Consumption )
  • Overview 
  • Example 
  • Server Side ( Service Creation )
  • Overview 
  • Example
  • Go further
  • Status
  • TODO 
  • Download
  • Changes list 
  • Author 
audit et mesure d'audience visiteurs internet par

 

Client Side 

overview

“Web Service Toolkit” is made of two parts:

  • a set of programs : “typ_lib_edtr” a WSDL based type library editor, a command line tool “ws_helper” and a Lazarus integration package which contains some wizards,

  • a collection of support units.

Given an interface definition file( a WSDL file or a pascal file describing à web service ), “ws_helper” ( or within Lazarus the WSDL file importer wizard ) will create a object pascal unit containing a proxy implementing that interface.

At runtime when a call targeting the web service is issued, the proxy's role is to :

  1. marshall the call paramaters,
  2. make the call to the target web service,

  3. receive the call return and unmarshall output parameters to the caller.

Behind the scene, the proxy will take care of the SOAP plumbing details.

Example

We will use the “Amazon E-Commerce Service”, freely available ( you will need to create a account, it's free) for personal use at this address “http://www.amazon.com/gp/browse.html?node=3435361”.In order to use this service, we have to download its exposed WSDL(Web Services Description Language) interface and translate it to Pascal language. To that end, within Lazarus we can use the import wizard; we can also use ws_helper as a standalone program.

The “WSDL importer” wizard

This wizard is contained in the wst_design.lpk package located in the \ide\lazarus directory of the toolkit. Once the package installed, it adds a menu section “Web Services Toolkit” with two (2) sub-menus items to the Lazarus “Project” menu ( figure 1 ) :

  • Import WSDL file ...

  • Type Library Editor ...

 Figure 1 : WST menu

 

Click on the first menu item(Import WSDL file ...), it will bring up the dialog below( figure 2). We specify the WSDL file and the directory where to store the generated files and click on OK button to finish.

Figure 2 : WSDL importer wizard

 Figure 2 : WSDL importer wizard

It will produce two (2) pascal source files :

  • AWSECommerceService.pas, the service definition file ( Pascal equivalent of the WSDL file )

  • AWSECommerceService_proxy.pas, this file contains a proxy which implements the service interface defined in the first file.

Import by ws_helper.

The ws_helper program is a command line version of the import wizard. In order to present its capabilities, below are printed the command line arguments it supports.

[Listing ws_helper usage]

ws_helper [-uMODE] [-p] [-b] [-i] [-oPATH] inputFilename

  -u MODE Generate the pascal translation of the WSDL input file MODE value may be U for used types or A for all types
  -p Generate service proxy
  -b Generate service binder
  -i Generate service minimal implementation
  -o PATH Relative output directory
  -a PATH Absolute output directory

[/Listing]

To translate the WDSL file from the “Amazon E-Commerce Service” execute the following command at the prompt:

[Listing WSDL parsing]
..\..\ws_helper\ws_helper.exe -uA -p -o. AWSECommerceService.wsdl
ws_helper, Web Service Toolkit 0.5 Copyright (c) 2006, 2007 by Inoussa OUEDRAOGO

Parsing the file : AWSECommerceService.wsdl
Interface file generation...
Proxy file generation...
Metadata file generation...
File "AWSECommerceService.wsdl" parsed succesfully.

[/Listing]


below is printed an extract of the two generated files :

[code File = AWSECommerceService] 

unit AWSECommerceService;
{$IFDEF FPC} {$mode objfpc}{$H+} {$ENDIF}

interface

uses SysUtils, Classes, TypInfo, base_service_intf, service_intf;

const
   sNAME_SPACE = 'http://webservices.amazon.com/AWSECommerceService/2007-04-04';
   sUNIT_NAME = 'AWSECommerceService';

type

   HelpRequest = class;
   Help_RequestArray = class;

(...)

   AWSECommerceServicePortType = interface(IInvokable)
     ['{305A7E48-DD92-4C20-B699-4F2B47C93342}']
     function Help(Const HelpParam : Help_Type):HelpResponse_Type;
     function ItemSearch(const ItemSearchParam : ItemSearch_Type ) : ItemSearchResponse_Type;
     function ItemLookup(Const ItemLookupParam : ItemLookup_Type ) : ItemLookupResponse_Type;
(...)
  end;

procedure Register_AWSECommerceService_ServiceMetadata();

[/code] 


The complete amazon sample is located in the \samples\amazon folder. The Synapse Library is required to compile the program as it is used for the HTTP communication. This library can be downloaded free of charge at this location. The Indy Library or ICS library can also be used.

 

Server side ( Service Creation ) 

Overview.

Web Service Toolkit contains a server side framework and a WSDL based type library editor for service creation. Key features are :

  • Service definition ( interface ) is separated from implementation,

  • Interface and implementations are not bound to message protocol,

  • WSDL generation

  • Support for SOAP 1.1 serialization

  • Support for XMLRPC serialization

  • Support for custom binary serialization

  • The framework is not bound to a transport protocol.

Example

In order to create a service, we have to :

  • define its interface,

  • provide an implementation and register that one for the service,

  • provide a binder that will route calls targeting the service to the implementation and register that one,

  • host the service into an application server( TCP server, HTTP Server, Library server,... ).

 

Defining the service Interface

Starting from the 0.5 version, the WST provides a WSDL based Type library Editor to define types and services used by an implementation. The figure (3) below presents the general interface of this tool. The Type library Editor is provided as

  • a Lazarus wizard as a menu item “Project/Web Services Toolkit/Type Library Editor..”

  • a standalone program typ_lib_edtr.exe

Figure 3 : Type library editor 

Type Library Editor features:

  • Graphical user interface

  • WSDL source view

  • Pascal source view of the library

  • Pascal Proxy source view

  • Pascal implementation skeleton source view

  • Pascal Proxy binder view

  • enumeration creation interface

  • class creation interface

  • array creation interface

  • type alias creation interface

  • service interface creation interface.


Below are showed some sample images.We will use the user_service_intf.wsdl file located in the \samples directory for our sample.

 Figure 4 : Class type definition

Figure 5 : Service interface definition 

Export the pascal files

The Type Library Editor has option to generate the pascal version of the WSDL file. To do so, click on the “Files\Save generated files ...” ( it can also be done through the contextual menu ); It will bring up the dialog box showed in the figure 6.

 Figure 6 : Export Pascal files

Click OK the button to complete. The program ws_helper has the same capabilities and the files can be generated with the following command :

[Listing ws_helper execution session]
ws_helper\ws_helper.exe -uA -p -o. "user_service_intf.wsdl"

ws_helper, Web Service Toolkit 0.5 Copyright (c) 2006, 2007 by Inoussa OUEDRAOGO

Parsing the file : user_service_intf.wsdl
Information : Parsing "tns:TUserArray" ...
Information : Parsing "tns:TUser" ...
Information : Parsing "tns:TUser" ...
Information : Parsing "xsd:string" ...
Information : Parsing "tns:TUser" ...
Information : Parsing "xsd:string" ...
Information : Parsing "xsd:boolean" ...
Information : Parsing "TUserArray" ...
Information : Parsing "TUser" ...
Information : Parsing "TUserCategory" ...
Information : Parsing "TNote" ...
Interface file generation...
Proxy file generation...
Metadata file generation...
File "user_service_intf.wsdl" parsed succesfully.

[Listing ]

The complete projects of the example is located in the folder “samples”. Below is printed a extract of the generated interface file.

[Code]
unit user_service_intf;
(...)

type

  TUserCategory = ( Normal, Admin  );

  TUser = class(TBaseComplexRemotable)
  private
    FCategory : TUserCategory;
    FUserName : string;
    FeMail : string;
    FPreferences : string;
  published
    property Category : TUserCategory read FCategory write FCategory;
    property UserName : string read FUserName write FUserName;
    property eMail : string read FeMail write FeMail;
    property Preferences : string read FPreferences write FPreferences;
  end;

  TUserArray = class(TBaseObjectArrayRemotable)
  private
    function GetItem(AIndex: Integer): TUser;
  public
    class function GetItemClass():TBaseRemotableClass;override;
    property Item[AIndex:Integer] : TUser Read GetItem;Default;
  end;

  UserService = interface(IInvokable)
    ['{CA6F6192-C3DE-4D9C-B3DF-E616376A0DC9}']
    function GetList():TUserArray;
    procedure Add(Const AUser : TUser);
    procedure Update(Const AUser : TUser);
    function Find(Const AName : string):TUser;
    function Delete(Const AName : string):boolean;
  end;
(...)
[/code]

 

Providing an implementation for the service

The user_service_intf_imp.pas unit generated above contains a skeleton implementation class for the interface. It defines a procedure named RegisterUserServiceImplementationFactory. The procedure registers the class as the service implementation provider in the implementation registry.

[code]
Unit user_service_intf_imp;
{$IFDEF FPC} {$mode objfpc}{$H+} {$ENDIF}
Interface
Uses SysUtils, Classes,
     base_service_intf, server_service_intf, server_service_imputils,
     user_service_intf, cursor_intf;

Type
  { TUserService_ServiceImp }

  TUserService_ServiceImp=class(TBaseServiceImplementation,UserService)
  Protected
    function GetList():TUserArray;
    procedure Add(Const AUser : TUser);
    procedure Update(Const AUser : TUser);
    function Find(Const AName : string):TUser;
    function Delete(Const AName : string):boolean;
  End;

  procedure RegisterUserServiceImplementationFactory();

Implementation
(...)

procedure TUserService_ServiceImp.Add(Const AUser : TUser);
var
  locObj : TUser;
Begin
  locObj := Find(AUser.UserName);
  if ( locObj <> nil ) then
    raise Exception.CreateFmt('Duplicated user : "%s"',[AUser.UserName]);
  locObj := TUser.Create();
  locObj.Assign(AUser);
  FUserList.Add(locObj);
End;

procedure RegisterUserServiceImplementationFactory();
Begin
  GetServiceImplementationRegistry().Register(
    'UserService',
    TImplementationFactory.Create(TUserService_ServiceImp)
      as IServiceImplementationFactory
  );
End;

(...)      
[/code]

 

Providing a binder for the service.

The binder's role is to:

  • unpack the incoming message,

  • set up the call stack,

  • make the call against the registered implementation,

  • serialize the execution stack to create the return message.


The user_service_intf_binder.pas unit generated above contains :

  • TUserService_ServiceBinder : the actual binder class,

  • TUserService_ServiceBinderFactory a factory class for the binder and

  • Server_service_RegisterUserServiceService : the binder factory registration procedure.

The following code extract shows the unit interface part and a method handler of the binder.

[Code part of user_service_intf_binder.pas]
unit user_service_intf_binder;
{$IFDEF FPC} {$mode objfpc}{$H+} {$ENDIF}
interface
uses SysUtils, Classes, base_service_intf, server_service_intf, user_service_intf;

type
  TUserService_ServiceBinder=class(TBaseServiceBinder)
  Protected
    procedure GetListHandler(AFormatter:IFormatterResponse);
    procedure AddHandler(AFormatter:IFormatterResponse);
    procedure UpdateHandler(AFormatter:IFormatterResponse);
    procedure FindHandler(AFormatter:IFormatterResponse);
    procedure DeleteHandler(AFormatter:IFormatterResponse);
  Public
    constructor Create();
  End;

  TUserService_ServiceBinderFactory = class(TInterfacedObject,IItemFactory)
  protected
    function CreateInstance():IInterface;
  End;

  procedure Server_service_RegisterUserServiceService();

(...)

procedure TUserService_ServiceBinder.AddHandler(AFormatter:IFormatterResponse);
Var
  cllCntrl : ICallControl;
  tmpObj : UserService;
  callCtx : ICallContext;
  strPrmName : string;
  procName,trgName : string;
  AUser : TUser;
Begin
  callCtx := GetCallContext();
  TObject(AUser) := Nil;
 
  strPrmName := 'AUser';  AFormatter.Get(TypeInfo(TUser),strPrmName,AUser);
  If Assigned(Pointer(AUser)) Then
    callCtx.AddObjectToFree(TObject(AUser));
 
  tmpObj := Self.GetFactory().CreateInstance() as UserService;
  if Supports(tmpObj,ICallControl,cllCntrl) then
    cllCntrl.SetCallContext(GetCallContext());
 
  tmpObj.Add(AUser);
 
  procName := AFormatter.GetCallProcedureName();
  trgName := AFormatter.GetCallTarget();
  AFormatter.Clear();
  AFormatter.BeginCallResponse(procName,trgName);
  AFormatter.EndCallResponse();
 
  callCtx := Nil;
End;
[/Code]


Host the service into an application server.

The application server's role is to route incoming service requests to the Web Service Toolkit runtime. For the runtime to process service requests :

  • The services and their implementations have to be registered ,

  • The message protocol (SOAP, XMLRPC, binary,...) have to be registered.

The runtime interface is defined in the server_service_intf unit. This unit contains :

  • GetServerServiceRegistry, which returns the service registry,

  • GetServiceImplementationRegistry which returns the service implementation registry,

  • GetFormatterRegistry which returns the message format registry and

  • HandleServiceRequest which is the unique entry point for request processing.


Starting from the version 0.5, the toolkit provides a simplified model to develop applications server . This is achieved using the listener classes. A listener implements a transport between the server and its clients. The toolkit provides three (3) listeners implementations:

  • TwstIndyHttpListener ( indy_http_server.pas ),

  • TwstIndyTcpListener ( indy_tcp_server.pas ) and

  • TwstSynapseTcpListener ( synapse_tcp_server.pas ).

All listeners derive from TwstListener defined in the server_listener.pas file.

Below is printed a Indy based HTTP server sample. The code is divided into three (3) parts :

  • messaging format registration : Server_service_RegisterSoapFormat(), Server_service_RegisterXmlRpcFormat(), Server_service_RegisterBinaryFormat(); )

  • service implementation and binder registration : RegisterUserServiceImplementationFactory(); Server_service_RegisterUserServiceService();

  • the listner creation and starting : the listener is created by the line AppObject:=TwstIndyHttpListener.Create();. It is started by the line AppObject.Start()

 [Code]
program http_server;
{$mode objfpc}{$H+}
uses
  {$IFDEF UNIX}{$IFDEF UseCThreads}
  cthreads,
  {$ENDIF}{$ENDIF}
  Classes, SysUtils,

  indy_http_server, metadata_service, logger_extension, server_listener,
  server_service_soap, server_binary_formatter, server_service_xmlrpc,  
  config_objects, user_service_intf, user_service_intf_binder,
  user_service_intf_imp;

var
  AppObject : TwstListener;
begin
  Server_service_RegisterBinaryFormat();
  Server_service_RegisterSoapFormat();
  Server_service_RegisterXmlRpcFormat();

  RegisterUserServiceImplementationFactory();
  Server_service_RegisterUserServiceService();
  AppObject := TwstIndyHttpListener.Create();
  try
    WriteLn('"Web Service Toolkit" HTTP Server sample listening at:');
    WriteLn('');
    WriteLn('http://127.0.0.1:8000/');
    WriteLn('');
    WriteLn('Press enter to quit.');
    AppObject.Start();
    ReadLn();
  finally
    FreeAndNil(AppObject);
  end;
end.  
[/Code]

Server_service_RegisterUserServiceService located in the user_service_intf_binder unit registers the UserService service by calling in turn GetServerServiceRegistry:

[Code located in user_service_intf_binder.pas]
procedure Server_service_RegisterUserServiceService();
Begin
  GetServerServiceRegistry().Register(
    'UserService',TUserService_ServiceBinderFactory.Create() as IitemFactory
  );
End;
[Code]

RegisterUserServiceImplementationFactory located in the user_service_intf_imp unit ( generated by ws_helper ) registers the UserService implementation by calling in turn GetServiceImplementationRegistry:

[Code located in user_service_intf_imp.pas]
procedure RegisterUserServiceImplementationFactory();
Begin
  GetServiceImplementationRegistry().Register(
    'UserService',
    TImplementationFactory.Create(TUserService_ServiceImp)
      as IServiceImplementationFactory
  );
End;
[Code]

Server_service_RegisterSoapFormat located in the server_service_soap unit ( provided by the toolkit ) registers the SOAP implementation by calling in turn GetFormatterRegistry:

[Code located in server_service_soap.pas]
procedure Server_service_RegisterSoapFormat();
begin
  GetFormatterRegistry().Register(
    sPROTOCOL_NAME,
    sSOAP_CONTENT_TYPE,
    TSimpleItemFactory.Create(TSOAPFormatter) as IitemFactory
  );
  RegisterStdTypes();
end;
[Code]

Server_service_RegisterBinaryFormat located in the server_binary_formatter unit ( provided by the toolkit ) registers the Binary message implementation by calling in turn GetFormatterRegistry:

[Code located in server_binary_formatter.pas]
procedure Server_service_RegisterBinaryFormat();
begin
  GetFormatterRegistry().Register(
    sPROTOCOL_NAME,
    sBINARY_CONTENT_TYPE,
    TBinaryFormatterFactory.Create() as IitemFactory
  );
end;
[Code]

Server_service_RegisterXmlRpcFormat located in the server_service_xmlrpc unit ( provided by the toolkit ) registers the XMLRPC message implementation by calling in turn GetFormatterRegistry:

[Code located in server_service_xmlrpc.pas]
procedure Server_service_RegisterXmlRpcFormat();
begin
  GetFormatterRegistry().Register(
    sPROTOCOL_NAME,
    sXMLRPC_CONTENT_TYPE
    TSimpleItemFactory.Create(TXmlRpcFormatter) as IItemFactory
  );
end;
[Code]

In order to give it a try one have to :

  1. compile the server ( \samples\tcp_server\tcp_server.lpi it is a console program),

  2. compile the client application ( \samples\user_client_console\user_client_console.lpi ),

  3. execute the server and start listening,

  4. execute the client.


Go further

The toolkit is provide with a documentation in the \doc folder. The complete Amazon example is located in the “samples\amazon” folder. It demonstrates use of class and array data types. The server samples is located in \samples\tcp_server, \samples\http_server and \samples\lib_server folders. Delphi samples are located in samples\delphi.


Status

The toolkit is usable for simple types and for class types. The serialization is designed to allow customization of basic types and class types by implementing classes derived from “TBaseRemotable. This classes have to be registered in the type registry.

The toolkit is provide with :

  • SOAP 1.1 message format implementation ( FCL XML units or Delphi xmlDOM )

  • XMLRPC message format implementation ( FCL XML units or Delphi xmlDOM )

  • JSON-RPC message format
  • binary one format implementation.

Serialization

The serialization is based on the IFormatterBase interface located in the base_service_intf unit.

 

SOAP serializer

The SOAP serializer implements SOAP 1.1. It has support for the following pascal types:

  • Available integers :

    • Byte mapped to unsignedByte

    • ShortInt mapped to byte

    • SmallInt mapped to short

    • Word mapped to unsignedShort

    • LongInt mapped to int

    • LongWord mapped to unsignedInt

    • Int64 mapped to long

    • Qword mapped to int

  • String mapped to string

  • Boolean mapped to boolean

  • Enumerations mapped to their string representation

  • Float types :

    • Single mapped to float

    • Double mapped to double

    • Extended mapped to double

    • Currency mapped to float

  • Object (class intances, not TP ones ) :

    The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the base class used by the formatter interface to allow customization of the serialization.. The toolkit provides the TBaseComplexRemotable class which implements serialization for its ( or its descendants ) published properties.


Binary serializer

The Binary serializer is more efficient in time and space compared to the SOAP serializer . It uses big endian to stream data. It has support for the following pascal types:

  • Available integers :

    • Byte

    • ShortInt

    • SmallInt

    • Word

    • LongInt

    • LongWord

    • Int64

    • Qword

  • String

  • Boolean

  • Enumerations

  • Float types :

    • Single

    • Double

    • Extended

    • Currency

  • Object (class intances, not TP ones ) :

    The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the base class used by the formatter interface to allow customization of the serialization.. The toolkit provides the TBaseComplexRemotable class which implements serialization for its ( or its descendants ) published properties.


Class type serialization.

The toolkit has support for instances of classes derived from TBaseRemotable. TBaseRemotable is the abstract base class used by the formatter interface to allow customization of the serialization.. The toolkit provides the TBaseComplexRemotable class which implements serialization for its descendants classes published properties. It also provides TBaseObjectArrayRemotable class for serialization of array of TBaseRemotable descendant classes.


Test cases

The toolkit uses FPCUnit (FPC) or DUnit (Delphi) for test cases. The test project is located in the \tests\test_suite folder for FPC and \tests\test_suite\delphi for Delphi. There are tests for basic types and classes types.

TODO( Client Side )

    • Basic array support for SOAP

    • Basic array support for Binary format

    • XMLRPC formatter

    • More documentation and samples !

    • WSDL to pascal converter


      TODO

      Extend the toolkit to Server side for :

      • SOAP

      • Binary serialization

      • XMLRPC

      • TCP transport ( first implementation).

      • WSDL generation

      • More documentation and samples !


      Download

      The up to date sources are available (Subversion) at https://lazarus-ccr.svn.sourceforge.net/svnroot/lazarus-ccr/wst/trunk
      0.5 version is actually outdated.

      Current version 0.7
      0.6

      0.5 

      0.4 

      0.3.1

      0.3

      0.2.3.2( Added proxy User and Password )

       0.2.3.

       0.2.2


      Changes list 0.7

      • FPC 3.1.1+ support
      • Many fixes and enhancements (Please see the commit log).

      Changes list 0.6

      • License change : All the code in WST is now using the FPC' RTL license (modified LGPL see COPYING.modifiedLGPL)
      • WSDL/XSD Parsers :
        • <import> and <include> schemas parsing
        • Pascal Unit renaming
        • Handle top level(global) declared attribute and references
        • More XSD constructs parsing
        • Parser case sensitivity
        • Document wrapped parameters handling : generation of an easy access interface
      • Type Library Editor:
        • Beautification : +images for menu items, +Tool bar
        • Support for External XSD Schema referencing.
        • Documentation for objects
        • Show object dependency
        • Show object XSD schema
        • Collection based arrays support
      • Run Time :
        • FPC's Native HTTP Client transport
        • FPC's Native TCP Client transport
        • FPC's Native HTTP Server
        • FPC's Native TCP Server
        • iOS client http transport
        • Client Data Filter (use to manipulate the payload : encrypt, compress, ...)
        • Client side HTTP Cookie management
        • Better SOAP headers support
        • Better WSDL generation
      • Others:
        • More test cases
        • General code improvements

      Changes list 0.5

      • Lazarus IDE WSDL importer wizard
      • WSDL GUI editor ( Type library editor ) 
      • Listener design : easy server developpement( Indy HTTP, Indy TCP, Synapse TCP )
      • XMLRPC support : client and server  
      • Switch from the custom pascal parser to fcl-passrc
      • Server side : implementation objects pooling
      • Better Delphi support ( SOAP XMLRPC and binary format support, DUnit test suite )
      • Better WSDL parsing
      • TStringBufferRemotable ( reading a node's raw buffer )

       

      • WSDL to Pascal translation support 
      • Hosting into Apache web server as Apache module  
      • Header support ( client and server side )
      • Server side service extensions
      • WSDL generation 
      • Pascal Basic types array are supported
      • All pascal Basic types are supported
      • Array support for Binary serializer
      • FPCUnit test cases 


      Author

      Copyright 2006-2014 Inoussa OUEDRAOGO.  

      inoussa12 <at> gmail <dot> com.

      The toolkit uses the same licence as the FCL one, Modifyed LGPL, which allows static linking to  proprietary software ( thank Sekelsenmat for clarification).