Creating Web Services
Note: NetTalk Web Services requires xFiles
There are a variety of common features that programs can offer that
collectively are known as Web Services. NetTalk 8 introduced two new
Procedure templates, called NetWebService and
NetWebServiceMethod to assist
with creating web services.
is the name for a collection of service methods.
application tree you can have multiple NetWebServiceMethods for a single
NetWebService. The NetWebService is just a container, and does not
include much more than just a list of the methods.
application can contain multiple NetWebService procedures (each one with
one or more methods.)
All methods support a variety of encodings,
for input and for output. Possible input encodings are XML (using xFiles),
JSON (using jFiles) or Form-URL-Encoded. Possible output encodings are
XML or JSON.
Supported Web Service standards include WSDL, SOAP
In NetTalk 7 and earlier a hand-coded approach to web services was
demonstrated in an example called SOAPServer (42).
This example is included for reference purposes.
However a new
example, WebService (77) is provided from NetTalk 8
and as it makes use of the new templates, is a better example for
developing future WebServices.
The service, and each method is self documenting. From any browser you
can see the documentation for the service and for each method. The
documentation includes as much information as possible so that other
developers can easily make use of your service.
To see the
documentation in your browser navigate to one of the following links;
A WSDL file is a formal, computer-readable way of documenting a web
service. Using this file, other programmers can write programs that make
use of your service, and the existence of this file makes their lives
NetTalk automatically (and dynamically) generates
this file for you when it is requested. A WSDL file for your service can
be retrieved by using one of these links;
The method template has a place where you can set the incoming data fields, and the returning
data fields. These fields need to be declared in your application either
as tables, table fields, global data or local data.
Incoming fields can
be any simple data type or a TABLE, GROUP or QUEUE structure. (If a QUEUE
structure is used then
declaring it as a local queue is recommended - global queues are strongly
Like a normal Clarion procedure the method can take
multiple incoming parameters and (depending on your code) the caller may
only need to send some of them - in other words some of the incoming
parameters may be optional.
If a Table is selected as an incoming
parameter, then another parameter (tablename_action
) is also
expected from the caller. The action parameter informs the server of the
nature of the change or fetch.
The template includes validation
options for the parameters, but you can also add your own hand-code
validation where necessary.
Unlike a normal Clarion procedure,
the method can return multiple variables. Return values can be simple
fields (including Local or Global variables), Queues, Groups, Tables and
All methods can return one or more errors in place of the
declared return values. If any errors exist, then none of the
other declared variables are returned. For more on errors see
All methods can also make use of
the standard ServiceResult
template code will especially make use of this when an incoming Table
parameter is used. For more on standard results see
Fields in Returning VIEWs
On of the Return types supported is a VIEW. Views are particularly
useful here because they allow you to send a subset of a table (ie
rows, or columns in the table can be suppressed.)
the fields in the Table are included in
the view. You can override this by adding only the fields you want
to export to the View Fields list in the template. The fields will
be formatted according to the dictionary settings, unless this
feature is disabled for this method, or globally.
 All is perhaps not quite true. You can set
fields in the dictionary so they will not be included.
Field User Option NETAPI
exists, and is
set to 0
, then this field will not be
included. If it is set to 1
field is included (even if it is over another field.)
switch allows you to
expressly exclude the "parent" field of the OVER, and expressly
include the "child" field of the OVER. It can be very important to
do this if you have a GROUP over a STRING.
following case;ArriveStamp STRING(8)
In this situation, by default, the ArriveStamp field
would be automatically included. This is almost certainly not
desirable though because "Binary" data is not allowed in XML (it's
ok in Json) and because unless the client is a Clarion program this
string is meaningless anyway.
In this case it would be
advantageous to set NETAPI
to 0 for
Probably the most common use
for a WebServiceMethod is to read and/or write database records. Since
this is a common use-case, the templates are aware of this situation,
and creating these sorts of methods is very straightforward.
- Create a NetWebServiceMethod procedure. Add it to a
NetWebService as normal.
- Create an incoming parameter of type TABLE, and select a Table.
Tick on the database actions you want to allow (Inserts, Updates etc)
- If the method allows the user to read the table then create a
return value, of type VIEW. Give the view a name, tick on
GenerateViewStructure and select the Table to view. If you leave the
Fields list blank then all the fields from the table will be
exported. Alternatively you can select a subset of fields to send to
the client. Also be sure to enter an appropriate filter here.
The service will look for an additional parameter, called
This parameter is a string and should contain one of
The wizard can now generate a web service and
a web service method for each table that you select.
Calling Methods from a Client program
The above calls are typically made from a browser, and the goal is to
determine what the service can do, and how to use it. To actually make use
of the service though you need to call one of the methods.
A service is a collection of one or more methods. You can create your
own methods using the NetWebServiceMethod
You can add as many methods as you like to a service, and your application
can contain as many services (ie collections of methods) as you like.
have a name, a list of incoming parameters, and a list of returned data.
While a Clarion procedure only returns a single piece of data, a web method
can return any amount of data, including GROUP and QUEUE structures.
procedure type supports a number of calling techniques;
Normal HTTP GET. The URL is of the form
/service/method. Incoming parameters are passed as part of the URL or
as cookies. The returned result is a simple XML structure. For example;
It's also possible to call the method using the PUT
or DELETE commands instead of GET. For
In the method itself you can determine which Verb was
used by checking p_web.RequestMethodType.
Typical possible values are one of NetWebServer_GET,
NetWebServer_POST and NetWebServer_DELETE.
- Normal HTTP POST. The URL is of the form
/service/method. Incoming parameters are passed as Post Data, but in
addition to the URL and Cookies, data can be passed in as plain "POST Data".
POST /School/GetSchoolTeacher HTTP/1.1
- SOAP 1.1. The URL is of the form /service.
In this case a POST is used, but the incoming Post Data is formatted as XML,
and (optionally) wrapped in a SOAP envelope. A HTTP Header called
SOAPAction: is also set to the method name.
POST /School HTTP/1.1
<?xml version="1.0" encoding="utf-8"?>
- SOAP 1.2. The URL is of the
form /service. the request is very similar to a
SOAP 1.1 request, although the SOAPAction header is not included. the
content-type of the SOAP 1.1 request is text/xml whereas the content-type
for a SOAP 1.2 request is application/soap+xml. For example;
<?xml version="1.0" encoding="utf-8"?>
If the API will be consumed by another web page, in other words
(Cross Origin Resource Sharing) is a mechanism whereby a
server "tells" a browser whether the resource can be consumed by the
This only affects browsers - other clients will not
be affected by this.
In order for your server to allow CORS
requests, from other browsers, you will need to set the
Your Code in the method
Services can include automatically generated methods (more on that in a
moment) but a NetWebServiceMethod procedure doesn't actually do anything
unless you add the necessary code.
In this sense a method can do
"anything" that you code it to do. The template will parse the incoming
request, and place it in the Parameter data structures. It will also
format the return structures into XML or JSON, and return them to the caller.
Your job is to write the code that populates the return structures.
You add your code to a routine called ServiceMethod. By the time
this routine is called the parameter structures have been primed. Once
this code is completed another (generated for you) routine will turn the
result data structures into XML (or JSON) - you don't need to worry about that,
you just write the code to populate the return structures with the
correct value. Your code should not need to care about the technique
used to call the method.
You add normal Clarion code here,
opening tables, reading data and performing calculations, just like you
would in any Clarion procedure.
You have access to the
SessionQueue here for fetching and storing information using the normal
p_web.SetSessionValue methods. However the
incoming request will be bound to the session ONLY if the
sessionID cookie is set in the request. Since the web client accessing this method
is usually not a browser, the cookie may not automatically be set.
Aside: If you are using a Clarion program as a client, and you are
using the NetWebClient class, then you can set the
NetWebClient.OptionAutoCookie property so that multiple
requests will preserve the cookies, and hence the session ID. If you are
using a different tool for the client, then you will need to research
that tool to determine how to send the cookie.
If your code does
not interact with the session queue, then you don't need to worry about
Your code can (and should) generate Errors when things go
wrong. See the next section for more on adding errors to your code.
NetTalk contains a generic Results queue (p_web.ServiceResultQueue
which is used to pass information back to the caller. This queue is
populated when the caller is adding, editing or removing records in the
database, using the generated template code.
The Queue contains
You can add your own information to the queue
if you wish. The method to call is
p_web.AddServiceResult (action, tablename, recordId, description)
A method may fail for any number of reasons. Services should return detailed, meaningful
error information to the client wherever possible. To this end NetTalk Service Methods include
a built-in, automatic, always consistent queue of errors.
can be generated automatically by the validation template settings, or
they can be generated in your method code. To Generate a method simply
(Number, Position, RecordID, Description, Recommendation)
You are free
to pass whatever you like to the AddServiceError
method, but the more information passed to the client the better.
If any errors are added to the queue in this way, then only the
errors list will be returned to the caller. None of the other return
values will be included in the reply.
is automatically added as a possible reply to the generated WSDL file
for all methods in the service.
SOAP versus REST
As you can see from the above your method will happily accept incoming
requests formatted as a SOAP XML packet. It is equally happy though to
receive the request as a simple GET, PUT, POST or DELETE command. This
is sometimes known as a REST request.
Your embed code does not
change greatly between creating a SOAP service, or a REST service. REST
is basically the same as SOAP without all the SOAP wrapping. Typically a
REST client will also use the different HTTP verbs (GET, POST, PUT and
DELETE) to match up to regular file activities. A SOAP service on the
other hand will typically use a parameter to determine the file action
The template generated code can easily handle both, for example in your
embed code you might have some code like this;
(In this example a
parameter called ACTION is assumed. If the
parameter is sent, then loc:act is set from
that, if the parameter is omitted, or set to 0, then the HTTP verb is
checked and the loc:act based on that.)
loc:act = action
if loc:act = 0
loc:act = Net:ViewRecord
loc:act = Net:DeleteRecord
loc:act = Net:ChangeRecord
loc:act = Net:InsertRecord
To be a truly RESTful method you should not need to access the SessionQueue in
order for the method to work. Ideally the client should pass you all the
information you need in order for the method to work.
Return Format Rules
NetTalk API's can return results in either XML or JSON format. NetTalk applies a number of rules to
decide which format to use. As soon as a rule is met then the decision
is made, and no further rules are tested. The rules are as follows;
- If JSON is not supported by the method then return XML.
- If XML is not supported by the method then return JSON.
- If the incoming ACCEPT header specifies XML then return XML.
- If the incoming ACCEPT header specifies JSON then return JSON.
- If the incoming request type is SOAP 1.1 or SOAP 1.2 then return
- If the incoming CONTENT-TYPE header is XML then return XML.
- If the incoming CONTENT-TYPE header is JSON then return JSON.
- If the
local template setting is not DEFAULT then use local
Overriding XML or JSON methods
The NetWebServiceMethod will generate two objects in the procedure.
The one (xml) supports XML structures, the other (json) supports JSON
These are simple object declarations based on
the xFiles xFileXML
and the jFiles
classes respectively. These
objects are used internally to parse incoming requests, and to create
There may be times when it is desirable to
override one or more methods in one, or both, of these objects. For
example when parsing xml input it may be necessary to add embed code to
described in the xFiles
.) To expose the embed points it is necessary for the
object to rather be generated by the appropriate extension template.
For XML, go to the Extensions tab and add the xFiles
extension. Set the
object name to xml
. (This is important, it MUST have
this name.) Then go to the settings for the method (Properties /
Actions), to the General tab, and turn off the Generate XML
option there. Doing these two steps means the xml object
declaration is generated by the xFiles template, and not generated by
the NetTalk template. Once this is done all the embed points for the xml
object will be defined and you can embed code in them.
you follow the same process as for XML, but add the jFiles
extension and set the
object name to json
. Then on the General tab turn off
the Generate JSON Object setting
 The JSON
object is only generated if the jFiles global extension has been added
to the app.
Formatting Fields in the Response
From build 9.17, support for automatic formatting
parameter and return fields is included.
This feature can be activated
. (It is on by default.)
It can be overridden at the
in the Method.
Specifically fields which are
stored as "not String" and have a picture, are automatically converted to strings (using
the picture) and vice-versa.
For Parameters this feature is limited
to parameter types TABLE, DATE, TIME, NUMBER and STRING.
GROUP, QUEUE, STRINGTHEORY and FILE are not supported.
For Return values this feature is
limited to FIELD (from the dictionary, not local data), TABLE and
VIEW return types. For VIEWs only Generated View fields are done
Embed points exist for both XML and JSON
which allow you to format, and deformat fields in hand-code.
Self.CurrentField = Deformat(pString,'@d6')|
For disconnected sync the server, and the desktop need to
be speaking the same language. For this reason the Desktop Sync
client now defaults to auto formatting. If you disable auto
formatting on the server side, then it needs to be disabled on
the Desktop Client as well.
is always formatted, so it is recommended that sync methods
leave auto formatting on.
For more information on formatting the output of a
field in xFiles, see here
For more information on formatting the output of a field in jFiles
Field prefixes are a thorny problem for a couple of reasons.
Firstly, they're a foreign concept in most languages, so they are
often not desirable in a public API at all. Secondly they make use
of a colon separator, which is not a valid character in a variable
name in most languages.
For this reason you have the option
to include, or exclude prefixes from parameters and return values.
In most cases having the prefix OFF seems the better option.
For builds prior to 9.17 the default for prefixes was ON. For
parameters and values added using 9.17 or later the default value is
OFF. the setting for existing parameters and returns are not changed
by the update, however support for the prefix support in the system
has been overhauled so definitely check that your
methods are still working the way you expect.
writing a web app, colons are translated into a double underscore to
make the names compatible with HTML. For
ServiceMethods though, colons are translated into a single
underscore. In builds 9.16 and earlier this was slightly
inconsistent as simple GET and POST calls still made use of the
double underscore, where XML and JSON used a single underscore. From
9.17 this has been made consistent so that all requests, and all
responses use a single underscore if prefixes are ON.
It is probable that many of the services you are providing will require
that the user authenticate themselves in order for the action to
As with normal Web apps, the method of authentication
is left largely under your control - you can determine the best approach
that suits your situation. Some approaches include (but are not limited to);
- Pass the Login and Password as fields on all incoming packets.
These fields are then first checked in your code to verify the
request before any other action is taken. The login and password
could be passed as data in the request, or in the HTTP
Authentication header. If using headers then be sure to add code to
your WebHandler procedure in the Authenticate method.
- Allow the user to log in with one request and set the session as
logged in (as you would in a normal web app). Then further requests
from the client can use the SessionID cookie with further requests.
Note however that normal session timeout rules apply here - if no
traffic from the service is received for a pre-determined period of
time, then the session will timeout.
Bear in mind
that because the service will typically be used by some service other
than a browser more complicated authentication schemes are possible.
Basic Authentication is preferred over Digest Authentication when used over a TLS connection.
To activate Basic Authentication in your application;
- Go to the WebServer procedure, Extensions, Settings,
Turn on the option Suggest Basic
Authentication for Logged In Pages.
- Go to the WebHandler procedure, to the .Authenticate method.
After the parent call add code to test the pUser
and pPassword parameters. For example;
If Access:Users.Fetch(User:Key) = level:Benign
if pPassword = User:Password
The above code is a simplification, Passwords should be stored
as SALTed, HASHed values whenever possible.
While this code is ideal for an API, it is also possible to call .Authenticate from a LoginForm
procedure, thus allowing this code to work for an API or for a Web App.
Digest Authentication is preferred to Basic Authentication when used over a non-secure connection. It
does not transmit the password in plain-text, but rather a hash of the password. However it requires the server
to store the actual password, not a hash of the password, which is not ideal. Then again, using any login over
non-secure connections is not ideal. If you are considering Digest authentication rather consider a TLS connection,
and if you have a TLS connection to the server use Basic rather than Digest authentication.
To implement Digest authentication;
- Go to the WebServer procedure, Extensions, Settings,
Turn on the option Suggest Digest
Authentication for Logged In Pages.
- Go to the WebHandler procedure, to the
.GetPassword method. After the parent call add code to
fetch, and return, the users password, based on the pUser
parameter. For example;
If Access:Users.Fetch(User:Key) = level:Benign
Authenticating a user on each request can be expensive. A SessionID is one way of "remembering" the user
and "knowing" they are already logged in. NetTalk automatically
creates sessions for all connections, and if the client makes use of
the Session then (even if they pass authentication information) then
the authentication code is not re-run.
Session ID's are
included, as a SetCookie header, in every result returned by the
server. If a client respects this cookie setting, and returns the
cookie with each request, then no authentication is required, and
the response will hence be faster. For this reason it is recommended
that clients respect the cookie field and return the cookie.
[End of this document]