Version Beta www.capesoft.com

Contents
	Recommended Reading   General Information   Introduction   Installation New Users   Jump Start   Examples The RULES!!!   [NB: Very Important!]   Adding FM3 to your application   Errors (Compile Warnings and asserts , Program GPFs or Hangs) FAQ Debugging & Support   Purchasing FM3   Distribution       In Depth   AutoUP   AutoBUILD   AutoFIX   AutoNET       Technical Reference Guides SQL Documentation (essential reading when converting from TPS to SQL) Technical Reference       Other Information   Helpful Utilities Hot Tips   Miscellaneous Topics   To Do List   Version History Licence, Copyright, and Disclaimer       Can't find what you're looking for?   Click here to Search the FM3 Docs

Introduction

File Manager 3 (aka FM3) builds basic automatic file management into your program. It is designed to be a part of your application, not a seperate utility. FM3 supports a growing number of file drivers and backend databases. FM3 also supports data conversion between file drivers.

Supported Databases:

• Btrieve (Pervasive - but not SQL - i.e. Scalable is not supported)
• Clarion (dat files)
• TopSpeed (tps files)
• MSSQL (SQL Server)
• Oracle
• SQLAnywhere
• ODBC: (currently limited to MSSQL, ORACLE, MySQL (2 and 3), Firebird, PostgreSQL*)
• SoftVelocity's IP Driver (IPD)

* PostgreSQL is limited to Clarion6 and up.

FM3 also endevours to support SoftVelocity's Dynamic File Driver (DFD). This driver is still reasonably new (as of the writing of these docs), and therefore has gone through a few changes between versions, often resulting in FM3 needing to find a new way to support them. Support for the In-Memory Database Driver (IMDD) is superfluous to FM3.

As per CapeSoft's standard policy, File Manager 3 supports 3 versions of Clarion, currently Clarion 5, Clarion 5.5, and Clarion 6. File Manager 3 only supports 32bit applications.

FM3 consists of a Template and a DLL.

Please note: FM3 is currently in beta, and may not support all features across all drivers at this stage! See Version History for updates on features.

Main Features include:

AutoUP: For Automatic File structure upgrading.
AutoBUILD: Runtime File Management control template.
AutoNET: For Automatic Program upgrading across a LAN or WAN.
AutoFIX: For Automatic detection and fixing of TPS data file corruption using TPSFix.
C-Scan: Standalone distributable TPS and Clarion file scanner or browser.
UPGView: Standalone utility for viewing the FM3 upgrade table "upg.tps".
BDE: Standalone utility Bulk Dictionary Editor.

Limitations:

FM3 does not support Group Arrays.
SQL Drivers do not support Memos (see the SQL Tips and Tricks section for workarounds).
SQL Drivers do not necessarily support the standard Date and Time datatypes (see the SQL Tips and Tricks section for details).
SQL Drivers do not support Dynamic Indexes.
SQL Relational Integrity on the backend.

Note: Normal Groups, and normal Arrays are supported. If using Clarion 6.x, then BLOBs are supported.

Installation

It is vital that you follow the complete instructions for installing FM3. As per CapeSoft standard, File Manager 3 adheres to the standards specified by the C3PA (Clarion 3rd Party Association).

Run the supplied setup program. (Download the Latest Version!)

In earlier versions of FM3, you were required to run the following Template Utility. If you're running version 3.26 or earlier, complete the installation as follows:

Running the Template Utility

Open the Clarion IDE.
Open any application.
Choose Application > Template Utility from the menu.
Select the Support ABC Template Utility from the FM3 Class.
Enter the name of your Clarion directory. For example: C:\Clarion5.
Click on OK.

Note: We highly recommend upgrading!!

Notes on the Template Utility:
It is strongly recommended that you do run this Template Utility, even if you're not using ABC.
You only need to run it once. You do not need to run it for each application, or when you upgrade FM3.
You will need to re-run this after upgrading Clarion in anyway (patches, ClarioNet, etc).
If you own FM2 also, you need to run both template utilities.
You will need to run it for each version of Clarion you're installed on your machine, if you're using them.

New Users

If you are using FM3 for the first time, pay attention to these details, and follow these recommendations.

Before You Start

Read the Documentation!
Vital Reading includes: The RULES, Getting Started, AutoUP section, etc...
Open, run, look, and play with the examples.
Do not assume you have completed the installation - check, and check again!
If the examples don't work, then it's unlikely that your app will, so find out why - read the docs!

Getting Started

In most circumstances, it is vital that there are no Invalid Record Declaration errors (Error 47) when you first add FM3 to your app.
If you do already have an Error 47 in your non FM2/3 enabled application, read the How to handle a pre-existing Error 47 section.
Jump Start helps you with the very basics in learning to add FM3 to your app in minutes.
Start with FM3 simple features, and add the more advanced ones as you become more familiar with the working of FM3.
For example, start by adding AutoUP to a TopSpeed driven application.
Topspeed to SQL Convertors: Do not attempt converting your app to SQL until you are more familiar with FM3.
If you are experiencing errors, or problems with FM3, check the FM3 Errors and FAQ sections for possible solutions.
If you are unable to find your error / question in the docs, carefully follow the instructions laid out in the Debugging and Support sections before contacting Support..
SQL Users, read the FM3 SQL Documentation thoroughly.

Where to from here?

Enjoy the time and stress you save by using FM3 in every application! <g>
See how AutoBUILD, AutoFIX, and AutoNET make your life even easier.
Once you've familiarized yourself with the basic functionality of FM3, you can look at more advanced features, such as Converting your app to SQL.

New to SQL?

FM3 makes a conversion from TPS to SQL very much easier than any other method, BUT... SQL is a whole new ball game as it were. Do not add FM3 straight off and expect it to do what you want. It is vital you have done some research into SQL. Specifically how it works with Clarion, what some of the oddities are, and how they can be overcome with various tips and tricks. If you are expecting your dct and app to stay the same for a SQL backend, then you're being a little too optimistic. SQL opens up a whole new world to your programming, and will be a very powerful tool, once used correctly. The nature of SQL is quite different to flat file systems such as TPS, and comes with more options or "variables in the equation" as it were.

Below are a few tips of where to start with Clarion / SQL programming, and then some FM3 specific tips and tricks.

Read the Clarion documentation under SQL Accelerator Drivers.
We are building up a pretty good Clarion SQL Reference ourselves. See our FM3 SQL Documentation.
Try out our SQL Examples and Jump Start before attempting your own SQL Apps.

Examples

File Manager 3 ships with a wide selection of examples. It is highly recommended you go through these examples before attempting to use FM3 in your own apps. It is important to have an understanding of the basic setups of FM3, as well as seeing it in action. If the examples work, and you follow our documentation for use, then your applications will work too!

You will find the examples installed in your Clarion\3rdParty\examples\FM3 folder. Each example has it's own folder described below.

Here's the break down of examples shipped:

Examples
		Name	Description	Drivers
		FM3Demo:	Main example demonstrating AutoUP, AutoBUILD, & AutoFIX.	TopSpeed, Clarion.
		SinglABC:	Single exe ABC app.	TopSpeed.
		SinglEXE:	Single exe Clarion / Legacy app.
		MultiABC:	Multi dll ABC apps.
		MultiDLL:	Multi dll Clarion / Legacy apps.
		MultiDET:	Multi dll with DET support.
		MultiDEF:	Multi dll with DEF support.
		MSSQL:	MSSQL example demonstrating conversion from TPS to SQL.	TopSpeed, Mssql.
		ORACLE:	Oracle example demonstrating conversion from TPS to SQL.	TopSpeed, Oracle.
		ODBC:	ODBC examples demonstrating conversion from TPS to SQL.	TopSpeed, ODBC.
		MABCMSS:	Multi dll ABC MSSQL example.	Mssql.
		MLEGMSS:	Multi dll Clarion / Legacy MSSQL example.	Mssql.

In order to get these examples working, you need to install the relevant database server (preferably onto your local machine as this will omit connectivity problems that may occur). Check the Firebird tips for some pointers on installing a Firebird server and database.

Keen to get started? Then this section is just for you - the basics on how to get FM3 into your application as quickly and simply as possible.

Adding FM3 to your application

Follow these simple steps below. If you're an FM2 user upgrading to FM3, then check out the Converting your FM2 apps to FM3 section in the docs.

Please note: At this point, please ensure you have completed the installation process by running the Support ABC Template Utility!

Adding the Global Extension

Open your app.
Click on the Global Button.
Click on Extensions.
Click on Insert.
Select the Activate FM3 Extension from the list (check out the Template Reference: FM3 global extension template for more details).
Click OK.

Adding the Project Define (Clarion5 users)

Click on the Project Button.
Highlight the very top line (Project: Generator).
Click on the Properties Button.
Click on the Defines tab.
If it does not already exist, add: FM2=>1
Click OK.

Please note: This define MUST read FM2=>1, and not FM3=>1 as you might expect. This ensures FM2 and FM3 can run side by side.

If you're using a SQL backend - adding the SQL_Connect window

Run the 'Import the SQL Connect procedure' template utility.
You'll need to set the Global Owner variable in the SQL_Connect window's "Connect To SQL Backend" Control Template prompts. To do this, double click on the SQL_Connect window and click on the "Connect To SQL Backend" control template to bring up the prompts and enter the Global Owner variable in the field provided on the General tab.
In the FM3 Global Extension template (on the AutoUp tab) you must set the SQL Connect procedure to the newly created SQL_Connect procedure. Goto Template Reference: FM3 global extension template for more details. 
You need to add the project define to your application (if you are not using MultiProj's Driver Substitution) - check out the SQL_Connect window Control template for the project defines you need to add to your project depending on which driver your application is supporting.

If you're using AutoBUILD/AutoFIX, then add the RuntimeFileManager procedure

Run the Template utility: ImportRuntimeFileManagerABC/Legacy.
In the FM3 Global Extension template (on the AutoFIX tab) you must set the RuntimeFileManager to the newly created RuntimeFileManager procedure. Goto Template Reference: FM3 global extension template for more details. 

In your Global properties (on the File Control tab), make sure that you've checked the 'Generate all file declarations' checkbox.

The Alternative - creating a separate application to handle file conversions.

What to do in your Dictionary

You will need to add a Version item to each file's user options list and set it to 1. You can do this easily with CapeSoft's Bulk Dictionary Editor (BDE). You will need to increment the version number every time you make a change to the file properties or file structure.

For other file options check out the section on How to make basic file structure changes

That's basically all you need to add to your application. To find out more detail on the different aspects of FM3, read the following documentation sections: AutoUP, AutoBUILD, AutoFIX and AutoNET. Oops - before getting too excited, don't forget to read The Rules.

The RULES!!!

Here are the RULES! If you break them, then you're on your own! If you obey them, you're well on your way to being a very happy customer with friendly support! <g>

NEVER EVER decrease the version number!!! Do not do it!
NEVER EVER delete the upg.tps file! Do not do it! (Check out the UPG.tps in the AutoUp section)
Once FM3 is applied, DO NOT change the prefix of a file. FM3 uses the File Prefix to identify files. If this is absolutely necessary, see Changing the File Prefix.
Add FM3 to your application and run it before making any dictionary changes.
ie: if you already have an Error 47 in your app, see the How to handle a Pre-Existing Error 47 section.
Before contacting Support, make sure you have obeyed the above Rules, and have read the Support section in the documentation!

Do not convert your file using the Clarion dictionary convertor. This will make your UPG.tps file out of sync with the file structure of the data table.

Do not re-use file-prefixes (if you delete a file and then re-use that file prefix, FM3 will not work). Use a new unique one.

AutoUP

Short Description: Automatic Runtime File structure upgrading, and converting.

AutoUP Contents
		Who Should Implement AutoUP?
		Description
		Implementation
		How to make basic File Structure changes
		Overrides (also check the Dictionary User Options Quick Reference and Dictionary User Options)
		The UPG File (upg.tps)
		Miscellaneous
		SQL

Who should implement AutoUP?

Anyone who understands the time and frustration of upgrading client sites with the new file structures for every change.
SQL Users, if you are using one of the currently supported drivers / databases, and you don't have a DBA.
Those wanting to convert from 1 supported file driver to another.

Description

AutoUP allows your program to automatically upgrade it's own data files when they need to be upgraded. This means that all you have to do is change the dictionary, re-compile and distribute. AutoUP will do the rest.

When you change your dictionary you can change anything. The file driver, the password, add fields, remove fields, change keys, make arrays bigger or smaller etc. You can also change field types, including memos and strings etc.

AutoUP will also create the file if it does not exist. For this, please ensure the CREATE file attribute is on.

Currently UnSupported Structures

Dimensioned Groups (e.g.: MyDimGrp   group,dim(5)).
SQL Drivers do not support Memos (see the SQL Tips and Tricks section for workarounds).
SQL Drivers do not necessarily support the standard Date and Time datatypes (see the SQL Tips and Tricks section for details).
SQL Drivers do not support Dynamic Indexes.
SQL Relational Integrity on the backend.
DAT file (Clarion 2.1) to TPS where the files are OEM collated. The DAT file driver uses a different algorithm to the TPS file driver to interpret OEM characters. So although byte-for-byte what's in the file will be the same, the characters are interpreted differently.

Note: Ordinary Dimensions, and ordinary Groups are supported!

Implementation

Before applying AutoUP, make sure you have assessed your current situation, and read the relevant documentation. AutoUP works differently in different applications, and it is vital you have an understanding of some of these things before attempting an upgrade. For example, AutoUP is applied differently between ABC and Legacy application, Single exe and Multi-DLL, and even between different file drivers. Please follow the instructions for your specific scenario.

If you haven't already done so, add the FM3 Global Extension. Goto Template Reference: FM3 global extension template for more details. 

Adding AutoUP to your Single exe ABC Application

On the Global Extension, AutoUP Tab, check on the file drivers used in your application, including any you have used in the past. Goto Template Reference: FM3 global extension template for more details. 

Adding AutoUP to your Multi-DLL ABC Application

For Multi-DLL ABC, the Global Extension should be added to the DATA DLL only.
On the Global Extension, AutoUP Tab, check on the file drivers used in your application, including any you have used in the past.
If you are using a Runtime File Manager and / or Connect Template (SQL), then these procedures should also be added to the DATA DLL. You can use the Template Utility to do this.
You may Export the Runtime File Manager.

Note: SQL users in Clarion 6 Gold: Due to a bug in Clarion 6, it is recommended you add the FM3 Global Extension to the main exe app aswell as the DATA dll. Make sure you check on the necessary driver support, but leave the Connect Procedure Blank! This will allow your program to be properly terminated, instead of the process hanging at 90% CPU.

Adding AutoUP to your Single exe Legacy Application

On the Global Extension, AutoUP Tab, check on the file drivers used in your application, including any you have used in the past.

Adding AutoUP to your Multi-DLL Legacy Application

For Multi-DLL Legacy, the Global Extension should be added to ALL DLLs and the exe.
On the Global Extension, AutoUP Tab, check on the file drivers used in your application, including any you have used in the past.
For each DLL app, add a source procedure named: <appname>:FM3Init, where <appname> is the name of the app.
Open the Embed Editor for this procedure, and double click on "Processed Code".
Select VLDS_DLL - FM3 DLL Init Function from the list:



Now, in the EXE app, Global Extension, Options Tab, click on the Root DLL Init Names button.
Fill in each of the names created above.
If you are using a Runtime File Manager and / or Connect Template (SQL), then these procedures should also be added to the DATA DLL. You can use the Template Utility to do this.
You may Export the Runtime File Manager.

Adding AutoUP to your IP Driven Application

Add FM3 to the Data Management DLL (That's in the IP Server app.)
Remove FM3 from your Client app if it exists, whether it's a multi-dll or single exe.
If you are using a Runtime File Manager, then leave FM3 in your Client app data dll, and add the RuntimeFileManager there.
If FM3 remains in your client app, ensure that the IPDriver check box is checked on in the Global Extension.
NOTE: It's a good idea to use a variable for the connection string for the IPServer (i.e. in the owner field). Otherwise, if you need to change it, FM3 will think you're making a file change and request a version number increment.
NOTE:  You must copy the FM3 dll (c6fm3x.dll for Clarion6 applications located in the Clarion\3rdparty\bin directory) to the ClarionDataServer directory (where the Clarion IPDriver Server resides).

Some additional cases:

Case 1:
Note: If you would like FM3 to manage local client-side files, then leave FM3 in your data dll or exe, and check the IPDriver check box on the Global extension.

Case 2:
If you have applications that use the TPS files on the IPserver as local TPS files, then these applications will need to use the upg.tps file that exists in the IPServer database. These applications can be treated as normal TPS applications (where adding FM3 is concerned). If you have Multi-Proj, then you can use the Driver Substitution feature to make 2 exes from the same application - one that is IPDriver enabled and one that is TPS enabled. Otherwise you will need 2 apps - one for IPDriver and one for TPS.

Case 3:
Your data does not exist in the root directory of the IPServer. Insert the following code into your Data DLL (in the 'FM3: Before initialization' global embed point):

ds_SetUPGPath()

Case 4:
You have data both locally and at the IP Server. You would like to access the IP Server only if there is a connection available.
At this stage, the table management is not optional at application level. In order to cater for this situation, you need to pull FM3 out of your application and make 2 separate applications for upgrading - one for the IPServer files, and one for the local files. In each app, you need to turn the Generate all Files checkbox off (This is on the Global Properties), and manually force the compiler to generate each IPDriver file (for the IP Driver management app) and then for each TPS file (for the local upgrade app) - also on the Global Properties - Individual File Overrides tab. On the Runtime FileManager of each new application you have just created, you'll find the RuntimeFileManager Control Template. Set the Use Files option to Files Used in this APP.

Known issue: Once adding FM3 to the IP_Server Dll, a Memory Block free'd twice error occurs on unloading of the dll. We are not certain of the cause, but this will only occur when you close the IP_Server Dll. The message will remain for about 20 seconds, and then disappear harmlessly. Not recommended: Further to this, you can set your IPREQ service to not allow interaction with the desktop. This will ensure the error is not visible - although be careful, because if you have any errors, they will not be displayed, so Possible file upgrading without changing the version number errors will cause your DLL to freeze on the IP Server.

AutoUP and the Dynamic File Driver (DFD)

Because the majority of DFD usage is using handcoded data structures, you will need to handcode the FM3 functionality

• You will need to make a call to the ds_AddDriver function - if you have no other files using that file driver in your dictionary (IOW this function should be called once for each driver that your application's files use, and that FM3 is required to upgrade).
• After creating the DFD data structure, make a call to the ds_UsingFile function. You will need to manually increment the version number in this call each time you make a file structure change.

The alternative - Creating a Separate Upgrading Application

Many times it's a good idea to have a separate application to upgrade your datatables. This means that the upgrading of the tables is done once at a scheduled time. Normally you would run this application at the end of the installation routine. You can do this very easily as follows:

• Create a blank application based on your dictionary.
• From the Application | Template Utility menu, select the CreateFM3ConvertorApplicationxxx utility.
• If you're wanting to include a SQL_Connect procedure in this application, then check the Include a Connect Procedure checkbox, otherwise leave it blank.
• You will see the RuntimeFileManager and SQL_Connect (if selected) procedure(s) added to this application. 
• You will need to setup the FM3 global extension template as you have in your application (for settings pertaining to AutoUP).
• You will also need to create the necessary project defines in your project to include the supported drivers.
• You will also need to modify the ConnectToSQLBackendControls control template prompts (on the SQL_Connect window) to match those in your application.

 

Where to from here...Assess your scenario:

You are adding AutoUP to a new Application.
You are adding AutoUP to an existing Application.
You are adding AutoUP for a cross-driver conversion (eg TPS to SQL).
You are adding AutoUP to a DET or DEF Application.


You are adding AutoUP to a new Application.
You are ready to run your application, make file changes, etc. Now read about How to make file structure changes.


You are adding AutoUP to an existing Application.
The key here is that you do not already have an Error 47 Invalid File Declaration present. This is especially important for TopSpeed and Btrieve files. AutoUP keeps a history of every file structure, and uses this to upgrade from the old structure to the new structure. It cannot do this if the file structure is already wrong. These structures are stored in the upg.tps file. Here are a list of suggestions for getting around this:
Change your dictionary back to a non Error 47 state for the file, and run your program to allow AutoUP to capture the file structure.
Build your own conversion program for this file.
Use the Dictionary Conversion tool.
The key is that there should be no Error 47s upon initial running of your AutoUP enabled application.

You are ready to run your application, make file changes, etc. Now read about How to make file structure changes.


You are adding AutoUP for a cross-driver conversion (eg TPS to SQL).
Please ensure you have first read the "You are adding AutoUP to an existing Application" section.
If converting to a SQL backend, it is recommended you read the SQL Tips and Tricks section, aswell as Converting your application to SQL.

You are ready to run your application, make file changes, etc. Now read about How to make file structure changes.


You are adding AutoUP to a DET or DEF Application.
DET Users should follow any Legacy specific code unless otherwise stated.
DEF Users should follow any ABC specific code unless otherwise stated.

You are ready to run your application, make file changes, etc. Now read about How to make file structure changes.
 

---

How to make basic file structure changes

As mentioned previously, you can change almost anything in your file structure. You can change datatypes, increase and decrease data lengths, add and delete fields and keys, memo to string and vice versa, etc. Please take note of the Limitations you can make to changing file structures.

Changing a File Structure
		How to add / change the version number (NB: Must be done for every file structure change!)
		How to change a file name
		How to change a field name
		How to change the file prefix
		How to prime new fields
		How to add / change a file password
		Other Dictionary User Options Available

 How to add / change the file version number

In some cases, there may be a little more to be done, but for the most part, this is all you need to do:

Make the necessary change in the dictionary.
Then, for each file you have changed, go to the file Properties, Options tab. Either add or increment the following:
Clarion 5:

Clarion 5.5 and above:
Add a string type property, name it Version, and set the value to 1 (or increment if already exists).


Important: Make sure you increase the version number every time you change the dictionary. If you forget, a warning messages will be displayed the next time you run the app. It is strongly advised that at that point you end the program, and go back to the dictionary, increase the version number without making any further changes, and re-compile.

Notes:
Never decrease the version number - ever!
You do not need to increase the version number of files you have not changed.
You do not need to change the dictionary version number.
You do not need to make any changes to your app.
You do not need to set and change the version number for an alias file. It uses the parent file's version.
You only need to add the version when you makes changes. FM3 defaults each file to Version 0 for files without a version.
Do not use the Dictionary Browse and Convert functions!
If using the Multi-Proj Driver Substitution feature, please read the Hot Topics section.
FM3, and other 3rd Party tools use other user options. To add multiple user options in the dialog, see below:
Clarion 5:

Clarion 5.5 and above:

How to change a file name

As mentioned above, in some cases of file structure changes, a little more info is necessary to enable FM3 to perform it's duty. In cases where you change the name of an object such as a file, you need to tell FM3 what the OldName was, so that it can match it up during upgrade. To do this, after changing the file name, add the following string user option to the file properties options tab:

	Property	Value
	OldName	MyOldFileName.tps

Note: No quotes should be used!

How to change a field name

As mentioned above, in some cases of file structure changes, a little more info is necessary to enable FM3 to perform it's duty. In cases where you change the name of an object such as a field, you need to tell FM3 what the OldName was, so that it can match it up during upgrade. To do this, after changing the field name, add the following string user option to the field properties options tab:

	Property	Value
	OldName	PRE:MyOldFieldName1
	OldName2	PRE:MyOldFieldName2

.
.
.

OldName14

PRE:MyOldFieldName14

Note: You must include the file prefix!
Note: No quotes should be used!

You can have up to 14 old field names listed.

How to change the file prefix

WARNING!! This is NOT recommended for use, but is documented for use when absolutely necessary. Please contact CapeSoft Support before attempting this change!

Changing the file prefix was previously not allowed in FM3. FM3 uses the file prefix as a unique identifier for the file in the upg.tps file. We have now added the ability to change the file prefix. However, this is not advised! If you change the prefix, you should not use the old prefix for any future file, as this could cause error 47's. FM3 will still associate this old prefix with the original file. Also, this will only work if your new prefix is alphabetically directly after the the old one! If you must change the file prefix, after changing the file prefix, add the following string user option to the file properties options tab:

	Property	Value
	OldPrefix	OldPRE

Note: No quotes should be used!
Note: Do not make any other changes to the file while performing this upgrade!

For more information, and syntax differences in Clarion versions, read the Dictionary User Options section.

How to prime new fields

When you add a new field, it is quite common to want that field to be filled with data for all existing records in the file. AutoUP provides an easy way to do this. Go to the Properties dialog of the field you've added, and on the Options tab, add one of the following:

To prime a field with numeric data

	Property	Value
	SetIfNew	44

To prime a field with character data

	Property	Value
	SetIfNew	Hello World!

To prime an autonumber field

AutoUP can automatically prime a new autonumber field for you. It will start at 1 and increment by 1 for each record.

	Property	Value
	SetIfNew	AutoNumber

Note: SetIfNew AutoNumber is not supported in SQL. Check out AutoNumber in the Dicitonary User Options for more details.

To prime a field with an existing fields data

In some circumstances, you may want to prime your new field with data from an existing field. A common example of this would be adding a new Postal Address field, and priming it with the existing PhysicalAddress field.

	Property	Value
	OldName	PRE:FieldName

Note: This variable will be set ONCE before each file conversion, so the same value will be populated into the new field for each record changed. If you are wanting to populate each variable with a unique value, then you need to create a process to run through the file and populate the new field with a variable after the file conversion.

Tip: To prime a field with an existing field in the same table's data, use the oldname user option of that existing field.

To prime a GUID field

AutoUP will automatically prime each empty GUID field it finds during upgrade. The field must be named GUID, or PRE:GUID.

To prime a field with a value from a function

AutoUP can automatically prime fields using a function (or variable). Note that you need to prime variables for this function before FM3 initializes:

	Property	Value
	SetIfNew	' & today() & '

Note: This function will only be called ONCE before each file conversion, so the same value will be populated into the new field for each record changed. If you are wanting to populate each variable with a unique value, then you need to create a process to run through the file and populate the new field with a variable.

How to add / change a file password (Owner)

For non-SQL drivers, the owner attribute serves as a way to password protect your files. For SQL, the Owner attribute of a file serves as a connection string. (For converting an encrypted TPS file to SQL, use the OldOwner user option)

For security reasons, it's better to use a variable for this value, and prime it inside your program. In order for FM3 to be able to manage the file correctly, you'll need to prime this variable before FM3 Initialisation. The best place to place this code would be in the Program Setup Embed with the Priority set to 2000. Please note, this priority must be set less than 3000 thousand to ensure it executes before FM3 Initialisation. Alternatively, you can place it in the FM3:Start Of Initialisation Embed.

NB: If you are adding or changing the password for a file, it is absolutely necessary to change the name of the file too, to allow FM3 to continue managing the file. See Changing the name of a file above for more info.

To set the owner attribute as a variable, place an exclamation mark ("!") in front of the variable name. For example:


For more information on the SQL Connection String, see the SQL Reference.
 

AutoUpLimitations

See also the SQL Reference for SQL specific limitations.

General:

• You can't dimensioned groups in an FM3 managed table.
• You cannot change a DATE or TIME to a LONG (or visa-versa) - as FM3 moves the data byte for byte without interpretting the bytes. This means that the data will be invalid because these datatypes store data differently.

BTrieve Specific:

• You cannot have the Memo=single attribute on BTrieve files.

---

Dictionary User Options

Introduction

Dictionary User options are a nifty feature in Clarion that allow you to declare and prime custom variables for use in custom templates. These user options are available at all levels, including dictionary options, file options, field options, key options, and relation options. To add user options to the dictionary, you go the Options tab of the desired object's properties. You can select what datatype the option should be, and then give it a name, and a value. The datatypes available are String, Boolean, Integer, and Ini. All FM3 User Options use the String type. The most common FM3 user option is of a String type. It belongs at File level, and is called Version. Here's a few screen shots demonstrating this:


The following section is the FM3 User Option Reference. Level refers to the object properties you need to use - ie, file, field, key etc. So, if you need to add a Field Level User option, go to the field properties dialog, and select the option tab there.
 

---

Dictionary User Options Quick Reference

Here is a Quick Reference guide to all FM3 User Options.

 

Dictionary User Options In Detail

Version

This is the most important user option of the lot. In order for FM3 to maintain a valid history of the data structures for a file, it must keep track of the file version. This is a STRING user option added at FILE level. You must increment the value of this user option after every change made to that particular file. When you first declare a file in your dictionary, you do not have to add this user option, as FM3 will assume the file is new, and therefore at version 0. The next time you change that file structure, you must increment the version number by at least 1.

Dictionary > Right Click on file > choose Properties > Select Options tab > Right click on list box > Insert > Select String type > Set the label to Version > Click OK > Click on the right hand side of list box in line with the Version label > set value to 1 (or n + 1).

For more information: See How to make basic file structure changes.

OldName (File Level)

When changing the physical name of a file, the OldName user option should be added to the FILE level options. It should be of STRING type. The label should be OldName, and the value should be the name of the previous file.

If this is a constant, then your OldName user option should like this:

(Note the lack of quotation marks in the value)

If the name of the old file must be a variable, then you can enter this in the value as follows:

(Note: You must prime the variable before FM3 initializes.

OldName (Field Level)

When changing the name of a field, the OldName user option should be added to the FIELD level options. It should be of STRING type. The label should be OldName, and the value should be the old field name INCLUDING the file prefix- eg: if changing the Product Name field from ProdName to ProductName, then the OldName user option value would be PRO:ProdName (where PRO is the Product file prefix). NO QUOTATION MARKS SHOULD BE USED.

OldOwner

When changing the owner of a file (this is particularly useful when upgrading an encrypted TPS file to SQL), the OldOwner user option should be added to the FILE level options. It should be of STRING type. The label should be OldOwner and the value should be the old encryption password of the TPS file. You'll need to use quotes (for a constant) - or a variable name (the variable must be set before FM3 Initializes). Note: you cannot use this feature to upgrade encrypted clarion dat files to tps.
Note: When changing the owner of a flat file (if you're changing the

SetIfNew

This option is used when priming data into a newly added field. Check the How to prime new fields section for more details.

UnRealField (SQL)

This is a SQL Specific User option, and is used specifically when converting from TPS files to SQL files. This option should be added to the String(8) field that is mapped to a SQL datetime. This string should have a group following containing a Clarion Date and Time field. This group should be declared OVER the string. This is a FIELD Level user option, and does NOT require a value.

RealField (SQL)

This is a SQL Specific User option, and is used specifically when converting from TPS files to SQL files. This option should be added to the Date AND the Time fields that are part of a group. This group should be declared OVER a String(8) that is mapped to a SQL datetime. This is a FIELD Level user option, and does NOT require a value.

AutoNumber (SQL)

This is a SQL specific user option. It tells FM3 to create the relevant backend autonumber support on the SQL backend. For example, in MSSQL, it creates an Identity field. Add this user option at FIELD level, and of String type. Set the value to 1.

For MSSQL, you can choose to use the alternative syntax in order to prime the seed and increment values. For example a value of 200,3 will result in the autonumber field starting with a value of 203, and incrementing by 3. Most common, and most recommended is to set the increment to 1. eg: 1000,1 will start at 1001, then 1002, then 1003 etc. If you just leave the default syntax (1) then the seed and the increment will be 1.

For other backends, only the increment portion is relevant, so you need just enter (1) for an increment value.

TableSpace (SQL)

This is an Oracle Specific User option, and it sets the TableSpace for your oracle TABLES. This is a DICTIONARY or FILE Level user option of String type.

For more information on the option: See Oracle TableSpaces in the SQL Section.

IndexSpace (SQL)

This is an Oracle Specific User option, and it sets the TableSpace for your oracle INDEXES. This is a DICTIONARY, FILE, or KEY Level user option of String type.

For more information on the option: See Oracle TableSpaces in the SQL Section.

ZeroNull (SQL)

This option is used when converting TPS to SQL. TPS does not support NULL. SQL does. In SQL NULL is not the same as zero or blank. This option tells FM3 to treat 0 and Blank data from TPS as NULL, and not insert a 0 or blank in the backend. This option can be used a DICTIONARY, FILE, and FIELD Level.

For more information: See the Hot Tips section in the SQL Section.

OldPrefix

WARNING! This action is not recommended unless absolutely necessary. In order for this to work, you must follow the rules. This is explained in detail in the AutoUP Section. For more information: See How to change a file prefix.

IgnoreDriver

This option is used when converting Clarion DAT files to TPS.

For more information: See Ignoring Pre-Existing DAT files in the Overrides Section.

FM3IGNORE

This FILE Level user option surprisingly enough tells FM3 to categorically ignore the file. FM3 will not manage this file at all. If the structure changes, you will get an error 47. It's a String type, and takes no value. The existence of this user option means the file is ignored, regardless of the value.

DctMasterFields (SQL)

This is a DICTIONARY and FILE Level user option. It takes a value of 1 or 0. Please see the SQL Field Management Section before using this user option.

DctMasterKeys (SQL)

This is a DICTIONARY and FILE Level user option. It takes a value of 1 or 0. Please see the SQL Key Management Section before using this user option.

DctMasterRelations (SQL)

Not yet supported.

DctMasterField (SQL)

This is a FIELD Level user option. It takes a value of 1 or 0. Please see the SQL Field Management Section before using this user option.

DctMasterKey (SQL)

This is a KEY Level user option. It takes a value of 1 or 0. Please see the SQL Key Management Section before using this user option.

DctMasterRelation (SQL)

Not yet supported.

ForceSQLDataType (SQL)

This is a very handy little user option. It's a FIELD Level User option, and does exactly what it says. It will force a certain SQL Datatype on the backend for the field. This comes in handy when wanting use use the Binary Datatype, or the UniqueIdentifier, or force another type for the field. Simply set the value to the desired SQL datatype.

Please note: This leaves the ball in your court to ensure the forced datatype matches, or is compatible with the dictionary structure. In other words, if you try to force a datetime datatype on a decimal field, you will get an error 47.

Some Examples:

ForceSQLDataType

tinyint

ForceSQLDataType

varchar(50)

ForceSQLDataType

uniqueidentifier NULL ROWGUIDCOL DEFAULT NewID()

---

Overrides

You may, for some reason wish to override FM3's default behaviour. This section provides some of those options, and when and why you would use them.

Ignoring Pre-existing DAT files

FM3 automatically searches for any pre-existing DAT files that you may be converting to TPS. If your application does not have pre-existing DAT files then you can disable the search that FM3 does for them. This is necessary when you have DAT files of the same name as your new TPS files, and they are in the same directory.

For example: App 1 uses file Customer.dat and App 2 uses file Customer.Tps, but the two "Customer" files are unrelated.

To set this option for all files:
Go to the Global Extension Options Tab.
Check on the "No pre-existing Clarion files" checkbox.

To set this option for a specific file:
Go to the Dictionary.
Open the File Properties Dialog.
On the Options tab, add the following String User Option:

	Property	Value
	IgnoreDriver	1

Note: For more information, and syntax differences in Clarion versions, read the Dictionary User Options section.

Ignoring a particular file

If you wish to ignore a particular file, for whatever reason, then you can do this by using the FM3IGNORE switch. If you ignore a file then it is removed from FM3's sight, as if it didn't exist in the dictionary. This includes the ability for C-Scan to scan the file, if it is a TPS or Btrieve file. If all you want to do is make it be ignored by the File Manager 3 template then use the "Exclude files" button on the Runtime File Manager Extension.

To set this option for a specific file:
Go to the Dictionary.
Open the File Properties Dialog.
On the Options tab, add the following String User Option:

	Property	Value
	FM3IGNORE	1

Notes:
In this case, the value can be left blank.
For more information, and syntax differences in Clarion versions, read the Dictionary User Options section.

Downgrading files

By default FM3 prevents your Topspeed, Btrieve, and SQL files from being accidentally downgraded.

By default it allows downgrading of Clarion files, this is a side effect of the support for pre-existing Clarion files. To disable the automatic downgrading of DAT files, see the Ignoring Pre-Existing DAT files section above.

If you want to force a downgrade of a file for any reason then you can attempt it by placing /Downgrade on the command line when running the program.
For example: c:\myexedir>myexe /downgrade

Here's an example circumstance when this may be required:
1) Your clients have version 1 of your app (with FM3 enabled).
2) You ship them version 2. They omit to take a backup of the data before running the new version.
3) The new program runs and updates one or more files. However there is a major problem in your program necessitating a move back to version 1 of the EXE.
4) Usually FM3 would *not* downgrade the files from Version 2 status, but if you run the Version 1 exe with the /Downgrade switch then FM3 will downgrade the files back to version 1 level.
 

---

The UPG file (Upg.tps)

The UPG file is what allows FM3 to work it's magic! But, how? you ask...trust me, it's magic! <g>

When do I distribute the UPG file?

	Case 1:	You've got a new app that you are distributing to new clients. You are using any file driver.
	What you do:	You DO NOT distribute the UPG file. It is created, and managed for you, at the client's site, automatically.

	Case 2:	You've got an existing app you want to upgrade. The app currently uses Clarion DAT files (maybe it was written in CPD or CDD). You may or may not be converting the data to the Topspeed / Btrieve file formats.
	What you do:	You DO NOT distribute the UPG file. It is created, and managed for you, at the client's site, automatically.

	Case 3:	You have an existing app using Topspeed or Btrieve files. You are thinking of doing an upgrade, but you haven't changed any file structures yet.
	What you do:	Before changing the file structures, add FM3 to your app, re-compile and distribute. You DO NOT distribute the UPG file. It is created, and managed for you, at the client's site, automatically.

	Case 4:	You have an existing app using Topspeed or Btrieve files. You have already changed some file structures in the dictionary since you distributed the program to your clients.
	What you do:	If you can revert to Step 3 then do so.
	Alternative 1:	Add FM3 in the normal way. Upgrade the existing clients to the new version using one of the older methods (i.e. using the CW IDE or Conversion programs). From then on later versions will upgrade automatically.
	Alternative 2:	Add FM3 in the normal way. Compile a version of your app using the OLD dictionary structure. Compile and run the EXE. Distribute the generate UPG.TPS file to the existing clients with the next upgrade. You do not need to ship it to new clients, or to clients who have already received it once. Make sure that in your install, you set the installer to only install the UPG.tps file if it is non-existant on the clients machine.

Forcing the position of the UPG file?

By default the UPG file is placed in the application directory, in the same place as the calling program. In certain circumstances you may need to override this. If for example you access the data from a number of EXE's, stored in different locations, or if you don't have write access to the application directory, then it makes sense to store the UPG file in the data directory.

In fact, there is a strong case for getting the UPG file to appear in the data directory.

Here are some of the advantages for this:
When the user backs up the data the UPG file is also backed up.
If you take a copy of the users data for support reasons then the UPG file is included.
On a network, the data directory is likely to be shared, but the application directory would probably not be shared.

To set the position of the UPG file, use the ds_SetUPGPath() function. This should be called in the FM3:Start Of Initialisation Global Embed.

For example:

   ds_SetUPGPath()            ! forces upg file to 'current' directory.

   ds_SetUPGPath(datapath)    ! datapath is a variable of yours containing the correct path.

Note: If you are writing a Multi-DLL ABC application, then make sure you call this function in the Data DLL.

Some more details regarding the UPG.tps file For those that care:

 

---

Miscellaneous

Support for Multiple data sets

Non-SQL:FM3 does support multiple datasets as is (for Non-SQL files). However, it may be a good idea to keep a upg file for each dataset, thus ensuring backup and restore integrity. Generally, changing the data path within the application is done by calling SetPath() or FileDialog(). Simply ensure that for each of these function calls within your program, you call ds_SetUPGPath() directly afterwards to ensure a upg file is created, and maintained in the working directory. You'll need to call ds_CloseUPGFile() before setting the path as well.

SQL: FM3 can only manage 1 SQL database at a time. This is determined by the connection information given on the FM3:Connect To SQL Backend Extension Template. Your program can still access files contained within other databases, but FM3 is not able to upgrade them. To upgrade files in a different database, restart the program, connecting to the required database. Alternatively, you can create a separate conversion application specifically for the other database, and run the conversion application form your installation program to upgrade that database's table structures.

Note: FM3 will still upgrade all non-SQL files if necessary while running your program on an SQL backend.

Very Large Dictionary Support

If you have a very large dictionary, and possibly a lot of Aliased files, together with FM3's initialisation code, this could extend the allowed size of a Clarion module. In such cases, you may receive an error such as: Link Error: Too many segdef in file Error(6): cif$fileclose The handle is invalid.

First thing to try is to clear all your obj and obj32 directories, restart your machine, ensure you are compiling in 32bit mode, and try again. If the error persists, read on.

In such cases, you will need to seperate FM3's init code into a few blocks. The VLDS:FM3 Init Code Template does just this!

To add the VLDS:FM3 Init Code Template:
Create a new soure procedure named FM3InitProc. (Don't forget to check the 'Declare Globally' checkbox on the procedure's properties).
Open the embed editor, and add the code template VLDS:FM3 Init.
This prompts for 2 values: Start and End. It also contains a check box called Generate File Loop Only?
Start is the alphabetic character of file names to start generating from (typically "A"). and
End is the Last alphabetic character of file names to genereate init code for.
It may well be that you'll need to split the init code into more than one procedure.
If this is the case, then you would break the init code by specifying smaller alphabetic ranges for each init procedure.
Eg: FM3InitProc might be "A" to "M", and FM3InitProc2 would follow on from "N" to "Z".
There is also a "File Loop Only" check box. This is to specify whether to generate all init code, or just the file loop.
For the first init code template, you would leave this box unchecked,
but for any further init code procedures you declare, you'll need to check this box on,