Version Gold

www.capesoft.com
Updated
Monday 28 January 2008
     
 



Replicate Support Documentation


 
If your LogManager is not replicating, check out the Replicate Error Flow Diagram.
Questions on programming my application
Dictionary/File-setup related questions
LogManager programming questions
LogManager setup and running questions
Logging and Logfile questions
General questions
Replicate error flow diagram
I'm getting Compiler Errors when compiling my application
I'm getting Errors when running my program
Debugging Replicate
Support

Can't find what you're looking for? Click here to search the entire Replicate documentation.

Frequently Asked Questions

Note: Some of these questions are answered comprehensively in the docs, so I've just linked the question directly there.

Upgrading from a previous version?

U.10. What do I do when upgrading from a Replicate version prior to 2.03 beta?
U.9. What do I do when upgrading from a Replicate version prior to 1.99 beta?
U.8. What do I do when upgrading from a Replicate version prior to 1.98 beta?
U.7. What do I do when upgrading from a Replicate version prior to 1.69 beta?
U.6. What do I do when upgrading your LogManager from a Replicate Version prior to 1.49beta?
U.5. What do I do when upgrading from a Replicate version prior to 1.46 beta?
U.4. What do I do when upgrading from a Replicate version prior to 1.39 beta?
U.3. What do I do when upgrading from a Replicate version prior to 1.30 Beta?
U.2. What do I do when upgrading from a Replicate version prior to Beta 16?
U.1. What do I do when upgrading from a Replicate version prior to Beta 15?

Questions on programming my application

1.1. How do I cater for auto-numbering in my database (across sites) if I have a primary key with a unique auto-number field?
1.2. I want to disable Replicate if it's not licensed. How do I do this?
1.3. How do I handle replicating fields that are calculated?
1.4. I have excluded a file from Replication, but it has a GUID field. How do I populate this field?
1.5. Which object should I use in my application?
1.6. I have a loop with a manual add that is coming up with a duplicate error. What am I doing wrong?
1.7. How do I derive my own class - I want to use my own method of...?
1.8. I have a MsSQL database and a TPS database that I would like to Replicate between. How do I do it?
1.9. I would like to create an audit trail of changes that a user has made. How do I do this?
1.10. I would like to turn logging off at some sites - how do I do this?
1.11. I want to insert code into the Replicate methods. How do I do this?
1.12. I want to make the file structure changes, but don't want Replicate at this stage in my app. How do I populate the GUID fields?
1.13. I'm having problems replicating my external files
1.14. From a parent form I insert a child, but the Parent's GUID is not primed for the child insert.

Dictionary/File Setup Related Questions

2.1. Is it possible to conditionally replicate record, based on the flag set in the record?
2.3. How do I know which tables I should add the Site field to and which I shouldn't?
2.4. I cannot get my records (based on an ID field) to auto-increment. How do I do this?
2.5. I have a Site-related (or Site-specific) file that I want to replicate throughout the Site-tree
2.6. I have one table definition and have multiple files which use this table definition (switch filenames at runtime).

LogManager Programming Questions

3.1. I want to license the amount of sites Replicate will replicate to. 
3.2. How do I choose the transport method on the fly (i.e. copy direct if the machine is connect or email if not)?
3.3. How do I encrypt my logfiles?
3.4. Is it possible to trigger replication manually everytime I update something in the Parent Site?
3.6. What is subset data replication, and how do I institute it?
3.7. How do I institute one-way replication?
3.11. I want to use my own method of transport. How do I implement this?
3.12. The SendEmail routine is never called. What am I doing wrong?
3.13. I need to send a document (image, Word, etc) with my logfile when a record changes.
3.14. I have FM2/3 added to my application, should I add it to the LogManager application as well?
3.15. How do I get replicate to do a dial up connection?
3.16. How do I disabling the warning messages from appearing?
3.18. I want to Ignore a field/an entire record based on a specific condition.
3.19. I want to Synchronise 2 sites - how do I do this?
3.20. I would like to suppress ALIASes when doing a full export.
3.21. How do I make that replication occur less often to one site than to the other sites?
3.22. I want to customise which tables are replicated. How do I do this?
3.23. How do I export my data files in a specific sequence?
3.24. Processes constantly, generating a request to server or client at least every second.
3.25. I selected "Direct Only" when I initially made my LogManager. Now I need to add FTP/Email but I can't find any of the template prompts anywhere?
3.26 I get an Access Denied message on one of the logfiles during import. How do I get around this?

LogManager Setup and Running Questions

4.1. I'm trying to implement the Site range, but I am not getting all the records passed to my relating site.
4.2. How do I start the Replication process (i.e. what must I do to start the automatic export and import)?
4.3. I'm doing a restore from scratch. What should I look out for?
4.4. I'm doing a complete import, but not all my (for example) customers are being imported.
4.5. I'm trying to run the examples, but I cannot find any files called abcmanb000.exe etc.
4.6. I can't get the replication working. What am I doing wrong?
4.7. I want to create a mirror-site of one of my sites. How do I do this?
4.9. I want to synchronize some sites via the local network (i.e. using the direct method) and some via an Email/FTP connection.
4.11. I get 0Byte logfiles (with a tmp suffix) in FTP directories. What is the problem?
4.12. All my logfiles (from 1 to x) are being sent over to my new child site. How can this be rectified?
4.13. I create a new site and a huge logfile is sent to the child, and the child logs all these new inserts and sends it back to the parent. Why?
4.14. I have a number of logfiles in the incoming directory of a site - even after processing at that site.

Logging and LogFile Questions

5.1. How do I peek at compressed logfiles?
5.2. I'm having new files created and often my log files are not logging my file changes.
5.3. In one procedure, my changes appear in the logfile, but not in the database.
5.4. My entries to a file are not being logged, and neither is the GUID field in that file being populated (i.e. it's blank).
5.6. I have a process whose file changes are not being logged. What must I do?
5.7. I'm getting really large, bloated logfiles - and they seem to be growing exponentially.
5.8. How does Replicate handle logout/commit/rollback?

General Questions

6.1. I am getting corrupt records when saving from a form. What am I doing wrong?
6.2. How does Replicate handle concurrency checking/conflict resolution?
6.3. How does Replicate handle updates to the same record at 2 different sites between synchronisations?
6.4. I am still not sure how the Hi & Low limit works.
6.5. How do I implement translation into Replicate?
6.6. What happens when the site value in a site related record changes?
6.7. I don't understand the Last LogFileReceived Counters. Could you explain them?
6.8. Why do Logged file changes go down and return up the site tree?
6.9. How come have I got some ztmp or zetmp files lying around?
1.1. How do I cater for auto-numbering in my database if I have a primary key with a unique auto-number field?

Answer: The concept of auto-numbering is probably the biggest difference between a "single" database type application, and a replicated one. The approach Replicate takes is to allow 2 invoices to have the same number. Obviously though they then need an additional field to make them unique - and we introduce this via the Site field.  Each database location is called a Site, and all auto-numbered keys then consist of 2 fields, the site field and the auto-incrementing number.  Thus the customer goes from being 120 to CPTN120, and JHBG120 and so on. The last field (in the key) must be the auto-incrementing field.


1.4. I have existing data which I am adding Replicate to. How do I populate the GUID field, since when I convert the files it is blank?

Answer: The best solution is to use FM3 (or FM2). This will auto-upgrade the files for you and populate the GUID as well. If this is not an option, you will need to write a routine for each file as follows:

ThisRep.SetGlobalSetting('NoLogging','+')
set(MyFile)
Loop until access:MyFile.next()
  MYF:GUID = ThisRep.GetGUID()
  access:MyFile.update()
end
ThisRep.SetGlobalSetting('NoLogging','-')


1.5. Which object should I use in my application?

Answer: There are 3 different objects that are part of the Replicate classes: the csLog, csLogManager and csLogConnectionManager classes. The csLog class is the most basic, containing the ability to log changes to files (adds/edits/deletes) and setup of the local site information. The csLogManager builds on the csLog class and handles the importing and exporting of the log files. The csLogConnectionManager builds on the csLogManager class to implement the transportation of the log files (i.e. the sending and receiving of the log files).

Each Site must only have one application with the csLogManager (or csLogConnectionManager) base class running at a time (the LogManager program). In the majority of instances, this means that you will require 2 separate applications - the LogManager program, and your application (if you want more than one instance of your program running at any site). In this case, your application must have the csLog as the base class. The only instance where you would use the csLogManager (or csLogConnectionManager) class in your application is if there will only be one instance of your program running at each site. The LogManager program should be running in the background all the time, so that it can handle the importing and exporting when needed.

1.6. I have a loop with a manual add that is coming up with a duplicate error. What am I doing wrong?

Answer: You need to clear the GUID before performing the add. 

clear(PRE:Guid)

You may actually prefer to clear the entire record which has the same effect:

clear(PRE:Record)

Note: If you are doing a recursive add on a form (using ABC) you will need to do the following to clear the GUID:

1. Go to your Global Embeds | Global Objects | Abc Objects | File Managers | FileManager for <YourFile> | PrimeAutoInc | CODE (Before Parent Call)
2. Insert the following code:
clear(PRE:Guid)


1.7. How do I derive my own class - I want to use my own method of...?

Answer: The simplest is to derive it locally (i.e. using the embed points), but if you want to override a method for all your applications, then instead of locally deriving the method in all your applications (and your LogManager), you can derive the class and use your own. Here is a simple example of writing your own methods to Get, Update and Add settings (for example if you want to store them in a TPS file).

1. You need a .clw and a .inc file (like Replicate's) - let's call it MyRep.clw and MyRep.inc. For example:

MyRep.inc:

OMIT('_EndOfInclude_',_MyReplicate_)
_MyReplicate_ EQUATE(1)
include('replicate.inc')
!--------------------------------------------------------------------------------
!Class MyLogConnectionManager
!--------------------------------------------------------------------------------
MyLogConnectionManager Class(csLogConnectionManager),Type,Module('MyRep.Clw')
GetSettingWindow PROCEDURE (<string Setting>),string,name('MyLogConnectionManager.GetSettingWindow') ,VIRTUAL
AddSetting PROCEDURE (string Setting,<long option>),string,name('MyLogConnectionManager.AddSetting') ,VIRTUAL
UpdateSetting PROCEDURE (string setting,string value),name('MyLogConnectionManager.UpdateSetting') ,VIRTUAL
GetSetting PROCEDURE (string setting),string,name('MyLogConnectionManager.GetSetting') ,VIRTUAL
END ! Class Definition
!--------------------------------------------------------------------------------
_EndOfInclude_

MyRep.clw:

Member()
Include('MyRep.inc')
!-----------------------------------------------------------------------------------
MyLogConnectionManager.GetSetting PROCEDURE (string setting) ! Declare Procedure
Ans String(255)
CODE
!Put your code here
Ans = parent.GetSetting(setting)
return(Ans)
!------------------------------------------------------------------------------
!-----------------------------------------------------------------------------------
MyLogConnectionManager.UpdateSetting PROCEDURE (string setting,string value) ! Declare Procedure
CODE
!Put your code here
parent.UpdateSetting(setting,value)
!------------------------------------------------------------------------------
!-----------------------------------------------------------------------------------
MyLogConnectionManager.AddSetting PROCEDURE (string Setting,<long option>) ! Declare Procedure
ReturnString cstring(255)
CODE
!Put your code here
ReturnString = parent.AddSetting(setting,option)
return(ReturnString)
!------------------------------------------------------------------------------
!----------------------------------------------------------------------------------
MyLogConnectionManager.GetSettingWindow PROCEDURE (<string Setting>) ! Declare Procedure
ReturnString CSTRING(256)
CODE
!Put your code here
ReturnString = parent.GetSettingWindow(setting)
return(ReturnString)
!------------------------------------------------------------------------------

A bonus extra is that you will still be able to use the object embeds to derive the class locally (in your application) if you want to.
2. In the Replicate Global Extension template, check the Derive checkbox and enter your MyRep.inc in the Include file entry field, and the name of the class (MyLogConnectionManager) in the Other Class entry field.
3. Enter the MyRep.clw file into your Project (External Source files).
You will be able to use this class for all of your applications (LogManager as well). 


1.8. I have a MsSQL database and a TPS database that I would like to Replicate between. How do I do it?

Answer: Replicate is driver independant so it won't be an issue with what driver you are using. You will need a complete EXE set for each driver that you use. In other words you will need your application and the logmanager application compiled for TPS and another application and logmanager compiled for MsSQL. Please note the SQL caveats in the docs. You must make sure that your dictionary uses external field names for all the fields in the files that are replicated.

For this kind of Project I recommend using:
1. Multi-Proj. You can use MultiProj to compile the same application to different EXEs using different FileDrivers from the same dictionary. This means that you don't have to maintain an application and a dictionary for each file driver that you are using.
2. FM3. Handles database upgrading automatically (for both TPS and MsSQL).


1.9. I would like to create an audit trail of changes that a user has made. How do I do this?

Answer: You simply need to set the user property as soon as it can be set. For example:

ThisRep.SetGlobalSetting('user',ds_CurrentName(AppNum))   !For Secwin users who want to use the Secwin loggin name


1.10. I would like to turn logging off at some sites - how do I do this?

Answer: This is best done by disabling Replicate completely at a site (this is laid out in one of the Useful Tips). You can also turn logging off on the fly as well. This needs to be done with care because of the implications of not logging data. Basically all you need to do is to turn the NoLogging flag on:

ThisRep.SetGlobalSetting('NoLogging','+')


1.12. I want to make the file structure changes, but don't want Replicate at this stage in my app. How do I populate the GUID fields?

Answer: Add the Replicate global extension template to your application as per normal (after making the file structure changes laid out in the What you need to change in your Dictionary section of the docs), and in the options tab, check the 'Make Replicate InActive (GUID generation only)' checkbox.


1.13. I'm having problems replicating my external files

Answer:

  1. After making a changing to the field containing the name of the graphics file, then check that the logfile contains the change correctly (check the An external graphics file entry in the logfile for more information).
  2. If it does not, then it means either you have not setup the replication correctly, or else replicate cannot find the file. Put a stop in the derive PrimeLog method:


    If the stop does not appear, then Replicate is not detecting a field change. If the stop does appear, then check the Path and filename, and if it's not correct, then your path or filename has not been setup correctly on the Replicate global extension template.
  3. If the entry into the logfile does exist, then check the outgoing folder for the file (your application will place it there on field change). If it's not there, then you probably don't have access to the outgoing folder for that machine. Check access rights for the outgoing folder (and that you have not customized the name of the outgoing folder name).

1.14. From a parent form I insert a child, but the Parent's GUID is not primed for the child insert.

Answer: If you don't have a autonumbered field, then the parent record is only inserted when your click the OK button on the parent form (which is when the GUID will get generated). What you can do is in the PrimeFields method (or the PrepareProcedure if you're using legacy) you can do a:

PRE:GUID = ThisRep.GetGUID()

Replicate will only prime the GUID on insert if the GUID is blank.



2.1. Is it possible to conditionally replicate record, based on the flag set in the record?

Answer: Yes. Please see the example in deriving your own methods. One thing you need to note: If you set the flag (upon insert), the record then will not be propagated, so any subsequent changes (even if the CUS:DontReplicate flag is cleared (although there are ways around this though - <g>).

You need to derive the PrimeLog method as well and place similar


2.3. How do I know which tables I should add the Site field to and which I shouldn't?

Answer: There are basically 3 types of replication tables: the global table, the site-related table and the global-site table.

Questions to ask to determine which table type your table is (you'll need to do this for each table in your dictionary):

1. Do I need to ADD records at more than one site.
2. Does this table have unique keys (other than the GUIDkey)?
3. Do I need to have every record in this file at all my sites?
4. Do I use Subset Replication?

If (1is No or 2 is No) and 3 is Yes then
  this is a Global table
Elsif 3 is No or 4 is No then
  this is a Site-related table
else
  this is a Global-Site-related table .

1. The global table does not have a site field. It is designed to be replicated throughout the site tree, but can only be added to at one site (usually the primary site) - unless it has no unique keys (other than the GUIDKey).
2. The Site-related table has a site field (which should be added to unique keys (other than the GUIDKey) - in order to preserve their uniqueness), which will allow that table to be added to at each site. The replication of those records are limited to the parent (and grand-parent) of that site (or a particular range of sites if desired).
3. The Global-site table is a combination of 1 and 2. It allows replication of the record throughout the site tree, and allows each site to add records to the table (containing unique keys other than the GUID). Add the site field to this table, but call the site field something else (like Origin). Treat the Origin field (in your programming and dictionary) exactly like you would for the site field. Replicate will not recognize the Origin field as a site field, and so it will not impose site limitations on these records when they are replicated.

If you omit a Site field from a table, then you must limit where this file can be added to, or else Auto-numbering keys (or any other unique keys) will clash - and 2 sites will contain two different records (albeit with the same unique key) - which cannot be replicated. In the above example, you should limit Product table inserts to the Head Office.

If we decide that all the branches should have access to all the customers, then we must either remove the SiteField from this table (allowing insertions only at the primary site), disable Subset Replication or rename the SiteField to 'Origin'.

2.4.I cannot get my records (based on an ID field) to auto-increment. How do I do this?

Answer: You probably need to Initialize the Site field in your table.

2.6. I have one table definition and have multiple files which use this table definition (switch filenames at runtime).

Answer: You can't use Replicate in this scenario. You need to change your file structure in order to have one file per table declaration. You can have Superfiles (i.e. multiple tables in one file) though.



3.1. I want to license the amount of sites Replicate will replicate to. 

Answer: This is a little tricky, because it's difficult to know exactly how many sites you are replicating to (especially with subset replication). The best way (and probably the only way) is to:

1. Limit the addition of sites to the Primary Site (disable the Site insert button if ThisRep.Site <> ThisRep.ParentSite or ThisRep.ParentSite = ''). 
2. Then to create a filter (for your license counter) based on the number of actual sites (not relationships). 

The easiest way to do this would be: SIT:ThisSite = ThisRep.site and make the SIT:ThisSite field in the SiteFile insert form READONLY. The problem with this method is that you will only be able to add children to the primary site - you will not be able to add children to a site that is not the primary site. 


3.3. How do I encrypt my logfiles?

Answer:  Simply check the Use Capesoft Encryption on the LogManager Setup  tab and enter an Encryption key in the field provided below, and your logfiles will be encrypted. Encryption occurs after compression (if compression is used). This optimizes compression.

3.4. Is it possible to trigger replication manually everytime I update something in the Parent Site?

Answer: Yes. You need a communication mechanism (like NetTalk to inform the relating LogManagers that they need to ProcessLogFiles). This is outside the scope of Replicate, but is very possible with NetTalk or other such communication tools.


3.6. What is subset data replication, and how do I institute it?

Answer: To institute Subset Data Replication, you need:

1. a SiteHi field in your SiteTable.
2. to define the SiteField in your tables.
3. to set a Site field identifier in your Global Extension template.

Note: Changes to files without the Site field will be propagated to all the children (as there is no site field to institute the limit on).

Note: If a Site's SiteHi field is clear, then a complete log file will be propagated to it. If SiteHi = Site, then a subset of data will be made containing only the changes that pertain to that site.

Conversely, if you do not want subset replication (i.e. you want to send the whole log file to all the children), you simply clear the Site field identifier in your Global Extension template - or omit the Site Range fields in the SiteFile tab of the Replicate global extension template (SiteHi and SiteLow).


3.7. How do I institute one-way Replication?
Answer: One-way Replication (on a file-level) is often more of an issue of authorization. Remember we want all changes to be propagated throughout (with subset filtering) in order to ensure that all the sites are mirrored. What you may actually be wanting is to prevent B100- B400 from editing the product information. A good way of doing this is disabling the update buttons on the product browse, and only enable them if this is the Primary Site. Example code to do this (after opening the window):

if (ThisRep.ParentSite = ThisRep.Site) or (ThisRep.ParentSite = '')
  do EnablePrimaryControls
end

If you are sure that one-way replication is what you're after, then One-way Replication by Site can be set-up fairly easily. Add a DontSendFiles flag to your SiteFile (in your dictionary), and set the field up on the SiteFile tab in the Replicate Global Extension template (of your LogManager application). Set the flag at the site from which you don't want to send files in the record where ThisSite = that site and RelatingSite = the site we don't want to send to using the Site update form in the LogManager at runtime. 

If you don't want any of your changes to be included into the logfile, then you can set the NoLogging property in your ReplicateObject.Init method (after the parent call) in your LogManager application:

if self.site = self.parentsite or self.parentsite = ''
  self.SetGlobalSetting('NoLogging',1)
end

3.12 The SendEmail routine is never called. What am I doing wrong?

Answer: 

1. Check the Send Email Procedure is correct on the SiteSetup tab of the Replicate Global Extension template.
2. Check the presence of the EmailAddress field  on the Site File tab of the Replicate Global Extension template.
3. Check that each relating site has (in fact) got an email address in the EmailAddress field in the site file. If there is no email address for a site, then the logfiles will not be emailed.

3.14. I have FM2/3 added to my application, should I add it to the LogManager application as well?

Answer: You must add FM2 to your LogManager program. 



3.15. How do I get replicate to do a dial up connection?

Answer: 

1. Create the DUN window (as per the NetTalk docs).
2. Call the Dial-up window from your ProcessLogFiles button (on your Browse Sites window) and remove the code template that is there.
3. In the 'ConnectionsChanged' method, place the 'Process incoming and outgoing logfiles' code template (the one you removed from the Browse Sites).
4. place a close window command at the end of the source code generated by the above code template.

3.16. How do I disabling the warning messages from appearing?

Answer: On the Replicate Global Extension template's Site tab, you'll find a checkbox "Suppress Warnings" - check this checkbox.


3.20. I would like to suppress ALIASes when doing a full export.

Answer: The best way of doing this is to skip the insert method before it writes to the file.

In the LogManager's Global Embed: Replicate | ThisRep | Insert | Before the Parent Call:

if lower(FileLabel) = 'aliaslabel' and self.FullExport
return
end

You need to also create the FullExport property (Other Properties embed point) and set it immediately before doing the fullexport and clear it immediately afterwards.

What you could do in the CRCCheck, is also create a property (say self.PerformingCRC), derive the FileCRC method, and set it as you go in, and clear as you come out.
 
In your OpenFiles method,
 
  if self.PerformingCRC
    case clip(FileLabel)
    of 'AliasTable1'
    orof 'AliasTable2'
      return (1)
    end
  end

3.21. How do I make that replication occur less often to one site than to the other sites?

Answer: 

1. Add 3 fields to your sitefile: TimeBetweenTransactions, LastTransactionTime and LastTransactionDate.
2. There's an embed point where you can write code to skip the processing. In your global embeds:
Replicate | ThisRep | AutoProcess | 3a) In loop - before processing

if SIT:TimeBetweenTransactions > 0
  if (((today() - SIT:LastTransactionDate) * 8640000) + (clock() - SIT:LastTransactionTime)) > SITLastTransactionTime
    exit
  end
end


3.22. I want to customise which tables are replicated. How do I do this?

Answer: You can derive (and in fact override) the Registration method. The tables (that are not suppressed) will be registered in this method. You can manual code the file registering before the template generated code and return before the template generated code is called.

For Example:

self.Register(Company,'Company',COM:record,COM:Guid,RepHCOM:Record,COM:Site)
self.RegisterArray('COM:Phone',RepCOM:Phone,RepHCOM:Phone,Rep_TableIsThreaded,'Company')
self.Register(Contacts,'Contacts',CON:record,CON:Guid,RepHCON:Record,CON:Site)
self.RegisterArray('CON:Phone',RepCON:Phone,RepHCON:Phone,Rep_TableIsThreaded,'Contacts')
self.RegisterArray('CON:TestDate',RepCON:TestDate,RepHCON:TestDate,Rep_TableIsThreaded,'Contacts')
self.RegisterArray('CON:TestTime',RepCON:TestTime,RepHCON:TestTime,Rep_TableIsThreaded,'Contacts')
if self.ParentSite = self.site        !Only handle table changes if this is the parent site.
 
self.Register(NotesFile,'NotesFile',NOS:record,NOS:Guid,RepHNOS:Record)
end 
return


3.24. Processes constantly, generating a request to server or client at least every second.

Answer: Check your TimeToProcess variable. This is moved to a global variable: RepGLO:TimeBetweenTransactions. Set the initial value to the time you would prefer. There is an option on the Replication ControlWindow to adjust this, but you may have deleted this control on purpose (to prevent tampering by users).


3.25. I selected "Direct Only" when I initially made my LogManager. Now I need to add FTP/Email but I can't find any of the template prompts anywhere?

Answer: I suggest you remake your LogManager including Email and FTP support. The direct only one is really for people who don't own (or don't want to include) NetTalk in their LogManager. This basically excludes all NetTalk code. However to get it all setup correctly, making a new LogManager from scratch is normally the way to go.


3.26 I get an Access Denied message on one of the logfiles during import. How do I get around this?

Answer: If you turn on silent (in the Global Extension Template), then this message will not be displayed - and the logfile will simply be skipped and will continue processing logfiles from the other sites. The next time that the LogManager does a process, the logfile will be processed. If you still want to display the Replicate windows, but only want the decompression to be on silent, then:

ThisZlib.silent = 1

is the property to set (the template will set it in the ThisRep.init method).



4.1. I'm trying to implement the Site range, but I am not getting all the records passed to my relating site.

Answer: You need to make sure that you have the Site range variables are filled in in the Replicate global extension template. If the Site range low is always the same as the Site, then you can omit then SIT:SiteLow variable. It is (however) a good idea to fill it in anyway, because at some stage you may want to have a mirror site (with the same data range as the parent site), and the site-related records < site will be omitted if this is not filled in.


4.2. How do I start the Replication process (i.e. what must I do to start the automatic export and import)?

Answer: Your own site record (in the SiteTable) will be automatically added when you first run either your own program or the LogManager program. The SIT:ThisSite (in your SiteTable) will be contain ThisRep.Site (your own site) for the records (in the SiteTable) that pertain to your site. 

The best way to introduce a new site to the site tree, is to create a child site from the parent. This is done in the LogManager and there is a routine that comes in the created LogManager that will do this for you. This will create a brand new child site and setup the connection between it and the parent. The only thing you will need to do, is make sure that the INI file that is created is placed in the correct place for the new site's logmanager to pick up it's connection details to the parent.

Alternatively, if you have 2 existing sites, that are independant from each other and need to start a relationship, then there are 2 ways to introduce a new site into the Site Tree:

1. Copy it's log files to the incoming directory of it's parent. You can then edit the site record (that was automatically added to the parent's site) giving the parent the correct details of the new site. The parent will then begin exporting it's logfiles to the new site's mailbox/incoming directory. The new site will then follow the same procedure as it sees a log file from a site that is not in it's SiteTable.
2. Alternatively, you can add a record to the parent site's (or the new site's) site table, ensuring that the mailbox/incoming directory details are correct, and the next time the site does an export, it will export it's log files to the new site. The new site (upon receiving the log files) will add a new site record to it's site table. You must ensure that the mailbox and/or incoming directory of the parent's site is correct.
 

4.4. I'm doing a complete import, but not all my (for example) customers are being imported.

Answer: Replicate is designed this way. You will see that the records that are imported into the Company table only pertain to that site (or in that Site range). It throws away records that don't belong to that Site (or it's children). If you add records to a table without a Site field limiter (such as the notes table), and import those, you will see that those are completely imported because there's no SiteField in the table, so  Replicate knows that it must import all of those records.


4.5. I'm trying to run the examples, but I cannot find any files called abcmanb000.exe etc.

Answer: When you run the install program, you need to check the 'Install examples' check box (which is a checkbox on the Install Options window).


4.6. I can't get the replication working. What am I doing wrong?
Check out the Replicate Error Flow Diagram first.

Answer: Somewhere, the transporting of the files is being broken down. Transporting the logfiles is basically made up of the following steps:

1. (optional - Indirect method) The logfiles are fetched (from the mailbox/FTP directory or whatever) and placed in the Incoming directory.
2. (optional - compression) The logfiles are decompressed.
3. The logfiles are then moved to the LogPath.
4. The Logfiles are then imported.
5. The LogManager then copies the Logfiles out of the LoggingPath and into the Outgoing directory. This is to prevent your programs from adding to the Logfiles while the LogManager is exporting.
6. (optional - subset rep.) The LogManager then creates a logfiles for each related site (this is the subset replication part) and places it in the other site InDir.
7. (optional - compression) Each logfile is then compressed into the z file.
8. (optional - subset rep.) The subset logfiles are then deleted.
9. (optional - Indirect method) The logfiles (or zipped logfiles) will then be transported 

Things to check:

1. This is where most people struggle to get Replication going.

There's a common collection-distribution point for the sending site to the receiving Site. If you are using the Indirect Method, then this means a mailbox, or if you're using the Direct Method then you must ensure that the directories that the log files are being copied to is the incoming directory of the other site. It's a good idea to place a string indicating the relevant site information (like the incoming, outgoing, site ID, parent ID and Site Hi range), which will make it easy to check these links.

Think of the SiteTable as a Site-address book. Each site has it's own address in it, as well as all it's relating site's addresses in it. The detail that that site needs is where to send the logfiles to the relating site - this is via email, FTP and/or Direct - and these details need to be there in order that the site knows where to put the logfiles that it is Sending to the relating site.

If you're wanting to send something to one of your clients, you send it to their address/fax machine/email - each client has a single address/fax machine/email. Similarly with sites, each site has it's own incoming email address/FTP directory/Incoming directory. All logfiles that it needs to import/receive will be at its incoming email address/FTP directory/Incoming directory. It will send logfiles to however many relating sites it has - to those site's email address/FTP directory/Incoming directory.

2. You may be doing subset replication, so your record inserts into a parent site have a site field that are out of the site range to be exported to the child site.
3. If you are doing subset replication, and you are compressing the files, then it may be that your OutGoing directory is the same as the InDir for the relating site. In this case, the Processing of the logfiles (Step 6 - creating the LogFile subsets) is aborted and so the logfiles are not compressed.

Still struggling?

OK - crucial thing is that you track the logfiles' movement. What I would do initially is to turn off compression and encryption (on the Advanced tab of your Replicate Global Extension Template in your LM). Stick to running 2 LMs on the same machine so you don't have to fight with file access rights just yet.
 
Stick to Direct transport at this stage - we'll move on to FTP (if you want) at a later stage.
 
Run both LMs and click the pause button.
 
Step 1:
Make a change in the child site, click process on the child's LM - check the OutGoing directory, then the OutGoing\ParentSite directory - any files in either of these?
 
Yes? Then you've got the parent's incoming dir wrong at the child's site. Back to step 1.
Nope? Move to the parent's incoming directory - see the logfile there? Cool.
 
Step 2:
Open the logfile (that's in the parent's incoming directory) and make sure your change is in the logfile - in there?
 
No? OK - you've made a change that's not replicatable to your parent (either outside the parent's site range - or else part of a table that's not replicated). Go back to step 1 and make a change to a table that is not site limited and that you know is replicated.
Yes? Cool.
 
Step 3:
Click Process at the parent LM.
 
Did it process the logfile? (Check in the incoming directory - is the logfile gone)
No? Then we've missed a logfile somewhere along the way. Click process on both LMs a couple of times to get them back in sync again.
Yes? Check your changes are brought across.

 



4.7. I want to create a mirror-site of one of my sites. How do I do this?

Answer: If you want a child to have complete replication, then you need to set the SiteHi range field in the SiteTable to blank. You must also set the SiteHi property (on the child side) to blank.


4.9. I want to synchronize some sites via the local network (i.e. using the direct method) and some via an Email/FTP connection.

Answer: You must use the ConnectionManager. For the sites that are in contact directly (across a LAN), omit the SIT:EmailAddress from the Site file and point the SIT:DirectInDir directly to the Incoming directory of the relating site. In this way it will act like a LogManager.
 

4.11. I get 0Byte logfiles (with a tmp suffix) in FTP directories. What is the problem?

Answer: It is probably an issue with your Firewall or DNS server. Check that your Firewall is letting through both commands and data (FTP has 2 channels) - and data should be unlimited as there could be a packet size limiter on the data channel (which is the channel in which all the logfiles are uploaded on).


4.13. I create a new site and a huge logfile is sent to the child, and the child logs all these new inserts and sends it back to the parent. Why?

Answer: Basically the new site is verifying the inserts that it has performed with the parent. This logfile forms a backup with which you can restore from. 

However, if you have your own backup system, then you won't require this large initial log file, in which case you can switch logging off at the new child site for this initial logfile. You do this in the Auto Create a Child Site control template settings in the AutoCreateSite procedure of your LogManager. Clear the Log full data import at child site checkbox.

If you have your own method of creating an initial dataset (i.e. copying directly, etc), and don't require the complete log of the dataset on child creation, then in the same place, you can clear the Perform full data export for the new child site checkbox to disable the complete log file creation.


4.12. All my logfiles (from 1 to x) are being sent over to my new child site. How can this be rectified?

Answer: You need to replicate your site table in order to get the correct logfile number to the new child when it is created, from where it should begin importing. This is done in the SiteFile tab of your Replicate Global Extension Template in your LogManager.


4.14. I have a number of logfiles in the incoming directory of a site - even after processing at that site.

Answer: If you are left with a number of logfiles in your incoming directory, then this indicates that that site (call it B100) is missing a logfile (or a number of logfiles) from the site that is sending those logfiles (check the first 4 digits of the filenames for the site ID - let's say B110). B100 will send a request to B110 requesting those missing logfiles. B110 will then send the missing logfiles and B100 will then import all the files in the incoming directory. If the request from B100 to B110 is not getting to B110, then it will not know to send the missing logfiles. At B100, look in the log\outgoing\B110 folder and see if there are a number of files waiting to be sent. If so, then this is indicative that there is a problem in the transport of the logfiles from B100 to B110.


4.15. I have a number of logfiles in the outgoing folders of a site - even after processing at that site.

Answer: This is indicative that there is a problem in the transport of the logfiles from the current site (lets say B100) to the relating site (lets say B110). If your are receiving logfiles from the relating site in the incoming directory (check after processing B110 if there are any B110xxxx.z or.ze files there) - then you probably have a setup issue at B100. Check the record B100/B110. For more details, check the JumpStart tutorial to get 2 sites Replicating section of the docs..


5.2. I'm having new files created and often my log files are not logging my file changes.

Answer: You could be using the Filedialog command, which could be causing your data path to change. If you use the Filedialog command, then you need to  set the necessary attribute that saves and restores your path.



5.3. In one procedure, my changes appear in the logfile, but not in the database.

Answer: You may need to clear the GUID field. If the GUID field is populated on an Insert, then the GUIDKey will have a duplicate error and the record will not be inserted. Simply add the following code before your add command:

clear(<FilePrefix>:GUID)


5.4. My entries to a file are not being logged, and neither is the GUID field in that file being populated (i.e. it's blank).

Answer: This means either that that file has been suppressed (if this is a multi-DLL, then you need to check your DataDLL and your current app for the suppression), or that Replicate itself has been disabled in either the Data-DLL (for multi-DLL applications) or the main exe. If you are using a legacy application, then you need to call the GetGUID method before your add command (for suppressed files):

PRE:GUID = GetGUID()
add(Filename)

5.6. I have a process whose file changes are not being logged. What must I do?

Answer: You are probably using Clarion5.5, where there is a bug in the FileCallback PUT(View) (Clarion limitation). The best is to either upgrade to Clarion 6 or don't use the Process template.


5.7. I'm getting really large, bloated logfiles - and they seem to be growing exponentially.

Answer: Replicate does have some overhead for the HTML tags, so it could be that you have 40 bytes that are changing (from a handful of different table). Each byte change requires a field header tag at the start and end of the tag. You could have very long field names. Each record entry change, requires a time (and sometimes date) stamp, as well as a site stamp, a GUID, the table identifier, possibly the user and machine (if these are set). This can relate to quite a large amount of overhead - in terms of logfile size.

The other thing that could be happening is that your users are often forcing a LogManager close in the middle of a logfile process.  Check out the Aborting in the middle of the ProcessLogFiles routine in the Useful tips section of the docs as to why and how to work around this problem.


5.8. How does Replicate handle logout/commit/rollback?

Answer: During logout, Replicate logs file changes writes to a temporary logfile. Upon commit, it will write the entire contents of the temporary logfile into the real logfile. Upon rollback it simply deletes the temporary logfile.



6.1. I am getting corrupt records when saving from a form. What am I doing wrong?

Answer: You are probably calling a routine (like a process or something) that accesses the file that is being updated from the form. The record and saved record variables are being populated with other values. You need to save and reload the variables on either side of the call to the routine as follows: 
      LocHSAL:Record = RepHSAL:Record
      LocSAL:Record = SAL:Record
      ProcessSalesDetails()
      RepHSAL:Record = LocHSAL:Record
      SAL:Record = LocSAL:Record

Note that you need to save both the History of the record and the Record itself in order for Replicate to log the changes correctly.

6.2. How does Replicate handle concurrency checking/conflict resolution?

Answer: Scenario:

If Joe changes Bruce's mailing address to 1 main street and an hour later, Mary changes the address to 11 main street - and BOTH Joe and Mary are and should be authorized to make this change - what happens?

On your multi-user LAN system today.

The only difference of course is that in a Replicate program Joe saw 2 when he changed it to 1, and Mary saw 2 when she changed it to 11. In a live case Mary sees 1 when she changes it to 11. Question is - since Mary and Joe both think they have the _right_ number, they're going to enter it _regardless_ of the existing value.

Thus, although the question seems to suddenly be important (in a replicated enviroment) the truth is that it behaves _exactly_ the same as your LAN program has always behaved. It assumes that people entering data are entering correct data.

And if you start with the assumption that users are entering _incorrect_ data then you're losing before you begin...

We actually looked at it in some depth at the start of the design process, and to be honest there is no good way to do conflict resolution, even when you detect that this situation occurred. Consider;

We'll make a log - give the log to a conflict-manager, and he must resolve it.

This is completely unworkable in the real world, and will be completely ignored. Imagine if you had 5 sites (which by their nature are geographically disparate.) Mark is the conflict manager (like he doesn't have enough to do already !!). So he gets a report saying "Joe says 1, but mary says 11". So Mark phones Joe:

M : "hello Joe, Mark here"
J : "Mark - I don't know no stinkin Mark... "
M : "Mark du Stud, Head Office, Data Conflict Resolution Manager"
J : "Yeah, so what..."
M : "Well you entered Bruce's Mailing Address as 1 main street"
J : "Yeah so what..."
M : "Well is that the correct value? Mary says it's 11"
J : "Of course mine is right... why would I enter a wrong value... get lost you poncy git..."

So Mark phones Mary. Guess what the outcome there is?

So what does Mark do? Most likely he just picks one. Say he get's 10 of these a day. By Friday guess what he thinks? He thinks _your_ program is a load of trash. It's made his life much harder, not easier. Besides what actually is having him do this achieving?

Not having a comflict manager, and refering the conflict back to the participants (Joe and Mary) is even more useless. All that does is make everyone think the program is a waste of time, and doesn't actually lead to any better data.

Plus it's such a limited case. What happens if Joe changes it on Monday, it gets replicated, and Mary changes it on Wednesday. You have _exactly_ the same situation, but the computer thinks this is all hunky dory. As it should. And indeed this is what we expect from our computers.

Now there may well be situations where the _from_ value matters, but most of the time the _from_ value is simply ignored. The person is capturing the data _because_ the from value is wrong. I don't look at the address and say "well 1 is almost 11, I won't enter 11". It doesn't matter if the from value is blank, 1 or 100. It 'aint 11. If I think 11 is the right value, then 11 I will enter. As programmers we assume people are thinking about what they're doing, and that the value they type in will somehow be different based on the from value. Of course in 99% of cases this is a myth.

Hence to answer your original question - what conflict?

6.3. How does Replicate handle updates to the same record at 2 different sites between synchronisations?

Answer: Replicate handles field level changes, so if a user changes the phone number at one site and at another the user changes the address, then these field-level changes will both be replicated correctly. If both operators change the field to the same value then both changes will be ironed out. If both operators change the same field of the same record to different values then read the question on conflict resolution.

6.4. I am still not sure how the Hi & Low limit works.

Answer: Low Limit is taken as your SiteID (or SiteLow field if you have this as a field in your file) and the Hi limit is an optional limit that you can set. This means that (if you look at the SiteTree) - changes that are propagated up from site B300 to B000, will not be propagated down the B100 branch for Site Identified records (if the Hi Limit is set to B1ZZ).

How it works: When you're running the ProcessLogFiles routine, it first imports all the files that are in your incoming directory. It then loops through your Site Table (for all sites, where SIT:Site = ThisRep.site) and exports the log files to the relating SIT:FromSite s' incoming directories. While it's doing this