OpenR66-Protocol

We describe here the logic of the protocol.

A simplified picture is shown here:

Waarp Protocol

First we describe the different value that can be found.

UpdatedInfo status

A Request (DbTaskRunner) can have several UpdatedInfo status:

  • UNKNOWN : no particular information on it.
  • NOTUPDATED : in used by other database object when they are taken into account.
  • INTERRUPTED : a request is interrupted but can be rescheduled.
  • TOSUBMIT : a request is proposed to be submited by the Commander (or other database object are supposed to be taken into account).
  • INERROR : a request is in error and can not be submited by the Commander until its status is changed explicitely.
  • RUNNING : a request is currently running.
  • DONE : a request is over and fully done.

Step values

A Request can have several Step values:

  • NOTASK : the request has never started.
  • PRETASK : the request is currently in Pre transfer step.
  • TRANSFERTASK : the request is currently in transfer step.
  • POSTTASK : the request is currently in Post transfer step (valid transfer only).
  • ALLDONETASK : the request is fully finished (UpdatedInfo is in DONE status too).
  • ERRORTASK : the request is currently in the Error step while an error occurs (either in PRE, TRANSFER or POST step).

A Request has two Step values:

  • GlobalStep : the current request step value
  • GlobalLastStep : this is the last valid request step value. When GlobalStep is in ERROR step, GlobalLastStep says in which step it was before entering in error. This information is used to enable restart of the Request from this valid last step.

Detailed information

Each Step values and UpdatedInfo has an ErrorCode detailed information:

  • InitOk : stands for correct initialization of the Request (startup and authentication)
  • PreProcessingOk, TransferOk, PostProcessingOk : stand for correct ending of the specified step
  • CompleteOk : stands for all action are correct (ALLDONE for step and DONE for UpdatedInfo)
  • Running : stands for current Step is in Running status.
  • StoppedTransfer, CanceledTransfer : stand for a Request where an action stopped or canceled the given Request. A Stopped Request can be restart from the current status. A Canceled Request starts from the beginning of the current step. For instance for the Transfer step, Stopped will imply that restart is from the current valid transferred block, while Cancel will imply to restart from the very beginning of the transfer (first block).
  • QueryAlreadyFinished : stands for special code where the Request is in fact remotely already finished and so can be finished locally.
  • Other code specifies different kinds of error (NotKnownHost, Shutdown, RemoteError, ...).

Sequential logic

A request of transfer follows a sequential logic:

  • The request is registered in the database with a TOSUBMIT status.
  • A Request that was in a INTERRUPTED status is changed in a TOSUBMIT status.
  • The Commander get some requests with the TOSUBMIT status and makes them as RUNNING status.
  • The Commander submits those requests as separates ClientRunners.
  • The ClientRunner first checks that the given Request is not a "Self Requested" request, meaning that only requester host can execute a ClientRunner, except if this request was in the POSTTASK step so that Requested Host can finalize the request.
  • The ClientRunner gets the remote requested Host address and tries to open the connection. If a network conection with the given requested host is already opened, this network connection is reused by the new ClientRunner.
  • The connection can use SSL support (different port than non SSL). This is an option of transfer. This option is selected while selecting the Host ID for SSL support for the remote Requested Host. This option is CPU and Memory consuming.
  • Once the network connection is found, a private connection (in memory connection) is opened to enable the multiplexing of this request with other requests on the same network connection. This private connection is attached to a new LocalChannelReference which references the Request, the session, the remote and private connections. A valid LocalChannelReference contains two private connection Ids, one for the local private connection, and one for the remote private connection.
    • A Startup Message is sent to the local private connection to initiate it.
    • The same Startup Message is sent back to the remote private connection to initiate too the relation between them and to instantiate the same LocalChannelReference on the remote host.
  • Once the LocalChannelReference is OK, the Requester host sends an Authent Message in order to authenticate this host.
  • The Requested host sends back its own Authent Message too. In case of Check of IP, the authentication is confirmed if both password and IP are aligned with the local definition of the remote partner.
  • Once authenticated, the Requester Host sends the Request Message.
  • The Requested Host check if the authentication and the request are compatibles, check some constraints (like CPU or connexion limits), check some specific options on the request itself (start, restart, ...) and sends back the validated request to the Requester Host.
    • The Request is in InitOk as detailed information.
  • The validated Request is now running the Pre Task step.
    • Once finished, the Request is in PreProcessingOK status.
  • Now the transmission can start. The sender (which could be either the Requested or the Requester host) launch its own RetrieveRunner. This RetrieveRunner sends to the other host all DataBlock Messages.
  • Each DataBlock Message can include a MD5 control of the packet in it (option of transfer). This option is not mandatory and is CPU consuming.
  • Once all DataBlock are sent, the RetrieveRunner sends an EndTransfer Message to the receiver host.
  • The receiver host executes first its PostProcessing actions.
    • The Request is first in TransferOk status for Receiver.
    • The Request is in PostProcesseingOk status for receiver.
  • The receiver sends back the validated EndTransfer Message.
    • The Request is in TransferOk status for Sender.
  • The Request is now on Finalize way. Sender host executes the Post actions on its turn.
    • The Request is in PostProcesseingOk status for Sender.
  • Once the PostProcessing is over, the RetrieveRunner (Sender) sends to the remote host a EndRequest Message.
    • The remote host sends back the validated EndRequest Message.
    • The Request is now totally finished and its status is CompleteOk ALLDONE on both side.

At each step, an error can occurs and will stop the request, setting its UpdatedInfo to INERROR or INTERRUPTED status. The GlobalStep could be in ERROR status if the ERROR step action is run.