Distributed Concurrent Processing

208.4.1.3 Example of Concurrent Agents

The following is an example of how Etere DataMover manages parallel agents; it illustrates all the phases involved in the use of similar agents, from definition to utilization:

1. An action (e.g. 'COPY Upload-Cache to FTP....') has been defined within the Etere Configuration module, setting its behaviour to perform up to 3 concurrent transfers using separate agents:

Etere DataMover 1

As shown in the figure above, three different data processors were created for the actions; they were launched in the following priority order:
#1) If Etere DataMover is not busy (i.e. it's not currently performing a request for the action), it would perform the request itself.
#2) If Etere DataMover is busy, the first DataMover [a]agent will be launched for the request.
#3) If Etere DataMover and the first DataMover [a]agent are busy, the second DataMover [a]gent will be launched to perform the request.

2. At a particular moment, Etere DataMover receives five requests via T-workflow (e.g. 'MM - UploadWeb...'); all these requests were for transferring media files using the same action (e.g. 'COPY Upload-Cache to FTP....') and were queued in the Job queue list:

Etere DataMover 2

As shown in the figure above, requests are not yet in the list of Active Requests since 17 seconds are remaining (as shown in the window's bottom part) for Etere DataMover to start performing the tasks.

3. Since the action's concurrency is '3', Etere DataMover can perform a maximum of 3 concurrent transfers. Since the action is set to use separate agents, it will be concurrently performed by Etere DataMover (e.g. which undertakes the 1st request) and the two DataMover [a]gents (e.g. which undertake the 2nd and 3rd requests):

Etere DataMover 3

4. As requests are completed, those that remain in the queue will be undertaken by free data processors (according to the priorities specified in step 1); in this example, Etere DataMover will launch the 4th request whilst a separate DataMover Agent will launch the 5th -and last- request:

Etere DataMover 4

Etere_DataMover_1
Etere_DataMover_2
Etere_DataMover_3
Etere_DataMover_4