An automatic failure notification mechanism for use in a computer network wherein several co-operating parts of an application program are each running on a different node of the computer network. The mechanism comprises a set of linked subroutines called by each part of the application program and operating through the use of a distributed lock manager to designate one part as a part to receive failure notification and to link and reverse link the selected part and the other parts of the application program. The failure notification mechanism utilizes the link and reverse link to initiate a failure communication upon the failure of a node executing one part of the application program.
The monitoring of proper network operation is an important yet time consuming function typically carried out by network managers and operators. Upon a CPU crash, the network manager must ascertain if the CPU was executing a co-operating part of an application program, identify the application program and the part running on the failed CPU and then take appropriate steps to restart the application program. Accordingly, there is a need for an automatic, reliable mechanism to detect a failure among network components running co-operating parts of an application program and to provide efficient, informative notification of the failure.
SUMMARY OF THE INVENTION
The present invention provides an automatic fail-over notification mechanism that is largely transparent to application developers and easily implemented by inserting a call subroutine instruction in each part of an application program. The mechanism comprises a set of linked subroutines called by each part of the application program and operating through the use of a distributed lock manager to link and reverse link the various parts of the distributed application program. The fail-over notification mechanism utilizes the links and reverse links to initiate a failure communication upon the crash of a CPU executing one part of the application program. The failure communication contains information that is sufficient to initiate an automatic recovery from the failure without the need of manual intervention by a network operator or manager.
A distributed lock manager, as for example the VMS Distributed Lock Manager marketed by DIGITAL EQUIPMENT CORPORATION, is a mechanism for coordinating network wide access to files and their individual records and to synchronize interprocess events across the entire network. Generally, the lock mechanism permits a program developer to name each resource on the network, whether physical or logical, and to require an application program running on a CPU (i.e. a process) to request a lock for that resource prior to acess to the named resource. For example, a database or a subset thereof can be a resource and a lock can be in one of several lock modes recognized by the lock manager. Lock modes can include various exclusive or shared write and read access privileges to the resource to share access with other processes or prevent access by other processes.
The lock manager also maintains granted and waiting queues for lock requests for each named resource and services waiting requests in a first come, first served manner. A lock request is placed in a granted queue when the request is granted and is placed in a waiting queue when the lock request is incompatible with an already granted lock request. Each process running in a network can enqueue and dequeue (release) lock requests from a lock request queue and, further, specify an asynchronous system trap in the lock request.
An asynchronous system trap is a routine that interrupts a running process upon the occurrence of an event and then executes. In the lock mechanism, the events can include the granting of a lock request or the making of a lock request that is incompatible with an already granted lock request. When an asynchronous system trap is invoked by the granting of a lock request it is referred to as a completion AST. When an asynchronous system trap is invoked by the making of a lock request that is incompatible with an already granted lock request it is referred to as a blocking AST. A process that specifies an AST with its lock request will be interrupted by the routine relating to the specified AS when the lock request is granted in the case of a completion AST or when another process makes a request for a lock mode that is incompatible with the lock mode granted to the process in the case of a blocking AST.
In addition, a lock manager can include a communication facility which may comprise a lock value block associated with the lock queues for the resource name. A lock value block may contain, e.g., 16 bytes of information which can be input and/or read by a process through the enqueuing and dequeuing of lock requests. The information can comprise interprocess messages.
The present invention provides a fail-over notification mechanism by naming certain logical resources relating to processes that constitute the parts of a distributed application program and requiring a certain set of ordered compatible and incompatible lock requests for those resources to link and reverse link the processes through the lock request queues. The present invention also specifies completion and blocking asynchronous system traps in the lock requests to cause the issuance of reverse linking lock requests and provide failure notification upon the crash of a CPU running one of the parts of the application program.
The logical resources include a "SPECIAL" resource, a "BROADCAST" resource, a "PERMISSION.sub.-- TO.sub.-- TALK" resource, and a resource uniquely named for each one of the parts of the application program. The SPECIAL resource is the process selected to receive failure notification whenever one of the other parts of the application fails and to take corrective action. The BROADCAST resource utilizes the communication facility of the lock manager to advise the SPECIAL resource when a part of the application program has entered or leaves the network. The PERMISSION.sub.-- TO.sub.-- TALK resource operates in conjunction with the BROADCAST resource to assure transmission of messages between processes. Finally, each process executing a portion of the application program names a resource that uniquely identifies the part, e.g., the resource name can be based upon the nodename where the process is located within the network.
The fail-over mechanism is activated by a single subroutine call instruction that is inserted in each part of an application program by a program developer. The subroutine call instruction calls a SETUP routine. The SETUP routine makes all of the lock requests and associated asynchronous system traps calls necessary to setup the fail-over mechanism. The use of the SETUP routine, which is resident in the network and available to all application programs, makes the fail-over mechanism transparent to the application developer. The SETUP routine makes three lock requests on behalf of the process which called it, as follows:
1. A request for a lock on the BROADCAST resource in a CONCURRENT READ lock mode (the SETUP routine will wait for the granting of this lock request);
2. A request for a lock on the resource named after the the specific part of the application program, called herein a SERVER.sub.-- xxx resource (wherein x uniquely identifies the part of the application program constituting the calling process) in a PROTECTED WRITE lock mode (The SETUP routine will wait for the granting of this lock request); and
3. A request for a lock on the SPECIAL resource in an EXCLUSIVE lock mode;
The lock request for the SPECIAL resource in EXCLUSIVE specifies a completion AST called SPECIAL.sub.-- AST. since the request is for an EXCLUSIVE lock mode, only one of the processes of the application program, usually the first to make the request, will be granted the lock request. The other processes of the application program will have their respective lock requests placed in a LOCK REQUEST FOR SPECIAL IN EXCLUSIVE MODE WAITING queue. The process granted the EXCLUSIVE mode request is then interrupted by the SPECIAL.sub.-- AST completion routine which makes that process assume responsibility for receiving and acting upon a failure notification (the process granted the EXCLUSIVE lock is referred to as the special server).
More specifically, the SPECIAL.sub.-- AST utilizes the lock manager to obtain a list of all of the processes in the LOCK REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue. As should be understood, the list derived from the LOCK REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue corresponds to all of the other processes of the application program. Thus, the list provides a forward link between the process assigned responsibility for receiving failure notification and the other processes of the application program through the LOCK REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue.
The SPECIAL.sub.-- AST will then issue a lock request in EXCLUSIVE for each of the SERVER.sub.-- xxx resources corresponding to the processes in the waiting queue list. Since the lock request in EXCLUSIVE is incompatible with the PROTECTED WRITE locks granted to the listed processes corresponding to the SERVER.sub.-- xxx resources, each of the lock requests made by the SPECIAL.sub.-- AST on behalf of the special server will be placed in respective LOCK REQUEST FOR SERVER.sub.-- xxxx IN EXCLUSIVE WAITING queues, one for each process in the list. The lock request in EXCLUSIVE for each SERVER.sub.-- xxx will also specify a completion AST called a FAIL.sub.-- AST.
Thus, the exclusive lock requests made on behalf of the special server by the SPECIAL.sub.-- AST provides reverse links between the special server and all of the other parts of the application program through the waiting queues of the SERVER.sub.-- xxx resources.
Whenever a particular CPU fails, the granted lock for the SERVER.sub.xxx of the process running on that CPU will be removed form the granted queue by the lock manager and the EXCLUSIVE lock request made by the special server will then be granted. Accordingly, the FAIL.sub.-- AST completion routine will interrupt the running of the process on the special server. The FAIL.sub.-- AST runs a routine to identify which process has failed through the use of the unique name of the SERVER.sub.-- xxx resource for which the special server has been granted the EXCLUSIVE lock and can, for example, either send a message to a specified mailbox or call a routine prepared by the developer of the application program to automatically handle a failure of a part of the program.
The use of forward and reverse links through the lock manager queues with the SPECIAL.sub.-- AST and FAIL.sub.-- AST completion routines provides an automatic failure notification mechanism which is conveniently implemented by the insertion of a single call subroutine instruction in each of the several parts of the application program.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a block diagram of a computer network.
FIG. 2 is a block diagram of the computer network of FIG. 1 including an illustration of a program running on several components of the computer network.
FIG. 3 is a detail of a job record stored in the disk file of the computer network illustrated in FIG. 2.
FIG. 4 is a flow chart of a SETUP routine according to the present invention.
FIG. 5 is a flow chart of a SPECIAL AST routine according to the present invention.
FIG. 6 is a logical block diagram of several parts of an application program linked and reverse linked to one another through lock manager queues according to the present invention.
FIG. 7 is a flow chart of a FAIL.sub.-- AST routine according to the present invention.
FIG. 8 illustrates side-by-side flow charts for the CLUSTER.sub.-- BROADCAST and MSG AST routines according to the present invention.
DETAILED DESCRIPTION
Referring now to FIG. 1, there is illustrated a computer network including several CPU's 10, 11, 12, 13, a user interface 14 and a disk file database 15. The CPU's 10, 11, 12, 13, user interface 14 and disk file 15 are all coupled to one another by a common bus 16. Each of the CPU's 10, 11, 12 and 13 comprises a self-contained data processing unit with its own main memory and can communicate with the other components of the network over the common bus 16. The disk file data base 15 is a shared resource that is accessible to all of the CPU's 10, 11, 12, 13 and the user interface 14.
Referring to FIGS. 2 and 3, the disk file database 15 contains a job record 16 for each application program on the network. As illustrated in FIG. 3, the job record 16 is a data field containing information which identifies the program and indicates such pertinent attributes of he program as, for example, a run program instruction, a request to run program, a CPU field to identify the CPU's 10, 11, 12, 13 where the program is to be run, a distribution list to identify which parts of the program are to be run separately on the various CPU's 10, 11, 12, 13, action to be taken upon the crash of anyone of the CPU's, a data output file location, a mail box location to receive messages and any other information required to run the program on the network.
To run a program, a network operator will utilize the user interface 14 to access the job record 16 for the program in the disk file database 15 and insert a request to run the program into the request to run program field of the job record 16. The network operator will then instruct a program executor 17 on, e.g., CPU 10, to look at the job record 16 of the application program.
The program executor 17 will read the data in the job record 16, including the request to run the program inserted by the network operator, and implement the execution of the application program in accordance with the information contained in the job record 16. For example, the program may comprise a batch data processing program which specifies, in its distribution field that three different parts 18, 19, 20 of the program are to be run concurrently, one on each of the CPU's 11, 12, 13.
Pursuant to a feature of the invention, each of a distributed lock manager 21 and an object library 22 is located on each CPU 10, 11, 12, 13. The object library 22 contains a series of subroutines that can be called by each part 18, 19, 20 of the application program. The subroutines operate through the use of the lock manager 21 to assign notification responsibility to one of the parts 18, 19, 20 (hereinafter referred to as the special server) and form links and reverse links between the special server and the other parts 18, 19, 20 of the application program.
The distributed lock manager 21 can comprise a VMS Distributed Lock Manager which permits a process to name a resource and request a specified "lock" on the resource name. The resource can be any logical or physical resource in the computer network and the lock represents a write and read access privilege to the resource. The specific type of lock is referred to as a lock mode. The various lock modes available on the VMS Distributed Lock Manager are as follows:
__________________________________________________________________________
Lock Mode Mode Name Lock Description
__________________________________________________________________________
NULL MODE (NL)
LCK$K.sub.-- NLMODE
This mode sets a
holder in the name
space and indicates
future interest in
the resource. Later,
the lock can be
converted to a higher
lock mode more
quickly than if the
lock were initiated
in that mode.
CONCURRENT LCK$K.sub.-- CRMODE
This mode allows one
READ (CR) process to read data
from a resource in an
unprotected manner
while other processes
can modify the data.
This mode is typical-
ly used when sharing
processes are only
reading data.
CONCURRENT LCK$K.sub.-- CWMODE
This mode allows one
WRITE (CW) process to write data
in an unprotected
manner while other
processes can
simultaneously write
data to the same
resource.
PROTECTED READ
LCK$K.sub.-- PRMODE
This mode allows
(PR) processes to share
read access to a
resource, but not to
write to the resource
at the same time.
PROTECTED WRITE
LCK$K.sub.-- PWMODE
This mode allows a
(PR) process write access
to the resource, and
allows other process-
es to read from but
not write to the
resource at the same
time. The other
processes must have
concurrent read
access.
EXCLUSIVE (EX)
LCK$K.sub.-- EXMODE
This mode allows
write access to the
resource and prevents
other processes from
reading from or
writing to the
resource. This lock
prevents any other
cooperating process
from taking ownership
until the process
granted the EXCLUSIVE
lock mode releases
the resource.
__________________________________________________________________________
The lock modes provide a facility to coordinate access to a resource among various processes running throughout a computer network. For example, the EXCLUSIVE lock mode makes a process the owner of a named resource since no other process can either read from or write to the resource while it is in the EXCLUSIVE lock mode. On the other hand, a PROTECTED WRITE lock mode is less restrictive because, while only one process can write to the named resource, other processes are free to read from the named resource. The other lock modes are less restrictive to provide varying degrees of concurrent and exclusive read and write access privileges to a named resource.
The lock manager 21 does not actively enforce the lock scheme so that it is important that each process makes an appropriate lock request before attempting to access a resource for proper operation of the lock scheme. Enforcement of the lock scheme is implemented by enforcing a convention by, e.g., always having a process make the appropriate lock request for a named resource and permitting the process to proceed only if the lock request is granted.
For example, if a process wants to read from a database that is a named resource, the process will request a CONCURRENT READ lock on the database. If that database is already subject to a granted EXCLUSIVE lock mode, the granting of the request for CONCURRENT READ will not be compatible with the already granted EXCLUSIVE lock mode for the resource. Thus, the lock manager 21 will not grant the request for CONCURRENT READ.
The following table illustrates the compatibility between the various lock modes of the lock manager 21:
______________________________________
Lock Mode Compatibility
Mode NL CR CW PR PW Ex
______________________________________
NL Yes Yes Yes Yes Yes Yes
CR Yes Yes Yes Yes Yes No
CW Yes Yes Yes No No No
PR Yes Yes No Yes No No
PW Yes Yes No No No No
EX Yes No No No No No
______________________________________
The lock manager 21 interacts with processes running on the various CPU's of the computer network through certain system services made available to the processes, as follows:
______________________________________
Lock Manager Services
Service Name
Meaning Function
______________________________________
$ENQ Enqueue lock request
Requests a lock on
a resource or
converts a resource
lock mode.
$ENQW Enqueue lock request
Requests a lock or
and wait lock mode conver-
sion on a resource,
and waits for the
lock to be granted
or converted.
$GETLKI Get lock information
Requests inform-
ation about the
lock database
$GETLKI does not
wait for the
information to
be returned.
$GETLKIW Get lock information
Requests inform-
and wait ation about the
lock database.
$GETLKIW waits for
the information to
be returned.
$DEQ Release lock request
Performs one of the
(dequeue) following
functions:
* Unlocks granted
locks
* Cancels a queued
lock request
that has not
been granted.
______________________________________
The lock manager 21 maintains a lock database that comprises a plurality of queues containing lock request information. Each named resource will have a series of queues for each lock mode including, for each lock mode, a granted queue that lists the processes granted the lock mode, a waiting queue that lists outstanding requests by processes which have not been granted because the lock mode requested is incompatible with another already granted lock request and a conversion queue that lists lock requests granted at one mode and waiting for conversion to another lock mode.
The lock manager 21 will process each lock request by referring to the lock database to check for compatibility with already granted requests and then proceed to either grant the lock request or place the request in a waiting queue. When a process releases a granted lock mode, the lock manager 21 will look up the waiting queue for that lock mode and then grant the lock mode, on a first come, first served basis, to the next lock request in the waiting queue.
The lock manager 21 also maintains a lock status block for each lock for a resource name to indicate into which queue the lock requests for that lock has been placed. A lock valve block is linked to each lock status block. The lock valve block can comprise, e.g., 16 bytes of information that can be either read or written by the processes granted the lock request for that lock. The lock valve block will be available to a next process when it is granted the lock request for the lock mode of the named resource. Thus, the lock valve block can be utilized as a communication facility to pass 16 byte messages between processes.
Moreover, each lock request can specify a completion AST or a blocking AST to synchronize resource access among the processes running on the computer network. As discussed above, a completion AST will interrupt a process when a lock request made by the process is granted and a blocking AST will interrupt the process when another process makes a lock request that is incompatible with the lock request already granted to the process.
In accordance with a feature of the invention, the object library 22 residing in each of the CPU's 11, 12, 13 contains a set of linked subroutines that can be called directly or indirectly by respective ones of the co-operating parts 18, 19, 20 of the application program running on the CPU's 11, 12, 13, respectively. The subroutines are SETUP, CLUSTER.sub.-- BROADCAST, MSG.sub.-- AST, SPECIAL.sub.-- AST, FAIl.sub.-- AST, LIST.sub.-- SERVER, SHOW.sub.-- SERVER and UPDATE.sub.-- INFO. The subroutines utilize the lock manager 21 to link and reverse link the co-operating application parts 18, 19, 20 by naming logical resources relating to the application parts 18, 19, 20 and making a series of compatible and incompatible lock requests on the named resource to link the parts 18, 19, 20 through the lock queues maintained by the lock manager 21. As discussed above, the logical resources include SPECIAL, BROADCAST, PERMISSION.sub.-- TO.sub.-- TALK, and a SERVER.sub.-- xxx resource corresponding to each part 18, 19, 20 of the application program.
The developer of the application program can utilize the fail-over mechanism of the present invention by inserting a single call subroutine instruction in each part 18, 19, 20 of the application program to call the SETUP routine from the object library 22 in the CPU 11, 12, 13 where the part is running. The call subroutine instruction will specify the SETUP routine and provide an argument to uniquely identify the application program to which the part calling the subroutine belongs. The call subroutine can also provide an argument identifying an existing mail box channel in the network for receiving messages from the fail-over mechanism. A zero indication from the mailbox channel will tell the SETUP routine to open a mailbox channel for the application program. Alternatively, the argument can specify an asynchronous system trap written by the program developer which is to interrupt the application program and run upon a failure notification. The asynchronous system trap can specify corrective actions.
All of the subroutines running on a particular CPU 11, 12, 13 in the network co-operate to provide a common memory space in the main memory of the respective CPU 11, 12, 13 to store information relating to the fail-over mechanism set up for the specific application program. The common memory space is automatically set up by the SETUP routine and is accessible to all of the routines in the object library 22. All of the names of the resources for a specific application program will contain a prefix corresponding to the specific application program, i.e. the prefix passed as an argument in the call to the SETUP routine. The common memory space stores information relating to the part 18, 19, 20 running on the specific CPU 11, 12, 13. The information indicates that the part 18, 19, 20 has called SETUP and is in the fail-over mechanism and, further, whether the part is the special server.
Referring to FIG. 4, there is illustrated a flow chart of the SETUP routine. The call SETUP instruction 100, as indicated, is in each part 18, 19, 20 of the application program such that the routine will be run separately for each part 18, 19, 20. The SETUP routine starts by making a request for the BROADCAST resource in a CONCURRENT READ lock mode 101 on behalf of the part that called the SETUP routine by utilizing the $ENQW service of the lock manager 21. The SETUP routine then waits for the lock mode to be granted 102. The $ENQW service automatically waits for the granting of the lock. The loop illustrated in the flow chart is for illustration purposes to indicate that the routine does not proceed until the lock is granted. The lock will be granted since all of the SETUP routines request CONCURRENT READ which are compatible with one another. The request for the BROADCAST resource will also specify the MSG.sub.-- AST routine which is a blocking AST, as will be described in more detail below.
After the granting of the request for the BROADCAST resource, the SETUP routine then makes a request 103 for a resource which it names in the request, e.g., after the node (i.e. the CPU 11, 12, 13 where the part 18, 19, 20 is running), e.g. SERVER.sub.-- CPU 11 in a PROTECTED WRITE lock mode. The VMS system used in the representative embodiment of the present invention provides a service called $GETSYI which can be called by any process to identify the node in the network where the process is running. Alternatively, the SERVER.sub.-- xxx resource can be named directly after the part 18, 19, 20 which called the SETUP routine. All that is required in that the resource name be unique to the specific part 18, 19, 20 of the application program. The SETUP routine will again use the $ENQW service of the lock manager 21. The SETUP routine then waits 104 for the request for the SERVER.sub.-- xxx to be granted. This request will also be granted since each SETUP routine is requesting a PROTECTED WRITE lock mode on a specific resource named after its corresponding node which is different than the resource names for which a lock is being requested by the other SETUP routines.
The SETUP routine will then make a request through the $ENQ service for the SPECIAL resource in an EXCLUSIVE lock mode 105. The request for the SPECIAL resource specifies the SPECIAL.sub.-- AST which is a completion AST. Since all of the SETUP routines are making a request in EXCLUSIVE, only one of the SETUP routines, usually the first in time to make the request, will be granted the EXCLUSIVE lock mode. The other requests made by the other SETUP routines running on the network will each be incompatible with the granted EXCLUSIVE lock mode. These requests will then be placed in a REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue by the distributed lock manager 21. As indicated, the part 18, 19, 20 granted the EXCLUSIVE lock mode is referred to as the special server and the SETUP routine for that part 18, 19, 20 sets a bit in the respective common memory space to indicate special server status.
After making the request for the SPECIAL resource in an EXCLUSIVE lock mode, the SETUP routine checks the common memory to see if the corresponding part 18, 19, 20 is the special server 106. If the corresponding part 18, 19, 20 is the special server, the SETUP routine returns control to the part 18, 19, 20 (107). If the corresponding part 18, 19, 20 is not the special server, the SETUP routine proceeds to call the CLUSTER BROADCAST routine 108, which will be explained in more detail below together with the related MSG.sub.-- AST blocking routine and the BROADCAST resource.
Since each SETUP routine specifies the SPECIAL.sub.-- AST completion routine in the request for the SPECIAL resource, the one part 18, 19, 20 granted the SPECIAL resource in an EXCLUSIVE lock mode through the called SETUP routine will be interrupted by the SPECIAL.sub.-- AST routine.
Referring now to FIG. 5, there is illustrated a flow chart of the SPECIAL.sub.-- AST completion routine. The SPECIAL.sub.-- AST routine initially utilizes the $GETLKIW service 201 of the lock manager 21 to obtain a list of all processes currently in the REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue. The SPECIAL.sub.-- AST then makes a request 202 for an EXCLUSIVE lock mode through the $ENQW service of the lock manager 21 for each of the SERVER.sub.-- xxx resources corresponding to the processes in the list obtained from the $GETLKIW service. The SPECIAL.sub.-- AST can either refer to the job record 16 or use the $GETSYI service to correlate the part 18, 19, 20 to the node name where the part is running. Thus, SPECIAL.sub.-- AST can formulate the SERVER.sub.-- xxx names of the resources for which it makes the requests for an EXCLUSIVE lock.
Each request for the SERVER.sub.-- xxx will specify the FAIL AST blocking routine. As should be understood, each request for the SERVER.sub.-- xxx resources made by the SPECIAL.sub.-- AST will be incompatible with the PROTECTED WRITE lock mode already granted to each process on the list (See 103, 104 FIG. 4). Thus, the request made by SPECIAL.sub.-- AST on behalf of the special server will each be placed in a respective REQUEST FOR SERVER.sub.-- xxx EXCLUSIVE WAITING queue by the lock manager 21. After making all of the requests 202, the SPECIAL.sub.-- AST will return control to the part 18, 19, 20 made special server 203.
Referring now to FIG. 6, there is illustrated a logical block diagram of the parts 18, 19, 20 of the application program as linked and reverse linked by the running of the SETUP and SPECIAL.sub.-- AST routines. As noted earlier, each of the several SETUP routines called by the parts 18, 19, 20 stores information regarding the corresponding part 18, 19, 20 in a common memory space in the respective CPU 11, 12, 13. The SPECIAL.sub.-- AST routine for the part 18, 19, 20 granted the EXCLUSIVE lock mode for the SPECIAL resource sets a bit in the field of the common memory space associated with the information on the corresponding part 18, 19, 20 to indicate special server status.
Assuming part 18 has been granted the EXCLUSIVE lock mode for the SPECIAL resource, the corresponding lock request made by part 18 through the SETUP routine in the object library 22 resident in the CPU 11, where part 18 is running, will be in the granted queue 51 for the EXCLUSIVE lock mode of the SPECIAL resource maintained by the distributed lock manager 21. The queue listing 51 and the special server status bit link 50 the part 18 to the lock manager 21.
A forward link between the special server and the other parts 19, 20 not granted the EXCLUSIVE lock mode on SPECIAL is provided by the REQUEST FOR SPECIAL IN EXCLUSIVE WAITING queue 53, which is acessible 54 to the special server through the $GETLKI service of the lock manager 21 (See FIG. 5).
A reverse link 55 between the special server and each other part 19, 20 of the app
