MQSeries
Introduction
MQSeries is an IBM supplied Business
Integration software (Middle ware).
It is a Messaging and Queuing system
provided for program-to-program communication.
Programs within an application communicate
by writing and retrieving application-specific data (messages) to/from queues,
without having a private, dedicated, logical connection to link them.
Messaging means that programs communicate
with each other by sending data in messages and not by calling each other
directly.
Queuing means that programs communicate
through queues. Programs communicating through queues need not be executed
concurrently.
MQSeries is used in a client/server or
distributed environment. Programs belonging to an application can run in one
workstation or in different machines on different platforms.
It allows business applications to
integrate and communicate across a spectrum of desktop and mainframe systems,
overcoming inconsistencies with different network protocols and all major
commercial platforms.
Key benefits of MQSeries:
·
A single, multi-platform API
·
Isolation from communications programming
·
Time-independent processing
·
Assured message delivery, even when
programs or networks fail
·
Robust middleware for high-performance
distributed application integration.
Basics of
MQSeries.
Message
A message consists of two parts.
·
Data that is sent from one program to another.
·
Message descriptor that identifies the message
(message ID) and contains control information.
MQSeries messages are of four types.
·
Datagram - A message containing information
for which no response is expected.
·
Request - A message for which a reply is
requested.
·
Reply - A reply to a request message.
·
Report - A message that describes an event
such as the occurrence of an error.
MQSeries differentiates between persistent
and non-persistent messages.
·
Delivery of persistent messages is assured.
They are written to logs and survive system failures.
·
Non-persistent messages cannot be recovered
after a system restart.
Queue Manager
The heart of MQSeries is the message queue
manager (MQM). A queue manager is a
system program that provides queuing services to applications. It manages
Queues of messages.
·
It provides an application programming
interface so that programs can put messages on, and get messages from, queues.
For example, the MQPUT API puts a message
on a queue to be read by another program using the MQGET API.
·
It provides additional functions so that
administrators can create new queues, alter the properties of existing queues,
and control the operation of the queue manager.
A program may send messages to another
program that runs in the same machine as the queue manager, or to a program
that runs in a remote system, such as a server or a host. The remote system has
its own queue manager with its own queues.
Queues
Queues are objects belonging to a queue
manager. Message queues are used to store messages sent by programs.
·
Local Queue - A queue is local if it is
owned by the queue manager to which the application program is connected.
·
Remote Queue- A queue is remote if it is
owned by a different queue manager.
·
Transmission Queue - A remote queue is
associated with a transmission queue. Transmission queues are local and used as
an intermediate step when sending messages to remote queues.
·
Dynamic queues – They are local queues.
They are often used in conversational applications, to store intermediate
results.
·
Model Queue - A model queue is not a real
queue. It is a collection of attributes that are used when a dynamic queue is created.
·
Alias Queue - Alias queues are not real
queues but definitions. They are used to assign different names to the same
physical queue. This allows multiple programs to work with the same queue,
accessing it under different names and with different definitions.
·
Initiation Queue - An initiation queue is a
local queue to which the queue manager writes a trigger message when certain
conditions are met on another local queue.
·
Reply-To Queue - A request message must
contain the name of the queue into which the responding program must put the
reply message.
·
Dead-Letter Queue - A queue manager must be
able to handle situations when it cannot deliver a message.
Channels
Messages are transmitted via channels. A
channel is a logical communication link. In MQSeries, there are two different
kinds of channels:
·
Message channel - An MCA is a program that transfers messages
from a transmission queue to a communication link or vice versa.
·
MQI channel - A Message Queue Interface
(MQI) channel connects an MQI client to a queue manager in a server machine.
MQI clients don’t have a Queue Manager of their own. An MQI channel is
bidirectional.
Process Definitions.
A Process Definition object defines an
application to a queue manager. For example, it contains the name of the
program to trigger when a message arrives.
Working of
MQSeries.
The queuing system consists of the
following parts:
·
Queue Manager (MQM)
·
Listener
·
Trigger Monitor
·
Channel Initiator
·
Message Channel Agent (MCA)
The working of MQSeries is illustrated in
the following steps.
1) When the application program wants to put a message on a queue
it issues an MQPUT API. This invokes the MQI(Message Queuing Interface).
2) The queue manager checks if the queue referenced in the MQPUT is
local or remote.
3) If it is a remote queue, the message is placed into the
transmission (xmit) queue.
4) Each remote queue must be associated with an transmit queue.
Usually, all messages destined for one remote machine use the same transmit
queue.
a) The queue manager adds a header that contains information from
the remote queue definition, such as destination queue manager name and
destination queue name.
5) Transmission is done via channels. Channels can be started
manually or automatically.
a) To start a channel automatically, the transmit queue must be associated
with a channel initiation queue.
b) The channel initiator is an MQSeries program that must be
running in order to monitor initiation queues.
c) When the channel initiator detects a message in the initiation
queue, it starts the message channel agent (MCA) for the particular channel.
d) This program moves the message over the network to the receiving
end.
e) On the receiving end, a listener program must have been started.
f) The listener monitors a specified port, by default the port
dedicated to MQSeries is 1414.
g) When a message arrives, it starts the message channel agent. The
MCA moves the message into the specified local queue .
h) The program that processes the incoming message can be started
manually or automatically.
i) To start the program automatically, an initiation queue and a
process must be associated with the local queue, and the trigger monitor must
be running.
j) The MCA puts the incoming message into the local queue and a
trigger message into the initiation queue.
k) The initiation queue is
monitored by the trigger monitor. This program invokes the application program
.
l) The application issues an MQGET API to retrieve the message from
the local queue.
MQSeries for AS/400
The AS/400 operating system, OS/400, already
has two queue systems
- Message
queues (MSGQ)
Message queues are used for system and
program messages for the operator.
- Data
queues (DTAQ).
Data queues are used to exchange data
between applications in one or more AS/400 systems, and between an AS/400 and
clients (PCs)
The Reasons for using MQSeries on AS/400
despite using these queuing systems are,
- MQSeries
for AS/400 can store queue data persistently. AS/400 standard data queues
cannot.
- MQSeries
for AS/400 can synchronize transactions and AS/400 databases with
commit/rollback. AS/400 standard data queues cannot.
- MQSeries
enables application programs running on different platforms to communicate
with each other using the Message Queuing Interface (MQI) which is
consistent over all platforms.
- Channel
sequence checking between systems assures that no data is lost (not
implemented for AS/400 standard remote data queues).
- MQSeries
allows remote administration across platforms.
- MQSeries
for AS/400 can browse a message (non-destructive read). AS/400 data queues
cannot.
- MQSeries
enables application programs running on different platforms to communicate
with each other using the Message Queuing Interface (MQI) which is
consistent over all platforms.
MQSeries is not just another AS/400 Queuing
System. These are the key advantages of
MQSeries over AS/400 Data Queues and Message Queues.
MQSeries objects for AS/400.
AS/400 MQ objects are known to the OS/400
operating system as object type *USRSPC
(user space) in the QMQMDATA library.
MQSeries queues, channels, and other
objects are stored in the AS/400
object type dataspace.
MQSeries
Queue Manager.
The AS/400 allows only one queue manager
instance.
To Create a Queue manager one should log in
as security officer (QSECOFR) or as a MQ administrator (MQADM).
The following MQ CL Command creates a Queue
Manager:
CRTMQM MQMNAME(MYQMGR)
The CRTMQM panel allows the entry for
various parameters in the command.
Every Queue manager needs some default
definitions for Queue manager objects.
This is done by calling the program “amqsdef4”
CALL qmqm/amqsdef4
To start the MQM and display its attributes
the following commands are used,
STRMQM
DSPMQM
MQSeries
client
An MQSeries client is an independently
installable component of an MQSeries product.
It allows to run MQSeries applications, by means of a communications protocol, to
interact with one or more MQI servers on other platforms and to connect to
their queue managers.
MQSeries clients do not have a queue
manager of their own. Client machines connect to a queue manager in a server.
This MQM manages the queues for all clients attached to it.
AS/400 only can be only a MQSeries server
and it cannot be a MQSeries client.
The MQSeries Clients communicate with the
Queue Manager in a server machine via a MQI
channel.
A Queue Manager communicates with another Queue Manager via a
Message Channel Agent ( MCA).
Manipulating
MQM objects
In Non AS/400 Systems the Utility RUNMQSC
is used to manipulate the Queue Manager
objects.
In AS/400 System the MQSC commands cannot
be run interactively.
To run the MQM commands interactively
AS/400 has separate set of CL commands.
All the MQ CL commands is provided in the
menu CMDMQM.
GO CMDMQM provides a menu to execute the MQ
CL commands interactively.
The MQSC commands can be run in AS/400
system in batch from a file member in a source file . The CL command used is,
STRMQMMQSC SRCMBR(MYCOMA)
SRCFILE(MYLIB/TST)
Communication
between Queue Managers
Message Channels are used for communication
between queue managers in residing in two separate systems.
Each machine has a queue manager installed
and each queue manager
manages several local queues.
Messages destined for a remote queue
manager are put into a remote queue.
A remote queue is not a real queue; it is
the definition of a local queue in the remote machine. A remote queue is
associated with a transmission (xmit) queue which is a local queue. Usually,
there is one xmit queue for each remote queue manager.
Each xmit queue is associated with a
message channel. The message channels are unidirectional.
Therefore two message channels are to be
defined for bidirectional messaging.
The required objects for connecting two
Queue Managers are,
·
A remote queue definition that mirrors the
local queue in the receiver machine and links to a transmission queue.
·
A transmission queue that holds all
messages destined for the remote system until the channel transmits them (B in
system A and A in system B)
·
A sender channel that gets messages from
the xmit queue and transmits them to the other system using the existing
network (A.TO.B in system A and B.TO.A in system B)
·
A receiver channel that receives messages
and puts them into a local queue (B.TO.A in system A and A.TO.B in system B)
·
A local queue from which the program gets
its messages (Q2 in system A and Q1 in system B)
|
System A
|
System B
|
Queue1
|
DEFINE QREMOTE(Queue1) REPLACE +
RNAME(Queue1) +
RQMNAME(SYSTEMB) +
XMITQ(B) +
DESCR(‘Queue 1 in system B’ )
|
CRTMQMQ QNAME(‘Queue1’) QTYPE(*LCL) REPLACE(*YES)
TEXT(‘Messages from system A’ )
|
Transmit Queue
|
DEFINE QLOCAL(SYSTEMB) REPLACE +
USAGE(xmitq) +
DESCR(‘Xmit Queue’ )
|
CRTMQMQ QNAME(B.TO.A) QTYPE(*LCL)
REPLACE(*YES) +
TEXT(¢ Xmit queue¢
) USAGE(*TMQ)
|
Channel
A to B
|
DEFINE CHANNEL(A.TO.B) +
CHLTYPE(sdr) REPLACE +
TRPTYPE(tcp) CONNAME(9.24.104.116) +
XMITQ(SYSTEMB) +
DESCR(
‘Sender channel from A to B’ )
|
CHLNAME(A.TO.B) CHLTYPE(*RCVR)
REPLACE(*YES)
TRPTYPE(*TCP) TEXT(‘Receiver channel from
A to B’)
|
Queue 2
|
DEFINE QLOCAL(‘ Queue2’) REPLACE +
DESCR(‘ Messages from system B’)
DEFINE QREMOTE(‘Queue2’ ) REPLACE +
|
CRTMQMQ QNAME(‘Queue2’ ) QTYPE(*RMT) REPLACE(*YES) +
TEXT(‘ Queue 2 in system A’) +
RMTQNAME( ‘Queue2’) RMTMQMNAME(‘ SystemA’) +
TMQNAME(B.TO.A)
|
Channel B to A
|
DEFINE CHANNEL(B.TO.A) +
CHLTYPE(rcvr) REPLACE +
TRPTYPE(tcp) +
DESCR(‘Receiver channel from B to A’)
|
CRTMQMCHL CHLNAME(B.TO.A) CHLTYPE(*SDR)
REPLACE(*YES) +
TRPTYPE(*TCP) TEXT(‘Sender channel from B
+
to A’)
CONNAME(WTR5317) TMQNAME(B.TO.A)
|
Start
Communication Manually
- Create
objects either interactively using
menu CMDMQM or in batch by the OS/400 command STRMQMMQSC (This is
done only once to create the objects).
- Start
the listeners and channels. Only the sender channels need to be started
manually in both the systems. The receiver channels are started
automatically.
- The
AS/400 listener is started with the command STRMQMLSR.
- Then
start the applications in both the systems.
The commands used to create and start the
objects in both the systems are,
|
System A
|
System B (AS/400)
|
1.
|
strmqm
SYSTEMA
runmqsc <system.tst> a.a
|
STRMQM
STRMQMMQSC SRCMBR(SYSTEMB) + SRCFILE(MQLIB/QMQSC)
|
2.
|
strmqm SYSTEMA
start runmqlsr -t tcp
runmqsc
start channel(A.TO.B)
Ctrl+C
|
STRMQM
STRMQMLSR
STRMQMCHL CHLNAME(A.TO.B)
|
Start
Channels Automatically
- Channel
Initiator is used to start Channels Automatically.
STRMQMCHLI QNAME(SYSTEM.CHANNEL.INITQ)
- The
Transmit Queue definition to start the channels automatically is
CRTMQMQ QNAME(B.TO.A) QTYPE(*LCL) +
REPLACE(*YES) TEXT(‘Xmit queue’ ) +
TRGENB(*YES) TRGTYPE(*ALL) +
INITQNAME(SYSTEM.CHANNEL.INITQ)
- The
queue manager can trigger the process that starts
the channel program in three ways:
· When
the first message is put into the transmission queue (*FIRST)
· Every
time a message is put into the xmit queue (*ALL)
· When
the queue contains a specified number of messages (*DEPTH)
Start
Applications Automatically
When a message arrives from another system,
the application program that has to process the message can be started manually
from the command line or as part of the startup sequence.
Another way is to let the queue manager
start it when it is needed, that is, using the MQSeries triggering mechanism.
- The
listener program monitors the port assigned to MQSeries, 1414, and invokes
the message channel agent (MCA) when a message arrives.
- The
message is moved into the appropriate local queue (specified in the
message header) and a trigger message is put into an initiation queue.
- The
Trigger monitor monitors the initiation Queue and starts the program. The
program name is specified in the process.
- The
program then gets the message off the queue.
Client
Server Connection
To form a Client Server connection, the
communication protocol and the addresses of the systems to be connected are to be
known.
Definitions in
Server
- Define
the queues that the application needs and a channel of the type
server connection.
- Define
a queue for the application to put messages in .
DEFINE QLOCAL(¢ JAVAQ1¢ ) REPLACE +
DESCR(‘ Queue for Java Application’ )
- An
MQI channel of the type server connection.
DEFINE CHANNEL(‘ JAVACH1 ’) CHLTYPE(SVRCONN) REPLACE +
TRPTYPE(TCP) MCAUSER(‘ ’ )
Definitions in
Client
- Define
an environment variable for the MQSeries client that defines the
connection on the client side. Set the variable with the following command
or place the command in the CONFIG.SYS.
Set MQSERVER=JAVACH1/TCP/9.24.104.206(1414)
1. MQSERVER is the name of the environment
variable.
2. JAVACH1 is the name of the channel to be used
for communication between client and server. The channel must be defined in the
server.
3. TCP denotes that TCP/IP is to be used to connect
to the machine with the address following the parameter.
4. (1414) is
the default port number for MQSeries.
Start Client Server Connection
- Before
the application in the client is started , the Listener program in the
server that listens to the communication link between client and server is
to be started.
start runmqlsr /t tcp /m JAVAMQM
/p 1414
- The
server can then be ready to process MQI calls from the application running
in the client.
Working
of Client/Server Connection
- All
clients use the queue manager in the server machine as there are MQI
channels of type Server Connection between the client and Server.
- When a client puts a message on a queue
it is to be read and processed by some program. This program can be
started when the server starts or the queue manager can start it when
needed by using the MQSeries triggering mechanism.
- The
below Figure shows two clients
connected to a server. Both clients request services from the same program
(Appl S1).
- Since
that application runs in the same system as the queue manager, we have
only local queues. Some queues are specifically for a particular client,
for example, QA1 is the reply queue for client A and QA2 is the reply
queue for client B.
- Other
queues are used by both clients and server. For example, QS1 is used as
output queue for both clients and as input queue for the server program.
- The
client starts a program that puts a message on a queue. For this function
five MQSeries API calls are executed:
- MQCONN
connects to the queue manager in the server.
- MQOPEN
opens the message queue for output (QS1).
- MQPUT
puts the message on the queue.
- MQCLOSE
closes the queue (QS1).
- MQDISC
disconnects from the queue manager (JAVAMQM).
- The
program can put many messages in the queue before it closes it and
disconnects. Closing the queue and disconnecting from the queue manager
could be done when the application ends.
- The
MQSeries client code that runs in the client machine processes the API
calls and routes them to the machine defined in the environment variable.
Server
Receiving Requests
- In
the server machine, the following queue manager objects are needed:
- A
channel, JAVACH1, of the type server connection.
- A
local queue, QS1, into which the clients put their messages.
- An
initiation queue into which the queue manager puts a trigger message when
a request for queue QS1 arrives.
- Initiation
queue (We use the default initiation Queue)
- A
process definition, process.S1,
that contains the name of the program to be started when the trigger
event occurs (S1).
- One
or more queues in which the program puts the reply messages (QA1 and
QB1).
DEFINE CHANNEL(‘ JAVACH1’ ) CHLTYPE(SVRCONN) REPLACE +
TRPTYPE(TCP) MCAUSER(‘ ‘ )
DEFINE QLOCAL(‘ QS1’ ) REPLACE
+
DESCR(‘ Queue for Server Application’) +
TRIGTYPE(EVERY) +
TRIGGER INITQ
(system.default.initiation.queue) +
PROCESS (process.S1)
DEFINE PROCESS(process.S1) REPLACE
+
DESCR(‘ Process to start server program’) +
APPLTYPE (OS2) +
APPLICID(‘ c:\mqtest\s1.exe’ )
DEFINE QLOCAL(‘ QA1’ ) REPLACE
+
DESCR(‘ Reply queue for client A’ )
DEFINE QLOCAL(‘ QB1’ ) REPLACE
+
DESCR(‘ Reply queue for client B’ )
- In
the server machine, two programs have to be started:
- The
listener with the command runmqlsr /t tcp
- The trigger monitor with the command
start runmqtrm
- The
listener listens for messages on the channel and puts them on the queue
QS1.
- Since
QS1 is triggered, the MQM puts a trigger message on the trigger queue each
time a message is put on QS1. When a message is placed on the trigger
queue, the trigger monitor starts the program defined in the process.
- The
server program S1 connects to the queue manager, opens the queue QS1 and
issues an MQGET to read the message.
Syncpoint and AS/400 Commit
Control
- The
data is as an MQ queue as it is in an AS/400 database file. This is done
by synchronizing the transactions,
for example, if records from a file are read and put in a queue, then data
from a single transaction should be:
- In
the queue and not in the file, if the transaction is completed.
- In
the file and not in the queue, if the transaction has failed.
- The
standard AS/400 operations for commit control also work for MQSeries for
AS/400. Therefore, the MQ operations MQBEGIN, MQCMIT, and MQBACK are not
needed.