Monday, January 25, 2010

Queue Monitoring using JMX

public static void main(String[] args) throws Exception {
String hostname = "localhost";
String portString = "7001";
String username = "weblogic";
String password = "weblogic";
MonitorJMS s = new MonitorJMS();
initConnection(hostname, portString, username, password);
s.getJVMData();
s.getJMSData();
connector.close();
}


public static ObjectName getServerRuntimes() throws Exception {
return (ObjectName) connection.getAttribute(service,
"ServerRuntime");
}



public void getJMSData() throws Exception {
ObjectName serverRT = getServerRuntimes();
ObjectName jmsRT = (ObjectName)connection.getAttribute(serverRT, "JMSRuntime");
ObjectName[] jmsServRT = (ObjectName[])connection.getAttribute(jmsRT, "JMSServers");
int compLength = (int) jmsServRT.length;
for (int y = 0; y < compLength; y++) {
ObjectName[] destRT = (ObjectName[])connection.getAttribute(jmsServRT[y], "Destinations");
int compLength1 = (int) destRT.length;
for (int k = 0; k < compLength1; k++) {
System.out.println(" -------------------- ");
System.out.println(" Destination Name : " + connection.getAttribute(destRT[k], "Name"));
Long l = (Long)connection.getAttribute(destRT[k], "MessagesCurrentCount");
System.out.println(" MessagesCurrentCount : " + l);
if(l>0 && y==1){
String curs = (String)connection.invoke(destRT[k],"getMessages",new Object[]{null,new Integer(0)}, new String[]{"java.lang.String","java.lang.Integer"});
Long curs1 = (Long)connection.invoke(destRT[k],"getCursorSize",new Object[]{curs}, new String[]{"java.lang.String"});
Long curs2 = (Long)connection.invoke(destRT[k],"getCursorStartPosition",new Object[]{curs}, new String[]{"java.lang.String"});
CompositeData[] curs3 = (CompositeData[])connection.invoke(destRT[k],"getNext",new Object[]{curs,new Integer(0)}, new String[]{"java.lang.String","java.lang.Integer"});
if(l>1){
CompositeData[] curs4 = (CompositeData[])connection.invoke(destRT[k],"getNext",new Object[]{curs,new Integer(1)}, new String[]{"java.lang.String","java.lang.Integer"});
Collection col = curs4[0].values();
System.out.println(col);
}
System.out.println(curs);
System.out.println(curs1);
System.out.println(curs2);
System.out.println(curs3.length);
System.out.println(curs3);
Collection col = curs3[0].values();
System.out.println(col);
//Long size = ((JMSDestinationRuntimeMBean)destRT[k]).getCursorSize(curs);
//Long size1 = ((JMSDestinationRuntimeMBean)destRT[k]).getCursorStartPosition(curs);
//Long size2 = ((JMSDestinationRuntimeMBean)destRT[k]).getCursorEndPosition(curs);
//System.out.println(size + " : " +size1 + " : " + size2);
}
}
}
}

BPEL vs OSB

Use BPEL only for workflow or long running transactions. Don't get into the Oracle Sales guy trap of using BPEL 10g on weblogic as JMS will lead to issues. Sales guys are there to sell the software and have no real life architectuere and production support issues.

OSB is for orchestration. Its lightweight and great tool. Easy to manage. Much faster than BPEL.
Avoid using BPEL if all you need to calling web services for read only purposes or simple transactions which commit quickly and use queues for retries.

Using BPEL you will need databases etc and its more hardware intensive, expensive to support. With OSB reporting those issues are rare. Probably a mix of heavy OSB use and BPEL for workflow could be a compromise.

Friday, November 20, 2009

AIA Foundation pack

AIA pack is great architectural framework.
But if you are looking for reliability then you probably have to look somewhere else as AIA is not the answer. Its a bloated architecture with so called goodies and if you have huge spare hardware and databases available then try putting AIA in production.

I believe its good to learn few things from AIA such as Error Recovery, Testing etc but keep away from AIA when implementing custom oracle solutions.

Its good if you have packaged software apps from Oracle such as Siebel CRM, Peoplesoft HR and Oracle ERP etc then you would save tons of money when implementing AIA as most of the integration you will need is already there for you in AIA.

Another point to mention is its compatibilty with Oracle service bus and Weblogic on 10g and 11g.

So Look at these point mentioned above before jumping on the AIA bus.

Saturday, July 19, 2008

JMS Architecture in SOA

JMS Architecture considerations

Why Asynchronous vs Synchronous
• Asynchronous consumers create less network traffic.
• Messages are buffered to the message listener.
• Aggregation of multiple messages into a single network call.
• Asynchronous consumers use fewer threads.
• An asynchronous consumer does not use a thread while it is inactive.
• A synchronous consumer consumes a thread for the duration of its receive call. As a result a thread can remain idle for long periods, especially if the call specifies a blocking timeout.
• For application code that runs on a server, always use asynchronous consumers such as MDBs.
• Asynchronous consumers prevents the application code from doing a blocking operation on the server.
• A blocking operation idles a server-side thread. It can cause deadlocks.
• Deadlocks occur when blocking operations consume all threads.
• When no threads remain to handle the operations required to unblock the blocking operation itself, that operation never stops blocking.

Topics Vs Queues

Topics
• The same message must be sent to multiple consumers.
• If there are no active consumers that would select it, then message should be dropped.
• There are many subscribers, each with a unique selector.
• A topic with a single durable subscriber is similar to a queue. The differences are ) Changing a topic selector for a durable subscriber will cause all previous messages in the subscription to be deleted
Queues
• A queue may have multiple consumers, and will distribute its messages in a round-robin fashion, whereas a topic subscriber is limited to only one consumer.
• Use Queues for parallel processing and load balancing.
• Use Queues when the consumers need selected transactions. Message selectors are expensive.
• If you change a queue selector for consumer, no messages in the queue are deleted.

Message Selector
• Using a selector is expensive.
• Its Critical where in the message to store application data that is accessed via JMS selectors.
• Use default Message Headers

Message Compression

• Compressing large messages in a JMS application can improve performance.
• This reduces the amount of time required to transfer messages across the network, reduces the amount of memory used by the JMS server, and, if the messages are persistent, reduces the size of persistent writes.
• Text and XML messages can often be compressed significantly.
• Compression is achieved at the expense of an increase in the CPU usage of the client.
• The benefits of compression is lost "smaller" messages. If a message is less than a few KB in size, compression can actually increase its size.
• Define Default Compression Threshold to trigger automatic Message Compression
• On the server side, messages always remains compressed, even when they are written to disk.

Message Header Fields

Instead of user-defined message properties, use standard JMS message header fields or the
message body for message data.
• Message properties incur an extra cost in serialization, and are more expensive to access
than standard JMS message header fields.
• Avoid embedding large amounts of data in the properties field or the header fields; only
message bodies are paged out when paging is enabled.
• If user-defined message properties are defined in an application, avoid using large string
properties.

Persistent Vs non Persistent Messages
• When designing an application, specify that messages will be sent in non-persistent mode unless a persistent QOS is required.
• If Synchronous writes are enabled, a persistent QOS causes a significant degradation in performance.
• Take special care to avoid persisting messages unintentionally. Occasionally an application sends persistent messages even though the designer intended the messages to be sent in non persistent mode.
• If your messages are truly non-persistent, none should end up in a regular JMS store. To make sure that none of your messages are unintentionally persistent, check whether the
• JMS store size grows when unconsumed messages are accumulating on the JMS server.

Distributed Destinations (Queues and Topics)

• a set of destinations (queues or topics) that are accessible as a single, logical destination to a client referenced by its own JNDI name.
• Members of the set are usually distributed across multiple servers within a cluster, with each destination member belonging to a separate JMS server.
• WebLogic JMS provides load balancing and failover for member destinations of a distributed destination within a cluster.
• When one member becomes unavailable due a server failure, traffic is then redirected toward other available destination members in the set.
• When a message is sent to a distributed queue, it is sent to exactly one of the physical queues in the set of members for the distributed queue. Once the message arrives at the queue member, it is available for receipt by consumers of that queue member only.
• The message is sent to only one physical queue member, there must be a queue receiver receiving or listening on that queue member.
• Queue members can forward messages to other queue members by configuring the Forward Delay attribute - the amount of time, in seconds, that a distributed queue member with messages, but which has no consumers, will wait before forwarding its messages to other queue members that do have consumers.
• Durable subscribers cannot be created for distributed topics. Instead create a durable subscription on distributed topic member and the other topic members will forward the messages to the topic member that has the durable subscription.
• If a JMS server contains two distributed topic members, then two MDBs are deployed, one for each member, so you will receive twice as many messages.

Queues and Topics Naming

• JMS_Type - Queue/Topic
• Domain - Sales, Finance, Field Support, Online Support, Back office
• App/Module - CRM, SAP, BI etc.
• Service - Schedule, Customer, Billing etc.
• Operation - Create, Update etc.
• Direction - in, out, ack.in, ack.out, req, res
• Physical servers - ms1, ms2, ... msn (Managed servers 1,2 ) n)
• Version – Service versions 1, 2

Other JMS Design Considerations

• Specify Quota and Thresholds
• Maximum Message Size
• Configure Error Destination
• Configure Expiration Destination
• Specify Redelivery Limit
• Specify Redelivery Delay
• Use Distributed Destinations
• Rotating Message Log Files
• Control either programmatically (using JMX and the runtime MBean API) or administratively (using the Administration Console) Message Operations on Destinations.
• Pause queue message operations at runtime
• Pause topic message operations at runtime
• Create Uniform Distributed destinations
• Use WLST tools for Managing JMS and Weblogic
• File-based stores must be configured on a shared disk that is available to the migratable target servers in the cluster
• Implement a SAN to make file store available from shareable disks.
• Create Custom File Stores and Avoid using Default stores.
• Monitor Pending messages.

Other JMS Design Considerations

• Synchronous Write Policy Must be Enabled if messages can’t be lost.
• Use file stores rather than JDBC stores if performance is critical.
• Disk Space must be monitored when using File based stores.
• Access to File store must be protected from being overwritten ( protect file store from multiple servers trying to access same file store).
• Two JDBC stores must not share the same database table, because this will result in data corruption.
• Sort destinations by priority by configuring a destination key
• For better performance, use message header fields as sorting keys, rather than message properties.
• Use JMSPriority for Sending high Priority Messages and set a default message Priority.
• Use JMSRedelivered property to check if it’s a redelivered message.
• Use JMSReplyTo property when seeking a response for a request.
• Use JMSXDeliveryCount to find out message delivery count
• Use JMSXGroupSeq to find out message sequence within a group.
• Use JMSXGroupID as a message group.
• JMSExpiration Specifies the expiration, or time-to-live value, for a message
• Use JMSCorrelationID is to correlate message
• Use different JMS System modules for more independent control over JMS applications
• consider using high-availability clustering software such as VERITAS™ Cluster Server, which provides an integrated, out-of-the-box solution for BEA WebLogic Server-based applications

Other JMS Design Considerations

• Paging Out Messages To Free Up Memory
• Configuring JMS Server to Actively Scan Destinations for Expired Messages periodically
• Automatic JMS Client Failover
• Use Transactions to Group Message Work
• JMS Clients Should Always Call the close() Method
• Use Hermes JMS for Message Browsing. The Link for a demo is
http://www.hermesjms.com/confluence/display/HJMS/WebLogicMQ+9.X+Tutorial
• Use Singleton for low volume Sequencing
• Use Unit of Order for high volume with Distributed Queues for Sequencing.
• Use Unit Of Work for Splitting and Merge Messaging Patterns.
• Use SAF service to enable local JMS message producers to reliably send messages to remote queues or topics

Message Unit Of Order For Sequencing

• Ease of configuration
• Leverage Infrastructure Out of the Box.
• Preserves message order during processing delays.
• Preserves message order during transaction rollback or session recovery.
• Many consumers on one queue.
• Unit-of-Order messages sent to a distributed queue reside on only one physical member of the distributed queue.
• Unit-of-Order messages are processed one at a time. The processing completion of one message allows the next message in the Unit-of-Order to be delivered.
• Member messages of a Unit-of-Order are delivered to queue consumers sequentially in the order they were created.
• WebLogic Store-and-Forward supports Message Unit-of-Order.
• Enable PreserveMsgProperty on the Messaging Bridge to preserve the Unit-of Order name

Issues With Message Unit Of Order

• Only available in Weblogic JMS.
• Under utilization of resources when the number of consumers exceed the number of in-flight
messages with different Unit-of-Order names.
• Test your applications under maximum loads to optimize your system’s performance and
eliminate conditions that under utilize resources.
• WebLogic Server Message Unit-of-Order does not support clients connecting to a non-Unit-of-
Order JMS provider
• At most, the first message within a Unit-of-Order is deliverable. Subsequent messages in the
same Unit-of-Order are not deliverable.

Store and Forward (SAF)

• Reliable Message delivery between application on different JMS servers and domains
• Transparent to JMS applications, JMS client code still uses the existing JMS APIs
to access remote destinations.
• Support the reliability of Web Services Reliable Messaging (WSRM)

Messaging Bridge

• Any two implementations of WebLogic JMS, including those from separate releases of WebLogic Server.
• WebLogic JMS implementations that reside in separate WebLogic domains.
• WebLogic JMS and a third-party JMS product (for example, MQSeries).
• uses JCA resource adapters to communicate with the configured source and target JMS destinations
• messaging bridge instance forwards messages between a pair of bridge source and target destinations
• Messaging Bridges increase latency and may lower throughput. Messaging bridges increase latency for messages as they introduce an extra destination in the message path and may lower throughput because they
forward messages using a single thread.
• Messaging Bridge Supports for foreign and legacy providers
• Use WebLogic Store-and-Forward to Forward messages between WebLogic 9.0 domains.
• configure one bridge for each member of a distributed destination using the member's JNDI Name.
• send to the distributed destination using the distributed destination's JNDI Name and disable server affinity

Best Practice Deploying Applications

• Use the Administration Console for interactive deployment sessions where you do not know the exact location of deployment files, or the exact names of target servers or deployed applications.
• Use weblogic.Deployer to integrate deployment commands with existing administrative shell scripts or automated batch processes.
• Use wldeploy in conjunction with the split development directory for developing and deploying new applications. wldeploy can also be used in place of weblogic.Deployer in administration environments that use Ant, rather than shell scripts.
• Use WLST when you want to create automated scripts that perform deployment tasks

WLST Scripting

• Propagate a WebLogic Server domain to multiple destinations using predefined configuration and extension templates.
• Retrieve domain configuration and runtime information.
• Edit the domain configuration and persist the changes in the domain’s configuration files.
• Edit custom, user-created MBeans and non-WebLogic Server MBeans, such as WebLogic Integration Server and WebLogic Portal Server MBeans.
• Automate domain configuration tasks and application deployment.
• Control and manage the server life cycle.
• Access the Node Manager and start, stop, and suspend server instances remotely or locally, without requiring the presence of a running Administration Server.
• Schedule scripts to run at various times
• Automate repetitive tasks and complex procedures
• Configure an application in a hands-free data center
• Automate WebLogic Server configuration and application deployment
• Apply the same configuration settings, iteratively, across multiple nodes of a topology
• Take advantage of scripting language features, such as loops, flow control constructs, conditional statements, and variable evaluations that are limited in interactive mode
• Running WLST From a Java Class and Ant or Run a Script from WLST

Monday, July 14, 2008

Exceptions in JMS Architecture

System Exception Steps
• These exceptions occur because there might be some issues with the system such as any component might be down, could be a database issue, a service is down.
• On any System Exceptions the transaction should be retried a certain number of times with Delay or no delay between message deliveries.
• Always Log the error in the log files and notify the support about the exception.
• Suspend sender process if possible after 1000 number of time (During a Prolonged Outage) - Create a script using JMX to suspend the message Producer
• Try Sending notifications every half an hour/one hour for same errors that you are encountering.
• If message expires then move the expired messages to the expiration queue - This queue must be defined in the jms configuration.
• Introducing a timeout delay in the message consumer could be one possibility
Business Exception Steps
• These types of exceptions could be due to some data issues, some synchronization issues between system. For example customer data is missing in the receiving system.
• These types of errors shouldn't be retried. The data must be fixed and then the transactions should be replayed or retried.
• Always Log the error in the log files and notify the support about the exception.
• Always Put these erred messages and the errors combined as a separate message in the error queue - this must be defined separately in the code
• Try to use a workflow process to generate exception workflows from this error queue.
• Have a UI to browse exception messages to debug or resend these exceptions by support personnel