Don’t Fluster the Cluster, part 2

Don’t let your BI platform become a cluster chuck!

I spied this fantastic new collection of Woodchuck Cider on an end cap at my local grocery store and just had to snap a photo. It’s a variety pack cleverly named the Cluster ‘Chuck Variety Pack.

Woodchuck Cider Cluster Chuck at Kroger

The Woodchuck Cluster ‘Chuck Variety Pack is our new cider sampler. Following suit with the rest of our ciders, the pack has a brand new look to go with our brand new Cidery. Our top notch in-house creative team set out to create a design that fit the spirit of our ciders, our local orchard partners, and our home state of Vermont.

From the Woodchuck Cider blog

I had so much fun talking about SAP BusinessObjects CMS clustering and Ben and Jerry’s Clusterfluff ice cream (see related article, Don’t Fluster the Cluster). A bit of humor can really help bring a dry technical topic to life. The 2014 conference season is now over for me, but I thought somebody out there would see this and be inspired to use it in their own slide decks about CMS clustering. We wouldn’t want any SAP BusinessObjects installations to turn into a cluster chuck, would we?

Raising my glass to SAP BusinessObjects administrators everywhere. Skol!

Flustering the Cluster with Inconsistent DB Parameters

Too much party time in Vegas? Or just an inability to read my own documentation?

Today was my first day back in the office after last week’s SAP Insider BI2012 conference in Las Vegas. The conference occurred in the middle of building my customer’s QA cluster for SAP BusinessObjects Business Intelligence 4.0. QA is our first clustered environment – all of the lesser environments are single-node virtual machines. The QA cluster is intended to be a mirror image of our future production cluster: two Tomcat servers with a load balancing switch, two CMS nodes, and four non-CMS nodes running various processing and job services. Before going to the conference, I installed and configured one Tomcat server and one CMS node. So the system was 100% functional, but not in its final size or configuration.

So today I started installing the second CMS node.  Custom / Expand install type – check. Change the default drive letter – check. Clear the web tier, integrated database, and Subversion checkboxes – check.  Everything was going according to plan until I entered the Oracle 11g credentials for the CMS database.

Cannot Verify Database login information. (INS00104)

Seriously?  OK, maybe it’s a typing error?

Nope.

Stuck.

So I went to the SAP Support Knowledge Base and searched for INS00104. Several articles appeared. Linux? Nope. mySQL? Nope. Ah, here’s one about Oracle- SAP Note 1638205.

If you’re able to connect and verify with SQL Plus or another test, remove special characters (specifically dollar signs) from the password on the Oracle DB. We’re not properly escaping them. This should be fixed in service release 3 for BI4.

The database password doesn’t include any special characters, but my fear during this project is that somewhere out there is a bug waiting to derail my project. A bug that won’t be fixed until Feature Pack 3. Could this be the one?

Panic.

Our operating system is Windows 2008, not Linux. So what the heck. I re-booted the server and took a few minutes to re-boot my head. Then I re-read the installation run book from the first CMS node.

Eureka!

The first CMS node was built with a generic tnsnames.ora entry I had created called SAPBISYSTEM. It was identical to the one provided by the Oracle DBA that used the server name as the system name. Even though the details underlying the two entries were identical, the installation program didn’t know that. It apparently also didn’t know to say “Rough time in Vegas? Why don’t you try using the same Oracle system name as the first node?” Once I entered the same Oracle system name as the first node, the second node’s installation proceeded as normal.

Success!

I wrote this short post so you won’t make the same mistake as me. Today’s experience made me glad that I had taken time to document each installation step with both screen shots and written assumptions.

Make sure your multi-node installations are identical in every detail. But especially with CMS database parameters.

Don’t fluster the cluster.

Don’t Fluster the Cluster

An improperly designed architecture may cause your cluster to become fluffed.

Ben and Jerry’s recently introduced a new flavor called Clusterfluff (now rebranded as the more politically correct What a Cluster), described as “peanut butter ice cream with caramel cluster pieces, peanut butter and marshmallow swirls”. Yummy. But today, let’s talk about a different cluster: SAP BusinessObjects Business Intelligence, also known as SAP BusinessObjects Enterprise.

Ben and Jerry's Clusterfluff Ice CreamA cluster in SAP BusinessObjects Enterprise is defined as a system with multiple Central Management Servers (CMS) that work together by sharing a common system, or CMS, database. Each CMS is typically on its own physical device, known as a node. SAP BusinessObjects enthusiasts who take the BOE330 training class, Designing and Deploying a Solution, get to team up with their fellow students and create clusters in class. Let’s discuss using Microsoft Windows; however, the same principles apply to Linux/Unix.

I recently encountered a BI system with a poorly devised cluster. It was the second time that I’ve seen this ill-advised configuration on the XI platform, so it seemed worth writing about. Especially since SAP BusinessObjects Business Intelligence 4 continues the server architecture of the past few versions. In both of my situations, each physical server, or node, was built with a “new” install.  Once the installation was completed, the second CMS was stopped and reconfigured to point to the same system database as the first CMS, as shown in the illustration below (click image to enlarge).

How to Cluster SAP BI - Bad

Each node has an identical configuration. There’s two of everything. Two Central Management Servers. Two Crystal Reports Processing Servers.  Two Web Intelligence Processing Servers. And, sigh, two Input File Repository Servers, or iFRS. There are two Output File Repository Servers, or oFRS, as well. Each file repository server points to local storage on the node, which is a significant flaw that could lead to corruption of the cluster.

In a SAP BusinessObjects cluster, only one FRS is actively working even when all FRS in the cluster are enabled. Just like many of your coworkers, the rest of the FRS sit around doing nothing in a passive state waiting for the active FRS to fail. All FRS connect with the CMS upon startup. The active FRS is the one that contacts the CMS first.

So how can the cluster become corrupted? Let’s assume that the iFRS on node 1 becomes active. When a report is published to the system, it is stored on the iFRS default location, which is the C: drive on node 1. The CMS database contains an InfoObject containing the physical path to the report.

Next, let’s assume that the iFRS on node 1 fails. The iFRS on node 2 becomes active. When Wanda in accounting logs into InfoView to view a month-end report, the processing server will fetch the report from the active iFRS on node 2. However, because the report was originally written to the C: drive on node 1, Wanda will receive an error and call the Business Intelligence help desk.

The cluster, sadly, has become fluffed.

A better solution would be to disable the iFRS and oFRS on node 2, guaranteeing that the iFRS and oFRS on node 1 are always active and that all documents are stored locally on node 1. This configuration is shown in the following illustration (click image to enlarge).

How to Cluster SAP BI - Better

You might argue that this configuration is not fault tolerant. And it isn’t. However, neither was the first configuration. Disabling the second iFRS and oFRS is the first step to preventing additional corruption.

The best solution is to create a file system that both nodes can share, as shown in the illustration below (click image to enlarge).

How to Cluster SAP BI - BestEach FRS is configured in the Central Management Console (CMC) to use the shared space, typically by specifying a UNC path. Since the shared space is on a different device, the File Repository Servers will need to run using a domain account rather than the default Microsoft Windows Local System account. This account is specified in the Central Configuration Manager (CCM), either directly on the File Repository Server (XI R2) or vicariously through the SIA (XI 3.x, BI 4.0, BI 4.1). Be sure that separate directories are created to distinguish the Input File Repository from the Output File Repository; however, both can share a common parent directory, as they do in the default out-of-the-box configuration.

There are other design considerations for a truly fault-tolerant BI architecture, such as failover in the web application server tier. But that will have to wait until a future discussion. In the meantime, I hope you will enjoy a pint of Ben and Jerry’s Clusterfluff ice cream. And inspect your BI system to insure proper configuration.

Resources

SAP KB 1378753 – How to configure the File Repository Server (FRS) to point to the Network Attached Storage (NAS)?

What are your experiences with SAP BusinessObjects clustering? Share your thoughts (and favorite bookmarks).

BOE330

B