When SAP Visual Intelligence was introduced in 2012, it could visualize data from any data source as long as it was SAP HANA. Thankfully, universe support came just a few short months later in version 1.03, opening up a wealth of data sources and allowing customers to leverage their existing investments in SAP’s “agnostic” semantic layer. Universe support in Lumira is interesting because the front-end doesn’t give the full BusinessObjects query panel experience that experienced universe consumers expect. But it’s also interesting because of how data gets from the data source, through the universe, and into Lumira. It uses a Web Intelligence mechanism originally designed to allow Web Intelligence documents to be exported from the browser in Microsoft Excel and text formats.
The SAP Lumira universe query panel
First, let’s take a look at the query panel experience. First, here’s the Web Intelligence Java-based universe query panel (from BI 4.1 SP3).
And here’s the Lumira universe query panel (from version 1.19).
At first glance I assumed that Lumira was offering some kind of reimagined query panel of the future. But upon closer inspection it’s a query panel that’s missing quite a few features from Web Intelligence panel, similar to Explorer (see related article, Family Planning), Live Office, and Query as a Web Service (see related article, What I miss in the Query as a Web Service (QaaWS) and Live Office query panels). Even Design Studio has more query panel functionality (although still a subset of the gold standard Web Intelligence panel). It’s possible that the Lumira product team assumed that its users would want something easier to use. But consider that SAP Lumira already has a free-hand SQL capability that’s still lacking from Web Intelligence. Free-hand SQL provides many things, but “easy business user interface” doesn’t immediately spring to mind (see related article, Free-Hand SQL Isn’t Free).
There definitely is an opportunity here to extend the functionality of the Lumira universe query panel.
UPDATE: Lumira v1.24 will integrate the Information Design Tool’s Query Panel, but only for UNX universes.
Consuming Existing Universe Queries
I’ve already made the unsuccessful pitch that Lumira should be able to extract queries from Desktop Intelligence documents (see related article, True Desktop Intelligence with SAP Lumira). And the longer I work with Lumira, it seems obvious that it should be able to consume a query from a Web Intelligence document, too. There are obvious differences in functionality between Web Intelligence and Lumira. Reporting is not data discovery and I’m not proposing to change that. But whether seconds, minutes, hours, or days were required to create a critical Web Intelligence report, it seems logical that a Lumira user might want to consume the same query logic without reinventing the wheel.
UPDATE (June 9, 2015): This idea was intriguing enough to APOS, who has developed the APOS Data Gateway plug-in for SAP Lumira.
SAP Lumira and the Web Intelligence processing server character stream size
The other issue Lumira users will encounter while squeezing the entire universe into their visualizations is the Web Intelligence Processing Server maximum character stream size. Experienced SAP BI administrators refer to this as the “10 kilograms of universe, uh, DATA in a 5 kilogram bag” problem. Users will see the following message and probably have their own nickname for it.
The following verbiage first appeared in the SAP Lumira 1.17 Release Notes. It was removed from the SAP Lumira 1.18 Release Notes but has been fortunately documented by SAP Note 2020352.
The data acquisition of medium-large, large, and very large datasets from UNV or UNX universes is not supported on default installations of both SAP BusinessObjects Business Intelligence platforms 4.0 and 4.1.
It is recommended for customers that want to acquire such datasets to install a SAP BusinessObjects Business Intelligence platform (SAP BIP) server dedicated to SAP Lumira and significantly increase the value of the Maximum Character Stream Size of the Web Intelligence Processing Server on that particular server. Note that increasing this value on a running SAP BIP server can impact the memory consumption and performance of any BI clients tool running on that installation, particularly Web Intelligence. While this practice is not formally discouraged, we advise customers implementing this solution that they might face memory consumption increases and longer document data refreshes so they will need to monitor the SAP BIP server’s behavior adequately to control the impacts.
In an era of “big data” hype, I have no idea what SAP means by the terms medium-large, large, and very large datasets, other than “don’t blame us if you didn’t buy a HANA server”. But as with Mobile BI (see related article, Viewing Large Web Intelligence Documents with Mobile BI), Explorer (see related article, Hacking SAP BusinessObjects Explorer 4.0) and Live Office, the solution is to increase the Maximum Character Stream Size and to a lesser extent the Binary Stream Maximum Size on the Web Intelligence Processing Server. SAP’s documentation is slightly unclear on this point, but it seems that XML and Microsoft Excel XLSX formats (which are zipped XML files- see related Wikipedia article about Office Open XML) are “character” files affected by the Maximum Character Stream Size setting. Adobe PDF and Microsoft Excel XLS formats are “binary” files affected by the adjacent Binary Stream Maximum Size setting. There isn’t an easy way to determine the optimal stream size, other than “keep increasing the value until the error goes away”. Keep in mind that poorly designed universes will return bloated data sets to any client tool, whether it’s SAP Lumira, Mobile BI, or any edition of Web Intelligence. So a code review of existing universes can be a healthy activity in addition to increasing server settings.
Oddly enough, Web Intelligence and its predecessor Desktop Intelligence contain SAP’s first in-memory database- the microcube. Long before HANA was a gleam in Hasso Plattner’s eye, the microcube facilitated multi-dimensional analysis of large datasets (called slicing-and-dicing back in the day) that may have taken a bit of time to be retrieved from the now-obsolete spinning disks in the database server. The Web Intelligence web application requests data from the microcube one viewable page at a time, but modern apps like Mobile BI and Lumira need the entire microcube before they can visualize data. So these apps are riding the coattails of the mechanism SAP originally created for exporting Web Intelligence data to Adobe PDF and Microsoft Excel files.
In a recent blog, Steve Lucas said
Build on what you already have: With 45,000 customers using SAP BI technology, there’s a pretty good chance you can take advantage of your existing BI investments by connecting directly to Universes and SAP Business Warehouse with SAP Lumira.
Steve Lucas, The Countdown on Data Visualization Has Begun – Top Ten Reasons Why SAP Will Deliver
If building on what we already have is a key selling point, there’s got to be a better, faster and more scalable way to get data from the universe into SAP Lumira.
I eagerly look forward to what is coming next from the SAP Lumira product team.
Is your organization using SAP Lumira with universes?
How these settings affect SAP Lumira
- SAP KB 1826015– SAP Visual Intelligence 1.0.7 connecting to universe data source
- SAP Note 2020352– Release restriction note for SAP Lumira 1.17
How these settings affect other SAP BusinessObjects products
- SAP KB 1527168– ERR_WIS_30272 when using LiveOffice towards a Universe Query
- SAP KB 1700085– The maximum size alphanumeric output was reached. Contact your BusinessObjects administrator. (Error: ERR_WIS_30272)
- SAP Note 1820378– Error: “Maximum binary output size limit reached. Contact your Business objects administrator (Error: ERR_WIS_30271) (Error: INF)” while saving WebI as Excel or PDF
- SAP Note 2025914– Maximum Character Stream Size” setting affects the export of Web Intelligence reports to xlsx format (not the case with the other formats)
- Impact of increasing the “Maximum Character Stream Size (MB)” value on the Webi Processing server? via SAP Community Network
- Office Open XML from Wikipedia
- SAP KB 2009536– “Could not load the Selected universe : An internal error occurred while calling ‘getSessionInfosEx’ API. (Error: ERR_WIS_30270)”
- SAP Lumira and (Insert Witty Pun Here) the Universe by Ashish Morzaria on SAP Community Network (added October 8, 2014)