I’ll just come out with it: you should share PDF files, not Excel files when distributing dashboard reports.

Sacrilege! Say it isn’t so! I know, this is pretty odd advice coming from someone who professes that Excel is one of the best tools available. But there are times when you have to remember that your focus is providing and distributing the information, not the vehicle that your audience uses to view information. It pains me to say it, but they don’t care how efficient your formulas are, or what controls or tricks you employed in Excel to develop the report that they’re inspecting.

Efficiencies in Excel spreadsheet development are important, and that has been discussed to no end on the Internet, this site included. But all of that becomes completely irrelevant when you remove 100% of the recalculation time for your real end users by providing the final charts, metrics, and calculations to them as printed pages.

These days when you distribute PDF files to your report readers, you empower them to be able to take your beautiful works of art with them everywhere they go on their smart phones, albeit at the cost of interactivity. I’m confident that it’s a trade-off they’ll gladly accept. Even better, you don’t have to spend time teaching them how to use your file, nor spend time building in safeguards and usability enhancements, outside of what you want for yourself and the core functionality of the document, that is.

After all, for every Excel user that as comfortable with it as you are, there’s at least one other in that same office or company that always seems to need help regardless of how easy to use you thought that you’d made it. You know it and I know it. That’s the reality and unfortunately it’s not likely to change any time soon. When you make it easy for them (by eliminating Excel altogether in this case) you make it easy for yourself too.

There are a number of additional benefits that distributing a PDF file offers you:

1. The presentation of your dashboard style report will be pixel perfect.
2. The readers won’t be able to accidentally alter the data or otherwise corrupt the workbook.
3. While there are a number of techniques to keep the file size down (and thus increase the performance) of Excel dashboard report files, they’ll never compare in size to the amount of analysis that you can pack into a PDF file that’s just a few megabytes in size.

Color Code Different Sections

At one time my group distributed a printed and spiral bound monthly financial analysis package to each manager of the various operations groups and other business leaders. To separate the different sections within the document, tabbed cardboard dividers were used. To cut costs for both materials and preparation labor, and to save on our environmental impact, I proposed that we color code reports for different departments, product lines or other important topics. This will allowed users to instantly know when they were looking at the correct section of the document for their numbers, and the dividers were no longer necessary.

Personally, I prefer a very minimalist color scheme for dashboard reporting: black, shades of grey and a splash of red when required, like this. But many prefer the style that Charley Kyd promotes. At the time, my audience preferred the latter, so I took advantage of it.

When reporting for different business lines, I used VBA code to change the color of the elements on the page: the title area, the charts, the tables. Excel is much more flexible now with its available color choices than when I was building my first dashboard reports with Excel 2003, but that’s a bit of a double edged sword. Making choices can a terrible distraction. Pick a few basic color combinations and get on with it.

I’m no graphic designer, so here’s what I did to achieve what I considered to be a professional result: I plagiarized. Well, not exactly plagiarized, but rather sought inspiration from experts. I knew that design experts are often hired for their assistance in preparing the annual reports for large public companies, so I spent a bit of time downloading some sample reports from the larger companies, looking for combinations that I liked and that I could emulate rather well with my Excel dashboard. So in essence I got expert advice for free!

As to the specific VBA code, preparing that wasn’t overly difficult either, as I used the macro recorder to have much of the code generated for me by Excel and then I modified only certain aspects of that generated code to suit my purposes. I’m confident that you’ll have no trouble handling that.

Automate with PDF Production with VBA

As I said earlier, my first “real” dashboard reporting systems were built with Excel 2003, and I must say that I had a heck of a time developing the required code to produce PDF files back then. The only thing that enabled me to do it was that I also had Adobe Acrobat installed on my system, and after an inordinate amount of online research, a ridiculous amount of hours to be honest, I finally determined how to automate the Adobe Distiller program. It might have been easy if my company had permitted me to register as a developer with Adobe but the cost was viewed as prohibitive.

Luckily for you, these days you won’t need to go through any of that.

You see as of Excel 2007 (SP2 I think), and now in Excel 2010, you can now produce PDF files with as little as a single line of VBA code:

This singular ability enables you to easily generate all of the benefits discussed earlier. What does that line do? It prints the active worksheet, as you’ve defined the printed area to be, to a good quality PDF file. You can easily change the file name and location. In fact, that’s key to your new reporting system.

It might take you a bit of work to develop a full PDF creation routine that expands on that single line, but it’ll be well worth it. Then when you click a “Make PDF File” button, your program will begin stepping through all of your important data points, recalculating the dashboard, and then preparing a PDF file for that data point. Then it will move on to the next data point and repeat the process until it has created a PDF file for each data point.

Meet the PDF Toolkit (PDFTK)

If you haven’t heard of it before, allow me to introduce you to an absolutely indispensable tool for those who don’t have high end PDF software like Adobe Acrobat: PDF Toolkit (PDFTK). Heck even Acrobat users can make good use of this one.

PDFTK is a utility that lets you do practically anything that you’ll ever need to do with PDF files: merge, split, rotate, and more. It’s free, and you don’t need Acrobat to be able to use it.

Download and install PDFTK now.

Make the work easier to repeat

My favorite approach to PDFTK, given my inclination towards batch files, is to script it on the command line (see examples). Of course, it’s easy to incorporate any of that into some VBA code too.

Now, seeing as you’re still with me, I’m going to reward you by doing something silly and literally give away a bunch of valuable program code. It still needs some details to be filled in, but the skeleton should help immensely.

Clearly, this is a simplified shell of an example but it shouldn’t be too difficult, especially with a good VBA book on your desk such as Excel 2010 Power Programming with VBA (Mr. Spreadsheet’s Bookshelf), to make it work for your specific needs.

Your raw data might have originated in report files, Excel tables, Access databases, other database formats, or even PDF files, but you persevered and created models that allowed you to capture and manage that data, and create summaries in formats that may never have been previously envisioned to help you and your group to gain new insights into your operations. Then you used the skills that you learned in developing Excel dashboard reports to present that data visually in ways that let you easily control the included data.

Even if you opt to forgo the convenience and speed that VBA programming can bring to your project, distributing the final results of your efforts with PDF files will demonstrate to your entire group that, in no uncertain terms, you excel with Monarch.

I was recently invited to write a post for Datawatch’s blog.

Today you can read my new post on the Datawatch blog to find out what focus, specialists and Bill Gates have to do with how well your organization will succeed tomorrow based on what you do today, and how all of that impacts our entire society when we choose to excel with Monarch!

photo credit: batmoo

Worldwide Quantity Shipments

Recently, I looked at how Monarch builds range names when exporting to Excel, and then some examples of how we can use those range names in Excel functions. Today we’ll look at the process of having Monarch create an Excel dashboard report workbook and how we can begin to easily add dramatic functionality to that workbook.

1. Build the required summaries

Again, for this demonstration I’m using the supplied sample reports as the data source (with permission from Datawatch). I have a few “secrets” for building summaries that will be used for the purpose of building Excel dashboard reports, including:

• Don’t use subtotals for the key fields. They’re not necessary for the purpose of developing dashboard style reporting.
• Don’t suppress duplicate key field values. For many reasons, we want each row to behave like a regular database record, with values in each key field.
• Limit the summary to very few measure values. Often one measure field is sufficient.
• Limit the number of key fields whenever possible. We’re after a simple, easy to query layout and multiple key fields adds to the overall size of the Excel file unnecessarily, and excessive or redundant data will negatively impact the speed and performance of the file.
• Never set a summary to use the “default filter”. You want to be absolutely certain of the data that you’re funnelling into the Excel file, and remember that Monarch will use the currently active filter in the Table as the “default filter” for that summary.
• Include grand totals only when necessary. Further, always use the same wording for the grand totals. By using the same wording, one Excel formula will be able to extract the total amounts for any summary.
• Name the summaries as descriptively as possible. You’ll probably have quite a few of them and it helps immensely to know the content just by the name. Short names are convenient to work with, but have limited usefulness.
• Build one field summaries. Summaries consisting of only a single key field, while seemingly simplistic and of no analytic use, are absolutely critical to building useful dashboard report systems.

Worldwide Shipping Revenues

Customer Shipments

Customer Revenues

Media List

Customer List

Country List

Top 10 Customer Revenues

Top Revenue Products

Top Revenue Products by Country

2. Create the Excel Workbook

Since this will likely be a process that we’ll repeat on a regular basis to update the file with current data, we’ll create a project export in Monarch. For all of the specific Monarch work here I’ll be referencing Monarch v10, though the process will be similar if not identical for all versions.

Define a new project export. Name it All Summaries, and choose to export summaries. From the Summary Name list, select the “All summaries” item. Elect to use Automatic Naming by tables, not files. Do not apply any additional filtering to the export process at this stage.

Now this next part of the project export definition process is really important: supply a file name using the “.xlsm” extension, not the “.xlsx” extension. This is because you’re going to add one single critical line of Visual Basic program code to the file (more if you’re up to it) and you cannot add that to an “.xlsx” file. I’ll supply that code later.

Typically when we export from Monarch to Excel files, especially reused files, it’s very common to overwrite the previously exported Excel file with a new export, but we don’t want to do that when developing dashboard report files. Instead indicate that you want to add data to the file. It’s only when specifying what you want Monarch to do with the individual tables that get created in the Excel file that you want to select the overwrite option.

Finally, to create the fastest, most responsive Excel file, do not add any further features to the exported tables such as titles, outlines, or formulas.

Run the project export to create your new dashboard report Excel file.

3. Add devices to facilitate dashboard report development.

First up is a little bit of housekeeping that is required due to a bit of a bug in Monarch Excel file creation process. When Excel files that Monarch updates are opened, Excel often requires that you force a manual recalculation. You can doing this simply by pressing the F9 function key, but it’s easy to overlook and that can cause problems. So we’ll add a line of program code to have Excel recalculate the workbook immediately when you open it.

In Excel, hold down Alt and hit F11. This will open Excel’s development window. You should see the current file listed sheet name by sheet name in the left side of the window in a Project window. Double click the item for the ThisWorkbook item. Now at the top of the window, you’ll see a drop down list with a (General) label. Select the Workbook item from this list. Note that Excel automatically creates a new subroutine for you:

To that add a single line of code:

Excel Workbook Open Event

Close the programming window and return to your Excel workbook.

For our dashboard report of the Classical Music Distributors data, we want to be able to spotlight data for activity in specific countries, media types, and customers. To do that, we’ll add drop down lists for each of those data items to the dashboard. The values for each of those lists will be automatically populated by our Monarch exports, but the lists need a little help to get exactly the right data into them. Let’s tackle the countries item first.

Go to the Country_List sheet, and define a new range name. Name the range “Countries”, and assign it the formula “=OFFSET(Country_List!$A$2,0,0,COUNTA(Country_List!$A:$A),1)”. With this formula, “Countries” will refer to every country name in the list, avoiding the field name in the first row. Because we often want the values in the heading row, we don’t have the luxury of exporting not including the field names in only selected sheets, but using this approach it’s easy to skip the heading.

Use the same approach to create range names for “Customers” (“=OFFSET(Customer_List!$A$2,0,0,COUNTA(Customer_List!$A:$A),1)”) and “Media” (“=OFFSET(Media_List!$A$2,0,0,COUNTA(Media_List!$A:$A),1)”).

Now we can add a new blank worksheet into the file. Rename the sheet to something more descriptive.

To quickly see how our new range names are behaving, add a data validation rule to a cell. Select “List” from the Allow list, and set the Source box to =Customers. Note that you must include the equals sign otherwise you’ll see only the text “Customers” in your drop down list. While this works well, there is a drawback to using the data validation list approach in that the font size of the list items can get a little small and hard to read depending upon the screen resolution. There’s a better mechanism available, but we’ll save that for later.

The attentive will have noticed that our range names are each returning one extra row with a blank value. This will become apparent later when we look at how the selected values are used to display information in the dashboard.

We’ve made some good progress on a dashboard reporting system today. To learn more details of how to build these types of reporting systems, be sure to pick up a copy of “Dashboard Reporting with Excel”. I’m positive that as you read it you’ll envision plenty of new ways to excel with Monarch.

I know that this sort of statement makes me sound old, but it’s amazing that some things in the past seem like they happened only yesterday. It was in 1991 that Monarch software (for DOS) first appeared, on two 5 ¼ inch floppy disks, or a 3 ½ inch floppy if you were able to use one of those new devices. This was to be a very important product, and it would prove to lead the way for a young company named Datawatch Corporation.

Allow me to fast forward to 2011, and to the introduction of another leader. On February 14, 2011 Datawatch Corporation announced that Mr. Michael Morrison was the company’s new president and chief executive officer. In the past year, Datawatch has undertaken many changes and has shown significant growth, and not just in terms of revenues. The stock price has skyrocketed, reflecting investor confidence in the dealings of the company. A new version of the company’s flagship Monarch software was released, and —along with a Datawatch customer – was a winner of Ventana Research’s 2011 Leadership Awards.

Today, just over a year later, I’m pleased to welcome to Mr. Morrison to ExcelWithMonarch.com for an exclusive interview.

SC: Why did you want to join the Datawatch team?

MM: I saw in Datawatch a company with a great product portfolio, a passionate – and extensive – customer base, and an emerging market opportunity that I believed to be very big. Since I have been on board, all of these observations have been validated. Our Monarch family of products is a truly differentiated solution that delivers real ROI. Our customers are zealots, and some of our most influential supporters. And the market opportunity for our report analytics solutions is even bigger than I initially anticipated.

SC: Time flies. Congratulations on your first anniversary at the helm of Datawatch! How has the past year been for you and Datawatch?

MM: For me personally, it has been a very enjoyable year. I have had the opportunity to work with a terrific group of people and meet with a number of our customers and partners. We experienced a lot of change as we embarked on transforming Datawatch to a high growth company – and change is always difficult at some level. Nevertheless, our employees, our customers and our partners persevered and are leading from the front in this company transformation. The past year has been very good for Datawatch as well. The company is now on a growth trajectory. The financial markets have taken note of us. And industry analysts are paying attention to us again. All in all, it’s been a solid start for Datawatch in getting to a high-growth revenue model.

SC: Fiscal 2011 ended on a very positive note for Datawatch, but you really knocked it out of the park, as they say, with the Q1 FY2012 results. How did Datawatch achieve such a terrific quarterly result?

MM: Our Q1 FY12 results reflect our new go-to-market model, our new market positioning and the contributions of many of the new sales people that joined Datawatch in the past year. One good quarter doesn’t make for a trend, but it’s a positive momentum builder for us.

SC: In 2011, ExcelWithMonarch.com had readers in over 120 countries. Globally, it seems to me that there continues to be a need to easily acquire data and convert it into meaningful, actionable information. With offices around the world, does Datawatch also see this need?

MM: Absolutely. Access to and the ability to analyze corporate data is a universal problem. Despite decades of advances in business intelligence technologies, reporting is still broken. There is a ‘blind spot’ in reporting and analytics that Datawatch uniquely addresses. On one end of the data analytics spectrum, you have structured data, addressed by the traditional business intelligence solutions. On the other end, you have unstructured data, with an ever-increasing number of ‘Big Data’ vendors playing in that space. In between the two exists semi-structured and loosely structured data. Datawatch is the only company that solves the problem of analyzing that data in the middle.

Our ability to make information from diverse data sources and formats accessible, usable and valuable is unmatched. There is a global need for our solution and we are investing in our international operations to meet this need.

SC: I know that Monarch is offered in the English, German and French languages. With such worldwide interest in the product, has offering the product in other languages been considered?

MM: We are presently evaluating Spanish and Portuguese. As opportunities arise that require other languages, we will evaluate which ones make business sense. Our goal is to bring our Monarch report analytics solutions to as many people as possible.

SC: It seems that in North America, there are many opportunities to sell Datawatch solutions as a great method to extend the useful life of a customer’s legacy information system. But outside of North America, companies seem much more inclined to replace their older information systems altogether with integrated applications. Does Datawatch encounter this situation, and if so, how do the approaches to sales differ?

MM: We see many varied use cases around the world, and do see some differences between North America and Europe in dealing with legacy systems. That said, even with the more modern transaction systems, we have customers using our technology to enable end users to be more self-sufficient and reduce the burden on IT. And every organization has to deal with reports and business documents that come into the organization from third parties. Even the most technologically and operationally advanced organizations struggle with incorporating report data from outside their four walls. On a related note, over the past year we have completely revamped our go-to-market model, which includes addressing the needs of local markets around the world. Our approach – regardless of geography – is to always engage the prospect and learn their unique pain points and specific data analytics needs. This allows us to assess how Monarch can help them. Even with the more modern transaction systems, we have customers using our technology to enable end users to be more self-sufficient and reduce the burden on IT. Another major factor is the integration and analysis of data coming into organizations from external sources. Even the most technologically and operationally advanced organizations struggle with incorporating data from outside their IT infrastructure and our technology plays a major role here for many Fortune 500 customers.

SC: Clearly the primary focus here at ExcelWithMonarch.com is your flagship Monarch product. Can you take a moment to tell the readers a bit about the role that Monarch plays in your suite of products, and how the skills that they develop with Monarch may be beneficial when moving to your other solutions?

MM: Monarch is the robust modeling engine for our Monarch Report Analytics platform. The Monarch Report Analytics platform also includes the capabilities to automate and validate the acquisition of data for the Monarch models (Monarch Data Pump) and the capabilities to distribute and archive the resulting Monarch reports in a secure, web-based, self-service analytic environment (Monarch Enterprise Server). Monarch Report Manager on Demand and Monarch Report Mining server offer the additional capabilities to archive and manage high volume reports and business documents and perform rich analytics on this stored content. The Monarch Report Analytics platform takes data from semi-structured and loosely structured diverse data sources and makes this information accessible, usable and valuable.

SC: My personal experience is that people tend to initially expect that Monarch is a tool for the Finance and Accounting group. Are you finding that Monarch is used primarily in that arena, or have other groups adopted it as well?

MM: We have many Monarch users in the office of finance. However, use of Monarch extends literally across the entire enterprise. We see many use cases in human resources, marketing, sales, procurement, IT and more. The elegance of Monarch and its ease of use make it an ideal solution for any business analyst.

SC: Clearly the need for more sophisticated and larger scale solutions increases with the size of the customer’s organization. Can you talk about some of the solutions that Datawatch offers larger customers?

MM: Our larger enterprise customers use the Monarch Report Analytics platform to address a variety of business challenges across all departments. Some specific examples include procure to pay (automating the matching of purchase orders to invoices to bills of lading), e-presentment (delivering settlement statements and commission statements to individual brokers generated by mainframe transaction systems) and audit analytics (performing forensics and reconciliations on a variety of internal and external reports).

SC: Datawatch has announced many new partnering arrangements over the last year. Will you be working towards developing more of these arrangements?

MM: A key element of our high growth strategy is aggressively expanding our partner ecosystem. We are engaging with implementation and reselling partners, OEM partners, geographic distributors and strategic alliance partners around the world. We have implemented a partner friendly business model that makes doing business with Datawatch easy and profitable for partners.

SC: My son happens to be entering high school next year, and in looking at his options recently, I was shocked at the business and computing classes that are available to young students today. Have you considered offering Monarch software to schools to introduce students to the benefits of the tool and familiarize them with your company, not unlike the strategy implemented by Apple Inc.?

MM: We make Monarch available to academic institutions today and intend to more broadly formalize this program for academic use of Monarch in the coming months.

SC: Speaking of education, many professional groups involved in everything from accounting to zoology have for many years now used certification to help market themselves as experts in their field. Indeed, certification markets not only the individual, but the field of expertise itself. Does Datawatch currently have any plans to implement such a programme for your users?

MM: Yes. We are currently developing a certification program for users of our Monarch solutions. Stay tuned for more details.

SC: Automation of Monarch processes has always been a focus point on ExcelWithMonarch.com, and has been appearing more and more of late on your MonarchForums.com. Do the automation features tend to assist sales of the software, or is it at times a distraction when proposing other scalable solutions?

MM: The automation capabilities in Monarch are being used in ways that we never intended and, at times, have led to issues with deployments and data fidelity. The automation capabilities in Monarch Data Pump and Monarch Enterprise Server are much more appropriate for business applications. We will continue to stress the value in automating your report analytic solutions with these enterprise offerings.

SC: Many have commented over recent years that the official Programmer’s Guide hasn’t been updated since it was released for Monarch v8. Will an updated Programmer’s Guide be published for Monarch v11? And on a related note, will there be an update to the Functions Reference Guide, which was last updated for Monarch v9?

MM: We do not intend to continue publishing a Programmer’s Guide. The Functions Reference Guide has been updated and is available on our website.

SC: Many software companies seem be jumping on the “cloud” bandwagon of late. Aside from Monarch Enterprise Server, does Datawatch envision a cloud-based, or perhaps a browser-based, Monarch offering?

MM: We do not have any current plans for a cloud-based or browser-based Monarch offering.

SC: Some foretell of a possible demise of green bar reporting, based on the proliferation of systems capable of PDF output. Given your experience in the marketplace, and an ever-growing list of PDF reader software, do you see Datawatch’s niche changing in regards to Monarch’s functionality to respond to marketplace changes? Simultaneously, how do you continue to further the status of Monarch as the “go-to” tool in all offices?

MM: Mainframe green bar reports are only a small fraction of the data sources that our customers access with Monarch. The sources and formats of reports that our customers can access with the Monarch Report Analytics platform are extensive: text, PDFs, Excel files, SQL databases, HTML, EDI streams, log files, business documents stored in a content management system, etc. In essence, the PDF output of current systems is really just another container for the same old “standard reports” we used to print. A G/L or Trial Balance looks just the same in PDF as on a mainframe green bar report – just with better fonts. Our ability to consume PDF in the same way as text reports means we bring our core competency to bear, which is the easy, end user-driven access, parsing, extraction and analysis of semi-structured business documents. As the market evolves, and new diverse data sources and formats become more relevant, Datawatch will continue to innovate to address the changing needs of the market. Our 20 years of experience and extensive customer base give us a great knowledge base from which to deliver to meet the demands of the market.

SC: In past years, we’ve seen announcements from Datawatch concerning acquisitions of companies and their products and technologies. Can you tell us a bit about what has happened with some of those products and technologies?

MM: The most important acquisition was Clearstory and the BDS solution. Last year, we re-branded this solution as Monarch Report Manager on Demand (RMOD). RMOD is a high volume report management and archive solution that is being used in some of the world’s largest organizations. Together, with our Monarch Report Mining Server solution, we offer the only end-to-end report management and mining solution in the market. RMOD has become an integral component of our overall Report Analytics offering.

SC: Thank you very much for your time and for sharing your thoughts with us today, Mr. Morrison. I hope that 2012 becomes a breakthrough year for Datawatch.

MM: Sandy, thank you for taking the time to speak with me. And thanks for your long-time support of Datawatch and Monarch. You are a great evangelist and advocate.

A Winning Team Is No Accident

Thanks again to Mr. Morrison for granting this interview and taking time out of his busy schedule. And thanks to all of the Datawatch team, who continue to develop new tools and solutions that enable all of us to excel with Monarch.

photo credit: IntelFreePress

In the previous post, The Secret is in the Names, I wrote about how Monarch populates Excel files with useful named ranges when exporting data. Today we’ll look at how we can capitalize on those named ranges with specific Excel functions.

In that previous post, the first point regarding the range names is that Monarch creates all of its names with an underscore if it needs to replace any space characters in our summary names. We’ll keep that in mind as we proceed.

One of the things that I like to utilize when building Monarch summaries is setting a key field to display its values across the top of the summary above the measure values, instead of down the left side of the measure values. This is particularly useful when working with dates or quarters, such as “12/1/2012”, “Dec-2012”, or Q4-FY12.

When working with a number of summaries, even across multiple models, I like to stick to the same title formatting. Plan ahead, and decide upon the time period (calendar months, fiscal periods, quarterly, what have you) that you’ll be using for reporting values. The reason for this will become even more apparent later when we look at using specific values in that selection of titles in individual Excel formulas.

For demonstration purposes, I’ll refer to the sample report files that are included when Monarch is installed, specifically the Classical Music Distributor reports, ClassJan.prn, ClassFeb.prn, ClassMar.prn and Classic.prn (which has April data). Datawatch has graciously given me permission to use those files for demonstration purposes here at ExcelWithMonarch.com

Each report has date on which the customer’s order was shipped, but we’ll choose to report shipments by calendar month. For simplicity we’ll present the date as m/d/yyyy, using the first of the month in each case. To get the Period field, we’ll build a calculated Date type field with this expression:

This will give us 1/1/2008, 2/1/2008, and so on. As mentioned, these become the column headings in the summaries that will be prepared.

Later, when we want to determine which column in the worksheet contains the February values, which we built another named range for called Reporting_Period, we can use a formula similar to:

and if the February date is in column D on the worksheet that Monarch exported, then the formula will return a value of four, meaning that the February date was found in the fourth position. This is immensely useful. The zero value at the end tells to Excel to find the exact search value.

Look at that MATCH() function again. It’s simple and easy to read/review, but there’s some else very convenient about it as well. What do we need to change if we want to use that formula to refer to a different worksheet? Correct: only the worksheet name.

As easy as that is to build, we can make it just a little bit easier to build a whole collection of similar formulas that refer to many worksheets. Since we’ll probably have quite a number of summaries in our models, we’ll also have a number of exported worksheets to help in our analysis of the data. While we could just revise the sheet name in each formula, there’s an even easier way: have Excel build a portion of the formula for us! OK, now you’re thinking, “What is he talking about?” We’ll use Excel’s INDIRECT() function. It works like this:

• Type the exact name of the exported worksheet in a cell, say A8.
• Now in cell D8, the formula becomes:
  • =MATCH(Reporting_Period,INDIRECT($C10&”!Print_Titles”),0)

Great, now we know how to find the proper column for a certain period, when the values go from left to right across the worksheet. But MATCH() also works nicely for vertical lists. That means that we can find where the first occurrence of “Big Shanty Music” is in a previously sorted list of customers, where the customer name was the first down key field in a summary. In cell B10, that would look something like this:

Since we know that the customer name is the first column, we can tell Excel to look in the entire column for our search name. Yes, for the Excel-knowledgeable this is a little inefficient, and there are other ways, but this works very well when the data in your Excel file isn’t too large (and these Excel files typically aren’t too large when exporting Monarch summaries – after all, that’s why we summarize with Monarch: to shelter us from vast amounts of data).

Great, now we know how to get a column number and a row number. “But Sandy, why do I care?” you ask.

Getting the row and column numbers makes the whole thing come together and you can practically hear your data make the beautiful sounds of a full symphony. All that we need now is one more Excel function, and it takes full advantage of one last Monarch exploit. Remember the third point in the Monarch secrets? Monarch creates a named range for the entire summary data table for each exported worksheet.

When we put together a cell that gives us a row number for “Big Shanty Music” (in B10), and another that calculates the column number for February 2008 (in D8), then we can use Excel’s INDEX() function in cell D10 to tell us what the value is for Big Shanty Music in February 2008, like this:

Short, clear, and lightning-quick. Formulas with the INDEX() function calculate much more quickly than the often used VLOOKUP() function. Even better, as Monarch users we can take advantage of Monarch practically handing all of the building blocks to us on a silver platter.

You can learn much more about what to do with these Excel functions that can capitalize on your Monarch data by reading the excellent eBook “Dashboard Reporting with Excel” by Microsoft Excel MVP Charley Kyd.

I’ve prepared a little demonstration of an Excel dashboard report using some of Charley’s techniques, and some of my own favorites too of course, with the Classical Music Distributors sample report data. We’ll have a look at that next time. For now though, if you can spend a few minutes getting accustomed to these functions then you’re bound to dream up your own ways to excel with Monarch.

P.S.: Don’t forget that the “30 Days to Become a Better Monarch Modeler” training package, which details my initial experience with Charley’s book, is only available at the reduced price until March 31^st. Get your copy today!

photo credit: chrisbb@prodigy.net

We all know that Monarch can export to wide range of file formats, and that here the local favorite for Monarch exports is clearly Excel files. But do you know the three secret things Monarch does when it creates those Excel files that we can always easily exploit when working with that data in Excel?

I’ll give you a hint: it’s in the names.

Restricted Spaces

When you tell Monarch what you want to name the tables (worksheets) within the Excel file that you’re about to export to, it will create the worksheet in the Excel file with underscore characters instead of spaces. Monarch v9 will allow you to specify a name like “No Spaces”, and it will produce the worksheet named No_Spaces. Monarch v10 however will complain loudly when you tell it to create “No Spaces”, with a nasty “Illegal name syntax” error. As much as I’ve complained to the Datawatch development team about such messages in the product that were clearly designed by programmers for programmers, a few of them still seem to persist. Honestly, would it have been that difficult to actually tell the user that spaces in the table name cannot be used? Or silently convert spaces to underscores as was done in v9? And I still don’t know how v11 reacts.

This is fairly obvious when you manually export from the Table window, but it may not be when you create project exports, especially where summaries are concerned. This is because spaces are perfectly acceptable in summary names. Just remember that Monarch will convert all of those spaces into underscores when it exports to Excel worksheets. Same goes for any worksheets named after key values, if you’re exporting to new worksheets based on the key field values in a summary.

Nonetheless, forcing names without spaces is a good thing, because it simplifies the development of formulas that reference cells or areas on other worksheets. For instance, the formula that you create in cell B10 on the worksheet named “Chart_Data” is =Customer_Sales!A2 it’s very easy to read. Perhaps that’s a fairly simple example, but when your formula references multiple sheet names (meaningful) brevity is wonderful.

Easy Printing

Another step that Monarch does for us automatically for every exported worksheet is that it defines the first row as a repeating row in the worksheet’s print titles, when you choose to output field names as the first row. This actually triggers a second automatic event: Excel automatically creates a new range name for the print title row, conveniently named Print_Titles.

Home on the Range

Lastly, Monarch creates another custom range name for the data that is contained on the exported sheet, and it names it exactly the same as the worksheet name. So if the last populated cell on our “No_Spaces” worksheet is G150, the formula for the range name “No_Spaces” will be “=No_Spaces!$A$1:$G$150”.

This brings up another point. If the first row of this worksheet is deleted manually, then the range name will essentially break, and it’s formula will change to “=No_Spaces!#REF!”. No big deal, right? You’ve got the data that you want anyway. Wrong.

Because the range name is not working properly, Monarch will not be able to export to this worksheet. Future Monarch exports will fail, and the only error message that you’ll see from Monarch will be “Export failed” when you run the export manually. This is particularly disastrous if you’re running Monarch from a batch file and don’t see the failure notice. You’ll assume that everything went fine, when in fact your file will be populated with old, potentially misleading data. There’s no way to programmatically check for such errors when running Monarch batch files.

In Summary

So what do we now know about what’s happening behind the scenes in our export Excel files? Three things:

1)      All exported worksheets names have underscores where there may have been spaces in Monarch.

2)      Every exported worksheet will have a range name called Print_Titles that refers to the first row of the data set.

3)      You can refer to your exact data range exported by using the range name named after the worksheet exported.

Now that we know some hidden secrets, the question becomes: What can we do with this information?

That’s a topic for another day, and another way to excel with Monarch.

photo credit: See-ming Lee SML

Locked up tight in every single report generated by everything from your desktop accounting package to your enterprise level information system is a vast data warehouse that’s waiting for you to unlock its secrets. Monarch offers you the unique opportunity to almost magically see what isn’t there. Just like pulling a rabbit out of a hat, that’s your advantage over the regular report reader in your group.

Let’s look at seven easy to implement ways that let you stop working so hard to generate important information, and let your data work for you.

1)      Build only the templates required, and use the right template for the right purpose. Just because your report has a clear page header, you might not need to develop a page header template. Get to know the different template types that Monarch offers, as well as how to build template traps.

2)      Do not paint only the fields required right now. It’s easier to build all of the fields that the report offers now, and then hide any fields that really aren’t necessary in the current model from view. You can always save other copies of the current model with different fields displayed for different purposes. Besides, you might dream up a use for that “unneeded” field when you build your summaries (we’ll get to those). Alternatively, you can import the templates themselves into new models, which may well be more efficient.

3)      Use the Advanced field properties to do some of the work for you. It can be tricky to get Monarch to capture some data properly, depending upon how the layout of the report was designed. Let Monarch do the heavy lifting for you, but you’ll need to understand the Advanced field properties dialog.

4)      Enhance the data that you’ve captured. Use the data table features to add new relevant data by way of calculated fields, external lookups and the Address Block feature as applicable, or remove unwanted data from the selection by using filters. Sometimes this is best approach when you can’t avoid bringing in certain report data regardless of how you build your templates. Learn more about calculated fields, filters, external lookup and the Address Block features to be ready for your modeling challenges.

5)      Make good use of the available functions. There are countless uses for the built-in functions in Monarch: converting data types, extracting bits and pieces of data from larger values, and generating date and time information are just a few. When using calculated fields, use Monarch’s internal functions or your own custom functions to your best advantage.

6)      Don’t skimp on the summaries! Build many summaries in each model to make the most of Monarch’s analytical abilities. Summaries are the Monarch modeler’s best tool for analysis work. Summaries are the reason that you’ve done everything else. Don’t admire your pretty, organized data table and stop there. Use Key fields to build exactly the presentation that you want to gain new insights from your static report data. Let’s think about the Classical Music Distributor sample reports that are included when Monarch is installed. There’s a separate file for each month. Within each file, activity for each customer is reported on a different page. Orders are broken down by media type. What could we do with a summary to generate information that isn’t immediately apparent in the reports? If we’re based in the USA, we could generate different summaries for each of domestic and international shipments by media type. We could list the 10 customers to whom we shipped the most media in the period of January through April 2008. We could even break that down by period. We could add summaries isolating our best, and worst customers, based on revenues earned. We could even easily calculate the average revenue per unit shipped by customer. There are countless ways that you can gain new insight from practically any report. Get to know the tools available when building summaries.

7)      Share your findings using project exports. Build project exports to easily repeat the potentially detailed process of sharing all of data, or selected data, with others in your group. You can make it easy to send the same customized information package to others in your group on a regular basis: daily, weekly, monthly, or quarterly. Publish it on the most appropriate schedule for the information at hand. All that you will need to do will be to open your Monarch project and run all of your defined exports at the click of a single button. What could be easier? You can share the report window as PDF files or text files, and the table window and summaries can be exported as Access, PDF, or Excel (even with built-in pivot tables in the case of table data), or other formats. You can even publish directly to SharePoint sites. The flexibility offered for file creation is staggering: you can apply filters, export selected summaries or all summaries, you can create new files for each summary or create separate tables within a single document. When you do it well, it won’t take long for those who receive your customized information packages to acknowledge you as the definitive “go-to” resource who can singlehandedly produce top-notch information quickly and routinely. That’s the personal side of “let your data work for you” that is perhaps Monarch’s most important intangible benefit.

With a tool that’s as flexible as Monarch is, there are bound to be aspects of program that some of us employ on a regular basis that I haven’t mentioned above. What’s your favorite Monarch feature that you use when you want to make your data stand out?

Spend some extra time working on your models and projects and it will be clear and important to your group that you excel with Monarch.

photo credit: wrestlingentropy

Monarch’s forte is in not only capturing the data that’s locked in static report, but in allowing us to easily add vital information to that data, and to easily create different views of the data and share those views.

Just as you can do in Excel, Monarch allows you to create new calculations based on the existing data. It also allows for connecting related data to the report data that is being captured by your model using facilities known as internal and external lookups.

Many Monarch users know that there are many tools at their disposal within the program to build new ways to analyze and think about their data, but the truth is that we all get a little complacent and comfortable from time to time, doing the same things and using the same tried and true approaches to solve different problems.

Let’s have a little fun and shake things up today.

When was the last time that you did something for the first time?

I’ll guess that one of the things that you’ve never done would be creating your own program using Excel’s integrated development environment, or IDE. This Excel feature lets you create your own customized solutions for performing repetitive steps very easily. You can write little programs – you may have heard referred to as “macros” – that are just a few lines, say to insert your favorite heading into a worksheet and format it just the way that you like, or longer programs that perform a large number of steps. You do this with a programming language called “Visual Basic for Applications”, or more commonly referred to as VBA.

One of the nice things that you can do with a little Excel programming is having your Monarch work done for you. You’ll click a single button in Excel and it’ll take over and start up Monarch, load a report or two, apply a model that you’ve already built and tested, and then maybe apply a filter to the table data and export that specific data to a file. Now apply a different filter and export that data to another file. Finally, you can have Monarch export a summary or two (or more) for you. And all of this will happen with you clicking a single button in Monarch.

You may know that you can also automate Monarch using batch files, but that method has a number of limitations, notably the inability to set filters dynamically or apply custom sort orders via the command line.

Datawatch kindly documented all of these programming tricks in a “Programmer’s Guide”, but personally I find it a bit too technical for the regular Monarch user. After all, wasn’t Monarch developed in the first place to basically shield the person using from all of the technical stuff?

With that I mind, I set out to make it as easy for you to program Monarch as possible. Readers who subscribe to the free FeedBurner service, which sends you ExcelWithMonarch.com content by email, receive my custom Monarch programming tool. The tool runs in Excel and you can learn more about it here.

Watch this video in full-screen mode to get an idea of what it takes to accomplish this sort of automation:

Now, that doesn’t look too terribly difficult now does it? Trust me: you CAN do this.

Beats Workin’

We all enjoy working with Monarch; it’s a terrific product. But sometimes the routine tasks get monotonous, and highly detailed and meticulous work and processes can be very stressful as there’s usually a fair bit of pressure to make sure that you get the end result right. Every time.

Do yourself a huge favor and devote some time and effort to develop what might be completely new skills. Nobody expects you to be able to create a huge reporting system for your first programming project. Start small. Pick something that you can do fairly quickly with Monarch, and try to reproduce those steps with your own custom Excel program.

There will be a bit of a learning curve, but take the time to learn the basics and build from there. Google is your friend for learning VBA. And you can’t go wrong with having a good book or two on your desk.

Then when you’re a bit more comfortable, start working on a larger project. There’s nothing better than watching your program do your work for you, say, while you’re sitting back enjoying your morning coffee.

We’re not quite to the point of having a robot do our work for us, but by automating your work in this way you can easily share your new versions of the truth with others in your group and it’s the best way to excel with Monarch.

photo credit: Steve Keys

I feel – and understand – your frustration.

Over the last few weeks, I’ve noticed that many visitors to ExcelWithMonarch.com have arrived by using their favorite search engines to find the elusive Monarch training book. They search for “Monarch for dummies”, “Monarch book”, “Monarch training book”, “learn Monarch guide”, and other variations of the same intention.

They all learn the same sad fact, though: that book doesn’t exist. And therein lies an interesting a story…

Meet Mr. Bond

I’d like to introduce you to my friend, Olly Bond, who you may have met as OllyinMunich on the Monarch Forums. He’s worked for Datawatch and for Datawatch customers, and now runs his consultancy www.greenbar.info. You might also have seen the “Invisible Data” webinar he presented for Datawatch on the advanced report functions of Monarch.

After having provided support to fellow Monarch users around the world on the Monarch Forums, at one point it became apparent to us that there were a large number of people who were looking for a book on Monarch. Many of the forum members complained that while they were interested in taking Monarch courses because they’d not yet had any formal training, they found it difficult to have their management approve of such expenditures. The most common reasons that we saw included the total costs of the training, and that they, the students, would be away from their work. They typically worked in a small group that couldn’t support such an absence, regardless of the future benefits.

Then there were those who wanted a reference book, and those who wanted to learn how to use the software on their own schedule.

It was apparent to us that there was a need for a Monarch book, and, after much discussion, we committed to developing it. We did everything that had to be done:

• We contracted a literary agent, who would help us navigate a business in which we beginners were unfamiliar.
• We prepared proposal documents for our book using the templates supplied by the major publishers.
• We had the commitment of Datawatch management at the time to assist with the project, should the publishers have wanted that involvement.
• We developed a detailed timeline and the initial chapters, all of which showed that we could deliver the book for the publishers.

With both of us having extensive experience with Monarch, and Olly’s experience with Datawatch’s Enterprise level solutions (all of which happen to use Monarch technology at their core) and my experience in developing automated solutions for Monarch, we believed that we’d be able to put together the definitive Monarch resource.

The project started off surprising well, and we quickly learned that our proposals had landed on the desks of representatives of several publishers who each were responsible for “pitching” book ideas to their respective review/approval committees. “Great, things are progressing nicely,” we thought.

But over the next weeks – it was shocking to learn how slowly these things move in some cases – our agent continued to inform us that the next publisher had opted to not proceed. The story was always the same: “The market’s too small.”

Who’s Protecting Whom?

Really? With well over 40,000 customers (including many Fortune 1000 companies) using Monarch software. Many of those customers have many users of the software, and you don’t think that enough users can use a good reference book? Well, again, we don’t have experience in your industry. Maybe you’re correct.

But it strikes me really funny that I can walk into most any bookstore and find volume after volume dedicated to very narrow market topics, and specific to computing, books on software that I’ve never even heard of, much less have ever used.

So what really happened? Of course we’ll never know for certain, but I do find it rather odd that the same publishers provide all kinds of books on the same general topic, that of Business Intelligence (and all things related), for applications sold by the big, long established BI tool providers.

Can you imagine the chaos that would ensue should a large number of people, people who just want to get on with their real jobs, learn how easy and inexpensive it is to get the information that they need? Surely there would be madness in the streets!

Alas, Olly and I realized that we weren’t going to win that first round of the fight. Subsequently, I convinced my partner that I still wanted to help the struggling Monarch modeler improve their skills, but without detracting from our still-not-yet-ruled-out Monarch book. That led to the “30 Days to Become a Better Monarch Modeler” training package, which is available until March 31^st 2012 for only $47 USD, after which time the price returns to $97 USD.

What would YOU like?

While I’m pleased with the feedback that I’ve received from those who’ve purchased my training, I have heard from many who want more of a step-by-step approach. The question then becomes: what is the best way in which to deliver such material? What would YOU like to use? An eBook? A printed book? Maybe videos? Something similar to my recent YouTube channel post perhaps, only which would provide much more detail? Maybe something altogether different?

Drop me an email, or leave a comment below. There are many ways in which we can all learn to excel with Monarch!

photo credit: pat7047

I’ve just come to learn of some very sad news.

A couple of weeks ago I noticed that the image for the MicroCharts product that I was linking to on the right side of my pages wasn’t appearing. I sent email off to BonaVista Systems asking if a replacement was available, but I hadn’t received a reply. Thinking that this was highly unusual, this evening I thought that I’d follow up, and visited the BonaVista Systems site, only to find the following:

I remembered that Andreas Lipphardt, who ran the company, had worked with the group at XLCubed.com, and there found the news that he had recently passed away. (I trust this information to be accurate, and have no reason to believe otherwise.)

For those who had purchased software (including MicroCharts) from BonaVista Systems, per the aforementioned news link, you’ll now want to contact XLCubed.com for any support issues that you may have. “Should MicroChart customers have issues then as a tribute to Andreas’ contribution to our company we will attempt to assist if and where we can.”

I tested my link to the page that let customers purchase the MicroCharts software, and it seems that the e-commerce provider that BonaVista Systems was using will still sell you a copy of the software. But knowing that support for it is probably not available in the long term, I’ve accordingly removed the links to purchase MicroCharts software from my site.

In memoriam.