The ability of Monarch to sort key fields in both the Table and Summary windows is tremendously useful as it enables us to further analyze our data. The ability to focus only on the most important data can be critical when working with data sets of any size.

photo credit: sam_churchill

Let’s assume that we have a list of 36 records reflecting banking transactions. We want to create a list showing only the largest value each transaction at each of the branches. In our data set, there are 12 transactions at each of three branches. Once the entire data is filtered, we expect to see only three records: one for each branch number, with the largest transaction amount that occurred for that branch displayed.

Monarch makes it easy to sort records on multiple key fields. In fact, within a single Monarch model, one can define multiple manners in which the data will be sorted at any given time as is demonstrated with the Sort Orders dialog box.

Users of Monarch v10 and later have an advantage over users with earlier Monarch versions when filtering data that employs a defined sort order, as they can build filters that take the active sort order into consideration.

By utilizing the Advanced properties of the filter definition, we can specify that we need to handle duplicate values for a particular field carefully.

By activating first the custom sort order and then the new filter Monarch will list only three records in our sample data set, as expected.

Notice that the option for duplicated rows reads “first row only, as sorted”. That means that we’re effectively restricted to only a single row when duplicates are encountered, as is the case with our 12 records per branch.

Note that we’ve selected the “Unique Rows” option in addition to the “Duplicated rows” option. By using this option we ensure that we always have a representative record for every branch location. This would be necessary if in our set of 36 records we had a distribution of 20 records for one branch, 15 records in another and only one record for the final branch.

Top N Records

What would we do if we wanted not just a summary of our data but rather a complete list of all of the fields available for the top, say, 10 records for a given key field? Monarch doesn’t make it easy (in that an option display such data readily doesn’t exist) but for Monarch Pro users it is possible.

The goal of this challenge is to filter the Table window to display only a particular number of records for every key item value. Employing summaries are effective for drill-up and drill-down solutions, but not for listing the top n detail records for all key values. We’ll need to build a filter in the Table window to satisfy this requirement.

First, build and apply the custom sort order as above. Next, add a new numeric calculated field named SortRowNo using only the expression Rowno(). We’ll use this field as a key part of the solution. Astute readers will be quick to point out that filters cannot be built when any component of the filter employs the Rowno() function. Monarch will complain loudly.

Instead, assuming that this is a process that would be repeated regularly, you’d define a project export to export the unfiltered Table window to a temporary file (Excel workbook, Access database table, or what have you, as applicable). Save the model and project files, and run the project export.

Now in a new Monarch session, open the freshly exported file as a database source. Import all of the fields. Now that the SortRowNo contains only values, and not the Rowno() function, we can use those values in a filter. But we need one more value before we create that filter.

What we’ll do next is a little odd. We’re going to create a new external lookup. The data source for the lookup will be exactly the same file that we just opened, and the key field will be, in our case, the branch identifier. Monarch will complain about the specified field not forming a unique key to the external table. That’s OK. Click the “Yes” button and continue to import the SortRowNo field as LookupRowNo.

When the external lookup is performed, all equivalent branch identifiers will be assigned the same value for LookupRowNo: the row of the Table of the first model in which that branch first appeared.

Now with a complete data set, we can define a filter which will result in the top 10 records that we’re after. For the filter use the expression:

LookupRowNo – SortRowNo < 10

Build in Flexibility

By replacing the fixed value of 10 in the filter formula with a new runtime parameter calculated field (named rtTopN for example), we can make the model much more user-friendly and flexible. Upon opening the model, Monarch will prompt the user for a value to be used in the filter.

Additionally, values for runtime parameters can be passed to a model via a batch file to make the process even smoother. Imagine a business process that monitored the top 10 records each week, but only the top three items were required for monthly reporting. A batch file could be created and copied with only a slight edit in the copy to change the value for the rtTopN amount. Use the /set command line parameter to assign a value to a runtime parameter calculated field.

Ignore the Restrictions

This approach demonstrates that we can sometimes circumvent the technical restrictions that Monarch imposes to build solutions that resolve the challenges that exist in your information systems, whether they’re large or small.

Earn a spot in your organization’s Top 10 list. Excel with Monarch.

Auto mechanics have cabinet after cabinet of tools, as do carpenters and plumbers, surgeons and dentists. Every tool has a special use and its shape and purpose has been refined over many, many years. The physical space required to house these collections of tools can be tremendous.

photo credit: Watt_Dabney

The other night I attended a performance by the country rock greats the Eagles, who are touring in support of their Long Road Out of Eden CD. I’d not seen them before, but I’ll definitely be there if I get another opportunity as it was a superb show. Don’t miss them if they come to your town.

Of course, musicians have their own tools too: their instruments. Some have more than others, and the boys in the Eagles must have dedicated several of their equipment trailers just to store their guitars as they each changed guitars after almost every song!

Monarch Pro: The Swiss Army Knife of BI?

It made me thankful that I don’t need a huge collection of different tools just to be productive. All that’s required is a little computer system, access to a data source such as a large ERP system or even a much smaller accounting system, Monarch and Excel. With those few tools, the possibilities for developing customized business intelligence and the ability to turn raw data into real information are just about endless.

Monarch is an amazing part of the complete toolkit, because it’s a collection of tools in and of itself. How many other standalone tools provide the complete range and flexibility to:

• Acquire data from just about any external source
• Convert unstructured data from existing reports (that can have almost any layout) into structured data in seconds
• Add new calculations to the acquired or converted data
• Create and manage ways to sort the data based on multiple levels
• Create and manage ways to filter the data in practically any way imaginable
• Summarize massive amounts of data and aggregate values in ways which satisfy most every need
• Send data to every common (and some not-so-common ones too) electronic file format in use today
• Provide tools to audit every single modification that’s been made to a data acquisition and manipulation model
• Save duplication of effort by easily reusing data tools that had already been created for other purposes, and even
• Prepare charts to help visualize your data

All of these can be done without any significant or special knowledge about information systems and without any knowledge of programming or query languages such as SQL.

But if you are able to write program code, or even small scripts, you can do yet more with Monarch by automating the work that you would do with it manually.

When you add Excel to Monarch, you add the final layer of the package. Excel isn’t the tool that transforms Monarch from being a lump of coal into a beautiful diamond, but rather Excel is the gold ring that secures the Monarch diamond and makes it easy for others to see what to them is Monarch’s hidden beauty. Excel is what enables you to polish the presentation of your work before you share it so that you and your efforts get the attention and recognition that’s deserved.

Monarch’s Tools Constantly Improve

The features that are built into Monarch continue to evolve with every version of the software, and in between releases we can combine our creativity with Monarch’s existing flexibility to create new tools to make the most of all by ourselves.

As much as I’d like to have a truckload of stunning guitars to play, I’m happy to say that my tools are much more portable, even if they don’t sound nearly as nice. Still, your work can be just like beautiful music when you excel with Monarch.

photo credit: fotographix.ca

Have you seen some of what is soon to be available in the upcoming release of Excel 2010? Microsoft has produced some videos to demonstrate the new abilities of the software.

But before I share those with you, I thought that I’d include their “History of Business Intelligence”. While a little lengthy, it’s cute, entertaining and informative, and is a pretty good recap of what’s happened with this topic to date. Watch the video to see how the Terminator, Donkey Kong and gophers played a role in getting us to where we’re at today.

History of Business Intelligence

Sparklines

Sparklines are small charts that help convey trends. Many Excel users have been taking advantage of sparklines by using tools such as BonaVista System’s excellent Microcharts, but Microsoft has addressed this need and deficiency in their software by incorporating this visual analysis tool into Excel 2010.

Slicers

Other products such as Tableau have been making it easy to dynamically filter the data included in tables and charts for a while, and slicers are Microsoft’s version of more interactive filtering. Up until now we’ve been able to filter content with Autofilters and pivot table page fields, but the major improvement offered by the new slicers is in how the secondary filter levels change dynamically based on the initial selection. Have a look:

The Impact of Excel 2010 on Monarch Modelers

Monarch v10’s ability to export summaries, especially when using key values to create individuals worksheets, in conjunction with sparklines will make quick work of producing really useful trend charts in small spaces, maximizing the impact of your dashboard reports.

As Monarch modelers and solution builders, depending upon the requirements of the audience, we might be able to spend less time modeling – building fewer filters up front (in the model) – and instead take advantage of the large storage capacity of Excel (and Access) and have the end users filter their data on their own. Using slicers seems to be pretty straightforward, though your Excel users, at least initially, might appreciate your setting up the worksheet so that they can just click.

Based on how Monarch works with Excel 2007, I’ll assume (I know, I know… yes, I am speculating here) that being able to export data from Monarch to Excel 2010 files will be just as straightforward as it is with Excel 2007 and that we’ll be able to export pivot tables and take full advantage of slicers and sparklines, and the other new features that await us in Excel 2010.

Monarch and Excel Continue Working Together

As long as Excel needs structured data to analyze, then Monarch will always be the tool of choice to:

• organize unstructured data,
• consolidate data from multiple sources, and
• make that data accessible to Excel, either directly or indirectly.

Have you used the Excel 2010 beta software? If so, what were your impressions? I’m interested, clearly, but decided to focus on other tasks rather than test beta software. Nonetheless, by the looks of it, Excel 2010 will open up even more ways for us to excel with Monarch.

Today’s post is the final contribution – for now – from UK-based consultant Grant Perkins who kindly offered his assistance while I was ill.

I can’t speak for everyone of course but personally I find that something that interests me is much more rewarding to work with and through than something that does not interest me.

Now we can’t always pick and choose what we have to work on but if we wish to practice to discover new skills or enhance existing skill doing so working with something of interest is likely to be more rewarding overall than spending time on something of little or no interest.

photo credit: tiffa130

Now I fully appreciate that some may not agree with me here and even those who do may have difficulty persuading their employer that spending time with their corporate version of Monarch pursuing personal interest projects is going to be of benefit to the company!

However if you are lucky enough to have access to a copy of Monarch, yours or a company copy, with which you can spend time on personal interest projects then I would whole heartedly recommend doing so if only for the variety of functionality you are likely to need to use. It’s a great way become familiar with features you may be able to apply in you work but had never had the opportunity to discover why.

Plus you may find some new ideas for analysis to suggest to your employer.

I know of at least one personal project that involved analysis of TV schedules for recording purposes. (As I recall, though I may be corrected about that.)

I do quite a lot of photography and use an editor that saves its edit information in a jpeg file along with text based information about where the original file resides. There are times when I find it necessary to move files (which the reference information can deal with if both files still reside in the same folder) and rename them (which will break the link even if they are in the same folder.) It’s possible to re-establish the connection, even to a different folder, provided you know which was the original file and where it is. I can use Monarch to quickly access the cross-reference information from the edit files and list links to make life easier. (Even better would be to allow the cross-reference to be corrected and then re-write the file but of course Monarch is not expecting to be saving jpg files!)

It occurs to me that there may be similar not-so-obvious applications for Monarch within a business environment.

Another useful source of experimental data is, as you might expect, the internet. Many organizations, especially government organizations, are allowing direct access reporting. Some private organizations too – banks allow you to download your account transactions for example, telcos offering electronic access to billing and so on. There may be some interesting analysis to be taken from those sources.

The UK Meteorological Office makes some long-term weather measurement data available for a selection of sites on a site-by-site basis. The data for each site can be downloaded as a text field and modeled with Monarch. There are a few format challenges to deal with – good practice for a modeler.

Once the files are downloaded they can all be opened and a large table of results can be obtained offering excellent opportunities for manipulating the various fields into summaries and graphs and deciding whether the weather has done anything odd from time to time in the last century or so.

Another Government site offers a fairly comprehensive report generation front screen for some of its database to do with Renewable (Energy) obligations Certificates. For full details one needs to grab another file from a different part of the site and then link the two together – good practice for external linking.

Once linked the potential for observing the progress of Renewable energy development, geographic location, its rated capacity and how much electricity is provided month by month becomes apparent, though not without a few challenges that require some calculated fields to ‘normalize’ the data and deal with a few anomalies. And of course such challenges are all good experience for a Monarch modeler with potential for spill-over of the techniques used into full business benefits for an employer.

I could imagine that other people might be working on share value reports, property listings, football statistics, etc. Any number of interesting projects that help modelers to enhance their skills and knowledge and provide insights that could likely benefit an employer.

What possible objections could the boss have?

So who else has their own pet projects that Monarch makes easy (or could do if it was available)? Post some examples in the comments and let’s see where that might lead us in the discovery of new ways to excel with Monarch.

I need to say a huge “Thank you!” to Grant for his time and efforts and for continuing to share his passion for Monarch with us.

I’ve been known to use Monarch on disk directory listings to assist with finding files on large volumes quickly and in the past have impressed my realtor with databases created from PDF files that he’d supplied. As Grant asks, what kinds of non-traditional tasks have you come up with to excel with Monarch?

UK-based consultant Grant Perkins continues to share his insights with us while I do my best to fight back against a touch of pneumonia.

As users will know (I hope!) Monarch offers a broad selection of Functions to help us extract information from a wide range of data sources and then shape it into the format required for further use in analysis or presentation. The list grows with every release and in recent versions the potential for people to be able to create their own “User Defined Functions” tuned to their particular regular needs has greatly extended the concept and aided productivity.

photo credit: Marcin Wichary

Some functions we will use pretty much every day, other so rarely we may almost forget about them or, worse, not be fully aware of what they can offer to make our lives easier or enhance our value to those authorizing our income payments!

So I thought a short list of some of my favorite irregularly used functions and features might spur others to suggest list of their own with the intention of providing everyone with a focal point for appreciating how they may gain by using tools with which they may not be familiar or, perhaps, in ways they has not considered.

Let’s start with functions.

TEXTLINE()

TextLine was introduced a few versions back and then quietly enhanced with an additional option that can be very powerful but may not be in everyone’s toolkit.

TextLine allows you to take a block of text derived from a multi-line field and split it into lines by specifying which line number you want. So far so good. The benefit is that Monarch, for database efficiency reasons amongst other things, will tend to ignore the format of the text and create a new concise display of the characters. However, sometimes the original format matters and so TextLine allows us to present individual lines or reconstruct the original text in its original lines.

What may be less well known is that TextLine also has a SEARCH mode. This allows you to show the content of a line if and only if the line contains a search string that you specify as a parameter for the function.

For those readers already familiar with the concept of the ‘Preceding String’ in the advanced field properties the TextLine SEARCH mode is quite similar in use. The obvious difference is that the search string will be embedded in the line rather than preceding it. (For anyone not familiar with the Preceding String feature …. you really should have that one in your tool kit. I encourage you to check it out.)

TextLine Search mode has many uses but perhaps the most common use is for financial type reports where encoded transactions appear as lists but the codes can be in any order. So it’s no good mapping line 1 to the field for the code “AA” since the value contained my not be AA’s value and, indeed, may not even be in the right format. (The latter situation would at least be likely to create an error to let you know there was a problem!).

TextLine with SEARCH mode would allow you to specify that you want the line that contains, for example “AA:” in the first 3 characters (or maybe position 3 to 5, or whatever you can define to make it unique) and so will populate you calculated field only when it finds that line somewhere in the target multi-line field extraction.

This is a very simple example and you can make the function perform much more complex searches by feeding in different variable parameters – say a reference to another field maybe, or a User Entry field provided when the model is run.

Why not take some time to experiment with it for a few minutes so that you can see and memorize its potential for a future model?

The SPLITS

I had been using Monarch for quite a while before something I read helped me to stumble across the real potential for using the SPLIT functions. These functions, variations on the same functionality, can be incredibly powerful and are usually very easy to apply, though some reports may introduce special challenges that require slightly more thought and effort.

NSPLIT is a specific Name Splitter that can be considered to be of rather specific application but the LSPLIT and RSPLIT functions are more widely applicable and provide easy to use facilities for dealing with reports that are not consistent in presentation or any sort of variable content input field that needs to be broken out into separate component parts.

Great tools and in my view a very important part of the Monarch modeler’s armory.

DTOC and CTOD

Or, in extended form, Date to Character and Character to Date conversions.

Once again these tools allow you to work in ways that direct working with a date of character field will not. One obvious use is for enabling lookup activity between sources that present date information in different ways but avoiding the need for extra work to convert one or other source file into the preferred format before making the connection.

A hidden calculated field as part of the model can make the link work for you.

Likewise if you have a specialized character based date input and wish to display in the same character format on output but perform date based calculations – like elapsed working days for example – you can convert to date format, calculate and then convert the result back to your date needs using hidden fields for the calculated part of the work

Being familiar and confident with these functions can make working with Monarch so much more productive.

Compound Filters

It’s usually relatively easy to build and test a simple filter but working on something complex can be a nightmare, and also may make it difficult to check the results.

So don’t.

Create simple filters that you are comfortable with and can be checked easily – and on sequential sets of data extractions if necessary so that you can see the effects as the filter build up one after the other.

When you are happy with the individual filters make a Compound Filter from all of them using the original stand alone results to test the final outcome.

The secondary benefit of this is that you and up with a known good set of single filters ready and waiting to be included in other Compound filters in many different combinations.

Better yet, if any of the single filters need to be changed (and the old version discarded for any future use) you only need to change one filter definition – all the compound filters that use that single filter will be updated.

(That said be sure that you fully document things for the future and if you work in an environment with shared models and multiple modelers you will need to be certain that you have some controls in place for the way the modeling ‘team’ operates …)

Filter by Value

This is a recently introduced feature that offers tremendous potential for specialized filtering work whether as the only filter applied to the data or as an additional filter used on a previously filtered subset of data.

From the modelers point of view it is probably the quickest way to see what values a field holds and of course to identify any and all records that do not have a value for the field.

By using the “Add current Values” button you make dynamic filtering (i.e. the values in that report) a possibility with virtually no effort. By ticking the option that will filter only unmatched values you can seek out anomalies compared to a desired list. Include empty or NULL value rows can help quickly identify records that might contain errors. All within the existing filter parameters of course.

Bear in mind that if you need a quick list of all possible values in a field this may be the way to get it. Un-filter everything, auto-populate from the table and the click in the top left hand corner of the 2 column display box. The box will turn black. Control-C will copy the entire 2 columns to your clipboard. (Sort the second column first if that is useful – just click the bar at the top of the column to flip the sort order.)

I’m sure that we will see a number of excellent uses of this feature as an output benefit presented on the forum as time passes. But for me the potential of Filter by Value as an assistance tool for model development to make life easier for the modeler is what stands out right now.

Summary

We all have our favorite ‘things’ and ways of working and to constrain people to specific approaches is more than likely to stifle creativity and stunt the options for problem solving. However at an individual level, as a Monarch Modeler, having comfortable familiarity with the tools available can make individual tasks that much more satisfying. Favorite tools, falling immediately ‘to hand’ and being usable without having to work out what to do with them each time you pick them up are important in our lives. I would encourage everyone think of the core tools they need for their Monarch activity in these terms. If your needs are similar to mine then the list above may be a good starting place from which to fill the ‘favorite tools’ drawer.

By my calculations, Grant has offered some tremendously valuable ideas. What are your “go-to” functions, features and/or other approaches that you use to excel with Monarch?

This is second in a series of contributions from UK-based consultant Grant Perkins. Today Grant offers some sweet ways to enhance your contribution to your organization.

How many times have you been asked to provide some information or an analysis from a new source and thought: “I have all that in my old sources”?

How many times have you thought, “They won’t listen, but I could really give my bosses something useful here that would help me AND help them”?

photo credit: ajagendorf25

Maybe they won’t know to ask and maybe they don’t like to be told. So, for everyone’s benefit, is there a way to allow them to ask for the information you know you can provide? What diversionary benefits can you present that guide them your way?

There are those who think – and this is by no means a new thought – that many businesses are failing to get the most, sometimes any, real benefit from their investment in the potential of their ERP and CRM systems.

From what I have seen over the years it is hard to disagree but that is not what I set out to discuss here. I think we all know that most business systems are able to offer more, by one means or another, than we use them for. But what can we get from them for no additional effort? Indeed can we get more for less? And if so can we make a pitch to the ‘upper levels’ that guides them to support your initiative by making the decision based on less ‘challenging’ questions that have an obvious and positive response?

One or two reports give all?

Some years ago I worked for a company that wrote and supported business software for service organizations, primarily the sort that sent technical people out on site visits to fix things.

The system we provided was modular and covered a lot of business operational territory. The core modules, common to just about all clients, managed their contract database equipment, locations, terms and conditions and recurring billing amounts in one module and their field activities, worker management, service level reporting, cost capture and some more billing opportunities in a second module. Some files were common between the two as you might expect. Mostly clients used both modules but not always to their full capability. Indeed it is probably safe to say that they were rarely used to their full capability.

We wrote hundreds of reports often with many and growing options for data selection and sort orders in an attempt to satisfy all the varieties of report ‘views’ individual clients perceived as ‘must haves’ without which they could not run their businesses. People still asked for report writers – despite some of the calculations being so complex to generate outside a piece of dedicated code that they had not a chance of getting it right. That potentially very bad news did not stop them trying though…

Having discovered Monarch – and with lots of greenbar reports to play with – I suddenly realized one day that all I needed to provide almost 100% of the information that most people wanted (or at least asked for) on a daily basis could be provided by Monarch from one of two system reports provided those reports were run for only date range selections and ignored all of the other complexity we had built into them. Even the date range selection was only required to pick relevant data – I could have taken the whole database (within reason) and used Monarch to do the individual analyses we were providing and distributing.

All the work to re-create the performance measures in other places was no longer required – the reports did that for me. Plus I could re-group and re-sort data any way I wanted it building up from the detail records or starting at more of a summary level to start with.

But the clients had, by then, already got what they thought they wanted. How would this be better?

Well to start with the existing processes were often specifically developed in house and required initial effort and continuing maintenance and support. So I could pitch a cost saving and process risk reduction by making it all simpler.

Secondly all of the overnight extractions for the reports used a lot of processing time (as did the printing using processing and, back then, paper). User run reports during the day could be even more disruptive. Running 2 reports with simple selection criteria (dates) was faster and reduced the overhead and for many needs eliminated the idea of the users running their own reports. We could move that task off the main server and on to the Office PC network, using Monarch (or, these days, perhaps a Data Pump server) to produce the outputs from the re-analysis of the overnight extraction.

So the argument would be for doing more for less. A small investment in a Monarch based front end would save recurring amounts in the server farm investment and operation. Would that be an easier proposition for the decision makers to understand? That was the primary benefit. The secondary benefit (but primary for you!) was how much more analysis you could offer as the sugar on the dessert.

So, two reports to get it all. But to see the full potential you may need to be at least aware of the potential capabilities of all the Monarch tools that allow you to tickle data out from even the less obvious routes – more to come on some of those subjects …..

When he’s not experimenting with traps and templates Grant enjoys a good coffee along with an occasional treat, all the while dreaming up new ways to excel with Monarch.

Somehow I’ve managed to wind up with case of pneumonia, and thus my recent absence. While I continue to fight it off, UK-based Monarch expert Grant Perkins has kindly volunteered to share a few of his ideas over next few posts.

Here’s Grant’s first submission.

Back in 2009 The Register ran a series of reports by Dale Vile of FreeformDynamics that presented, among other things, the result of some polls of IT industry workers regarding how well they thought there employers were making us of their IT systems.

The last one in the series caught my eye – “The Unrealised potential of ERP and CRM” – which carries the sub heading “Digging up that buried treasure”.

photo credit: LaPrimaDonna

Now to me that strap line reads like a perfect introduction to the potential for involving Monarch and the Datawatch family of applications.

I can’t easily summarize what is already a very trimly presented report and therefore encourage you to have a look for yourselves. It’s only 2 pages and has some graphs that get the message across very succinctly. The message seemed to align very closely with my own experiences as a vendor consultant for business applications over the past three decades. How do they fit with your experiences?

I’ll pick a few headlines.

10% of respondents thought that the ERP/CRM systems they were involved with were fully implemented.

Around 75% thought that “More value could be unlocked” by “Making better use of the information held in the system”.

Even access to the systems and their information, whether from corporate offices or in more mobile situations, is reported as being less encompassing than it might be.

Now whilst some of the benefits mentioned by respondents involve taking steps to make better use of the software (in other words complete the implementation as originally envisioned – or at least get closer to that vision) by moving from desk top based ‘office systems’ (Masses of small databases, word-processed documents and spreadsheets) – much of the benefit comes from being able to get at all the collected information and do something with it.

Thus the interest in “BI” in recent years.

The report’s closing line;

“… in many cases, a significant amount of additional value can often be tapped into with a relatively modest level of incremental investment.”

That sounds good (and familiar) to me. Where to start?

Well, given that the poll suggests that visions from above often fade before full benefit can be obtained …. how about we start ‘at the coal face’ and promote ideas upwards in the chain of management?

It may work in some companies and not in others – but it has to be worth a try.

Monarch users may be in a strong position to develop and offer benefits to their bosses, their company and therefore themselves. All that is required is enough business understanding to spot the best areas of need for which you can provide a solid solution. Some may be obvious, so go for them. Some may look challenging. The need is there but the method of delivering the result may not be obvious.

That’s where awareness and a wide exposure to the potential within Monarch – even if you don’t use that potential on a regular basis or, indeed, have never used it – can be the catalyst you need find a way forward.

In future posts I will offer some suggestions that might fit into this area of murky knowledge for some users, not so much to provide packaged answers but more to offer ideas to be stored for when the time is right.

Meanwhile I really do encourage readers to read the report linked above and consider how the observations and outline conclusions fit with your own experiences.

Grant enjoys being a professional photographer at a racetrack when he isn’t finding new ways to excel with Monarch.

After much development, and overcoming some all-new learning curves, I’m pleased to launch ExcelWithMonarch.com version 2.0!

photo credit: Jule_Berlin

I’d always had some challenges with the old look and feel, but it was the best that I could do to get the site going. I’ve been working on this change for some time now.

This necessitated a pretty major upgrade to the Wordpress platform, which honestly made me exceedingly apprehensive. The last thing that I wanted to do was corrupt the site and possibly lose content. The biggest worry was that something would go wrong with the required structure changes to the new Wordpress database. My worry was for nothing though as I carefully went through the upgrade checklist and thankfully everything went pretty smoothly.

Also new to me was the implementation of the Thesis theme for Wordpress, which allows folks who aren’t experts in web design (like yours truly), to gain full control of the design of a site and enable customization of the layout and presentation reasonably easily.

I’ve tested the changes pretty thoroughly, but if you find that something isn’t working, please take a moment and let me know.

I hope that you find the site easier to use and read. I find that the text much cleaner and easier to read, the overall feel is lighter, and the new inclusion of images should make for a much more engaging experience.

I’d not included many images in the previous posts because the old layout just didn’t really accommodate much more than small pictures, which reduced the effectiveness. This new presentation should allow for some terrific new content (he says hopefully).

To make my own life a little easier, the site now uses Captcha when supplying new comments. I hoe that you don’t find this to be a burden, as it was really quite necessary. It should reduce required management of the spam load; it’s driving me crazy. Don’t these guys have something useful to do with their talents? Honestly…

A big new feature on the site is a sitemap, which lists every post available by category. This should benefit new and current readers alike.

Another improvement over the last version of the site is the custom Google search. This should provide much better search results when looking for content that exists on the site.

Welcome to the Future!

The new platform will enable me to tweak the design and add new aspects to the site much more easily than before. This should make for a better site for both you and me, and ought to provide a richer experience as we continue to excel with Monarch.

With its unique abilities and affordability, Monarch software is well placed to assist those working in the healthcare industry, an area in which I must say that I don’t have much firsthand experience. But while information is information, and data is data, and I hope that much of what is available here is both useful and applicable, there’s a certain something about having direct involvement in an industry.

photo credit: Edgar Zuniga Jr.

On that note, I’m very pleased to welcome Mr. Joe Berry as a guest contributor! Joe’s entire career has been spent in the business offices of hospitals, and he now shares his knowledge and expertise by providing consulting services to his hospital clients.

Please join me in welcoming Joe to the site, and enjoy his story, which I’ve titled…

Seeding Information Solutions for Healthcare

This week it has been unseasonably warm – around 65 degrees – and sunny in central Illinois. Even though it is a little early in the season, it makes me think of gardening.

Gardening is one of my favorite pastimes. I spend time each year planning the types and varieties of vegetables that I will grow. I decide where I will plant each variety and how much seed to purchase. The planning helps me make sure I have enough room for what I want to grow and helps me reduce the possibilities of disease by rotating my vegetables. Planning makes me a better gardener.

What does gardening have to do with excelling with Monarch? Like gardening, a good Monarch model/project should include a good plan. Planning not only helps us be better gardeners but also helps makes us become better at creating actionable data.

I have worked in hospital business offices for 35 years. During most of those years, I have produced reports to analyze data and improve the quality of processes. Currently I am an independent consultant helping hospitals get the data they need to make data based decisions.

I find the reporting capabilities of most healthcare information systems to be somewhat limited. They usually have many standard reports and a simple Ad Hoc report writer, but few have the ability to provide in-depth analysis, dashboard, or flash reports.

Excel and Monarch work well in healthcare. Alone, each is very powerful, but together they offer a vast array of solutions that do not require the assistance of IT professionals.

Define the requirement

I was recently consulting for rural hospital. The Chief Financial Officer wanted a way to summarize their payroll data – a one page flash report for each department. The data would be stratified by pay code with totals by pay period and year-to-date. He also wanted the flexibility to be able to add budget totals in the future. His request was to have the output in Excel using one workbook per pay period. Each department would be reported on its own sheet inside of that workbook.

The CFO shared a copy of the spreadsheet he had created as an example of the final report. This really helps me to get a visual of what the end result should look like – an expectation from the user. This jump-started my planning and gave me a map to follow. This is a huge help and a time saver.

From my review I noticed that the formatting of the report was much more than basic and included bolded text, underlining, line drawing, and inverse printing. At this point, I had no idea how I was going to replicate the formatting, but I knew that Monarch and Excel were up to the task of gathering, manipulating, stratifying, and reporting the data.

Determine the data sources

We spent some time discussing what source data he used for his report. He told me that he had obtained the data from several different reports, from two staff members. He gave me the details and reasoning for his formulas, and explained that his sample report had taken over ten man hours to produce. While his sample report was for only one department, his hope was that automation could separate the data by department and make it cost effective to produce this valuable report.

Since I had very little experience with the payroll data from this hospital’s information system, I needed to browse the manual, talk to users, and get to know the system in order to establish what data was available. Determining the best way to gather data is part of a good plan. What reports or data sources would contain the data needed and what calculations should be used? What filters were needed? Were any summaries going to be used and what was going to be exported?

Set the schedule

Another part of a good plan is a timeline. The CFO was very flexible with this. As part of my plan I would need to determine what was realistic and build in a couple of emergency buffers. My goal is to always exceed the expectation, so a realistic goal with room for emergencies is vital to maintain that goal.

Gardening requires many steps in the plan. For example a gardener must prepare the soil, fertilize, sow the seeds, cultivate, and harvest. With this report, it was apparent that multiple models/projects would be required to get the data in the format desired with the desired totals and stratifications.

Document, document, document

After talking to different staff members and reading a few pertinent pages in the manual, I was ready to start documenting the plan. I determined that it was going to be easier to create the year-to-date totals as the year progressed instead of running multiple reports to get that data on an ad hoc basis. Interestingly, the hospital’s information system did not have a single report with totals for hours or dollars by pay code.

Indentifying each step and along with what needed to be accomplished, I determined the need for eight different models/projects. I usually create a project with each model as it makes it easier to add exports later if needed.

I also separate complex tasks into smaller, easier to manage models. Since most of what I create will be used by others, this practice helps with subsequent revisions and improvements as they arise.

Build a process

The department name was not available through the Ad Hoc Report Writer, but it was part of the sample report. Getting it was an easy fix. First, create a simple model (Model1) that would trap the department number and the department name from a standard report. Now export those two data elements to Excel (Payroll1.xls) and use that file as the source for an external lookup.

Another bit of missing data was a pay period calendar: a file that has the calendar year, the beginning date and the ending date of each pay period along with the pay period number. Again, there was a simple solution. First, create a model (Model2) to trap the needed data from a standard report, and then export the data to Excel (Payroll2.xls) and use it as an external lookup data source. This file would only need to be created once per year as the pay period calendar never changes during the year.

Two different Ad Hoc reports from the hospital information system would be needed. Each report was for a single pay period. One would contain hours and dollars by pay code for non-productive hours; such as, vacation and jury duty. The other would contain hours and dollars for productive hours; such as, regular and overtime. Unfortunately, the system had a limit on the number of bytes of data that could be extracted in one report; therefore two reports needed to be used.

The data elements, pay period and calendar year, were available as filters in the ad hoc reporting system, but were not available to export from that system. Two calculated fields, defined as runtime parameters, were added to the next two models to capture these two fields for the final report.

The Monarch model built to process the Excel file created for the non-productive hours and dollars is named Model3. It used Payroll1.xls as an external lookup to get the department name field. It would format the data and rename the fields so that they could be more easily understood. This practice is especially helpful when others will be using your models and projects. The next model (Model4) would process the Excel file for the productive hours and do much the same with that data.

Both models required a filter to exclude the records that had zero hours and dollars since the ad hoc report writer had no way of excluding those. Both would export their in Excel format. For simplicity’s sake, the output for Model3 was named as Payroll3.xls and the output for the Model4 as Payroll4.xls.

The next model (Model5) would open the database file Payroll3.xls and use Payroll4.xls as an external lookup file. To make sure we were combining like data, the link was on department number, pay period, and calendar year. This would allow the entire dataset of pay period data to be combined into one file for easier processing. The export would be to Excel – Payroll3.xls.

The next model (Model6) would open the database file Payroll3.xls and use Payroll4.xls as an external lookup file. To make sure we were combining like data, the link was on department number, pay period, and calendar year. This would allow the calendar year data to be combined into one file and create a summary for the current pay period totals using department as the key field.

The export would be to an existing Excel file named Payroll6.xls and the summary data would be appended to existing data. This would be the file used for year-to-date totals and would contain a single record by department for each pay period.

Yes, model5 and model6 could have been combined into a single model/project with two exports. When I completed the original plan, keeping pay period and year-to-date data separated would facilitate any possible troubleshooting. In retrospect it wasn’t necessary, but this doesn’t create unwanted effects and adds minimal time to the process.

The next model (Model7) would open Payroll6.xls and create a summary of year-to-date data for the entire calendar year. It exports to Excel – Payroll7.xls. The exported data has a single record by department containing year-to-date totals for all pay codes.

The final model (Model8) would open Payroll5.xls which is the pay period specific data. It uses Payroll7.xls, the year-to-date data by department, as an external lookup. It also uses Payroll2.xls, the pay period calendar fields, as an external lookup. This model gathers only the data that will be used in the sample report, but it will contain all of the departments and creates a summary for the entire organization by pay period. This export would be named Payroll8.xls.

Produce the best presentation, easily

So far, success has been achieved. However, I hadn’t researched the “pretty” formatting in Excel quite yet. I toyed with the idea of going back to the CFO and explaining I could get his export in Word using a Mail Merge process and have the data populate a pretty Word form, but decided to seek an Excel-based solution first.

It was time to do an Internet search and see if any add-ins for Excel that might work. On the first search, I saw the words “mail merge entirely in Excel”. I’d found an add-in that fit the task perfectly, fittingly named “Mail Merge for Excel”. It takes a row of data per merge, which was exactly what was needed. They even allow you to try before you buy, so I downloaded the software.

As I started to learn the add-in, I found that a shell or a blank form of the final report would need to be created. That would be simple since the sample was supplied to me by the CFO. I decided that I would use the shell as a sheet in the Payroll8.xls file. After testing and making a minor change to Model5 to export a new calculated field – one that would trigger the mail merge – I was ready to purchase the add-in. Costing about $36 USD, this excellent tool is truly a bargain.

Automate for optimal execution

After testing the entire process from start to finish, running everything manually and making sure all of the numbers balanced, I began automating the process. While, the two reports from the hospital’s information system would still need to be created manually, the eight Monarch projects were automated using a command file. For this task I used the handy tool, ExcelWithMonarch Monarch Batch File Generator, found on this website.

The mail merge portion was left as a manual process.

I was almost ready to present the final version to the CFO, but first I had to prepare the final documentation of the Monarch models/projects and write a “how to” document to instruct the staff that would use these processes to create the reports. I do this for almost all of the processes that I supply to my customers as it helps them when they have staff changes and it reduces the number of support calls that I receive. The documentation also benefits me, at it makes it very easy to quickly help if I am called upon to troubleshoot or revise the process.

Delivering the solution

When I presented this to the CFO, he was pleased with the final product. His only question was how long it took to create for each pay period. Since we were already on pay period three when I was asked for the report, four pay periods needed to be processed when it was finished. That gave a great opportunity to provide a solid number based using the report multiple times. The final answer was “less than 30 minutes”.

Since the CFO was pleased with both the final report and the time required to produce the data, I considered this project a success.

Like gardening, once in a while you find a way to improve your skills. In gardening it may be a different variety or a better fertilizer. In creating actionable data it may well be a tool that you have never used. Indeed, I found an additional tool for my toolkit to continue Excelling with Monarch.

Thanks Joe!

With 35 years worth of experience in the healthcare sector, Joe Berry can help you to diagnose and cure your information challenges. Contact him and learn more by visiting www.bbshc.com.

Joe has a special reason for his continuing interest in healthcare: “I am just as passionate about education and awareness for organ donation and transplantation as I am about Monarch. I received the Gift of Life on October 17, 2007 and will spend the rest of my life helping save more lives.”

As Joe wrote, by giving your work good strong roots as a result of planning, documenting, and organizing projects into easily manageable parts, you’ll be certain that you and your business will grow, succeed, and even excel, with Monarch.

I was introduced to Scaredy Squirrel today. I’d met him before, but I’m afraid that I didn’t know his name. You’ve probably already met him too.

Scaredy Squirrel is a cute little guy. He lives in his tree and he knows it well. He enjoys his regular routine, which includes looking at the view and eating nuts. He’s not just carefree; he’s carefully planned his strategies in case of attacks by killer bees, sharks and other threats.

photo credit: law_keven

One day something happens and Scaredy Squirrel suddenly discovers that he’s more prepared than he’d ever dreamed. This experience changes his whole world for the better.

As mentioned, I’ve met Scaredy, but he certainly didn’t look like a squirrel at the time. In fact, I wonder if he’s a secret agent or something. Now, that I’m thinking about it, I must have encountered him a few times…

A Master of Disguise

Once he looked just like an accountant. He’d been using the same old limited accounting software for years without seeking ways to enhance the system or provide more value to his clients. Yup, it must have been him.

Then one day I overheard an IT manager discussing the solution that his team was about to implement with the President. It was a system that had been adopted by many companies much larger than theirs. “Great decision,” said the President. “Nobody ever got fired for buying that brand, right?” asked the IT manager, only half joking.

Scaredy Squirrel is one talented little guy. And he’s SO fast, being able to take two roles at once, and all.

Later, something happened to change that happy-go-lucky critter. He must have been preparing for winter, because he was being quite aggressive and clearly was feeling the pressure.

In order to accumulate a bigger stockpile than any other squirrel, he assumed the guise of a salesman for a big database company. His task was to sell software to other squirrels to that could help them manage their nuts. The software would be theirs for a mere half of their nuts.

All of the other squirrels knew that they liked their nuts, and wanted more of them, so this guy really seemed to know what he was talking about, so this must be a great product that they must invest in so that they could acquire more nuts.

I wish that you could have been there to hear him. “Once you agree, we’ll work with you to implement such an integrated system for handling your nuts you’ll wonder how you ever got along without it. Your tree will never need anything else; everything’s included!”

I felt badly for Scaredy Squirrel back then. He was so focused on maintaining the only reality that he’d ever known that he couldn’t see the forest for the trees. I guess that was to be expected though. He’d spent so long cooped up in his tree looking at the horizon that he’d never even tried to expand his own horizons.

But, as mentioned, one day Scaredy was forced to jump out of his tree. That’s when he discovered that he was a flying squirrel! He learned that he could leave the tree and go to new places and try new things and that nothing bad would happen.

Oh, he still enjoyed his tree and his view and his nuts, but now that he liked trying new things, he became a much happier squirrel now that he wasn’t constantly afraid.

Have you seen Scaredy Squirrel? What was he up to when you saw him last?

Did somebody say “refund guarantee”?

If you still use only Excel, perhaps in addition to a larger information system, and haven’t added Monarch to your toolkit, then you’re working too hard to get and manage your data, and with Datawatch’s “unconditional 30 day refund guarantee” (see the last page), you’ve got nothing to lose and everything to gain.

Don’t be a Scaredy Squirrel. Stop working for your data, and make your data work for you. Get outside of your comfort zone, and try, just try, to excel with Monarch.