Skip navigation

Food-Trak Import Exception Reports

Last year my post about using Pentaho Data Integration to extract Micros 9700 sales data for importing into Food-Trak discussed exception reporting. Well a year has passed and I am pleased to report that I have made a large improvemet to the way we handle exception reporting. (And also, the Micros 3700 extracts are I wrote about in August are working perfectly.) The last couple of months I have been mentoring two very talented individuals in order for them to take over all my Micros 9700 and Food-Trak operational responsibilities so I can focus more on Business Intelligence and Dashboarding using the Pentaho Business Intelligence Suite.

Our company’s restaurants must keep their menus fresh and relevant. This means that quarterly major revisions are done and monthly new items are added and old ones removed. We have utilized the menu engineering capabilities of Food-Trak for two of our biggest restaurants to help us do this. On the data extract side of things, it is very challenging to keep up with all of these changes. It is relatively quick to add new menu item to the Micros 9700 and Micros 3700 systems. The time consuming part is entering a plate recipe into Food-Trak and completing the sales record, especially when there are 15 new items in one restaurant, 10 new items in another, and another adds a few drink specials!

The way I was handling the exception reports before was by copying and pasting the audit report created by Food-Trak. Then, I would save the text file and run it through a Pentaho transformation to create an Excel Spreadsheet, which included the menu item name from Micros. This was still labor intensive, but it was easier than searching through Micros EMC for each menu item number and name.

This month, my two proteges handled entering all the new menu changes that rolled out for 4th Quarter. There was a major over haul of the menu in our largest restaurant, and several changes in the others. I was able to spend my month creating dashboards, cubes, transformations and jobs with the Penthao Business Intelligence Suite.  One project that was completed this month was automating the Food-Trak exception reports. As well, I created a current check analysis dashboard for all the restaurants and prototyped a KPI dashboard for our hotel division.

The Food-Trak exception reports are created using Pentaho Data Integration. Instead of copying and pasting the audit report and filtering out the ‘not founds’, I query the POS Audit table in each Food-Trak database (we use three) for the ‘not founds’. Then, I simply join the results to the Micros 9700 and Micros 3700 menu item masters to extract the menu item names and output the results to an Excel file and a text file.

Food-Trak requires a specific format for the POS code. To make it easer to enter the code into the Food-Trak sales record, I create the POS code formatted to the Food-Trak specifications. Then, our Food-Trak admins just have to copy and paste the code into the sales record, saving a little time and effort for each entry.

Using Pentaho Data Integration, I then created a job which runs the transformation and emails the files to the Food-Trak admins, their manager, our F&B Director, and to myself. I also send a results email to myself so I know if the jobs and transformations worked. Thus today, we automatically upload hundreds of sales records each morning for every restaurant(there are currently six in all and a few more in the works), and autocreate the exception reports which are delivered to the appropriate parties before the work day starts.


Today I created a Micros 3700 to Food-Trak transformation with Pentaho Data Integration. The SQL was the easy part. The hard part was creating the JDBC connection. At least it was hard for me since I haven’t connected to Sybase since 1998. But after 4 hours of googling and experimenting, I figured out the connection configuration and created the transformation.

I used Jconnect and modified the spool.bat file to include its path in the classpath variable. I have a problem with column names being passed through the stream. But, I created a work around and presto, changeoh … a file to import into Food-Trak.

Tomorrow I will test and validate the extract. Once I am 100% sure it works, I will write another post about it.

The Micros 9700 data extract for Food-Trak integration is a step in the Micros 9700 ETL process.  The core transformation extracts Micros 9700 transaction data from MCRSPOS database.  The individual sales transaction is identified in the totals table. The Food-Trak format required for the Micros 9700 interface must include the revenue center, the menu item master number, the menu item definition sequence number, and the menu item  price number. The transformation merges these dimensions with the transaction record from MCRSPOS.Totals.

Pentaho Data Integration transformation

Pentaho Data Integration transformation

The entire extraction and transformation is written with Pentaho Data Integration. Pentaho Data Integration is an open source Extraction, Transformation, and Loading application. Using the graphical user interface of Pentaho Data Integration allows rapid application development. It is relatively easy to customize the extract using simple SQL statements in the transformation table input objects.

A required step for  Micros 9700 and Food-Trak data integration is the creation of product records for each product sold. Each product record is attached to either a recipe or a purchased item in Food-Trak. The Micros 9700 Food-Trak interface requires the revenue center number, the menu item master number, the menu item definition sequence number, and the menu item price number. Creating product records is a very time consuming process and requires access to Micros 9700 EMC and a deep understand of the products sold.

My experience with my company is that our restaurants actually have several thousand products.  Remember from my last article that one of our requirements is fine granularity. Therefore, we can have several product records for one item. For example, a food menu item might have a regular price, a half off student price, and a late night menu price. A draft beer will have a double draft regular price, a double draft happy hour price, a double draft Monday Ladies’ night price, a double draft half off price, a pint regular price, a pint happy hour price, ….

We also have very dynamic menus. Each restaurant changes their menu at least seasonally. Seasonal dishes are offered with various themes or featuring fresh local food. New wines are offered monthly, new beers and liquors appear weekly. Our business strategy requires us to change our menu mixes often, while, of course, maintaining guest favorites. Our business strategy also requires us to manage our food and beverage assets at a very granular level. The Micros 9700 and Food-Trak interface is in a near constant state of flux. Thus, menu changes to Micros and Food-Trak product record updates are all handled by Corporate IT.

Especially at the beginning of the Micros 9700 Food-Trak data integration project, exception reporting is very useful. The Food-Trak Micros 9700 interface creates a results report. Exceptions are flagged <Not Found>. By copying and pasting this report into a text file, it can be merged with Micros data via Pentaho Data Integration. The transformation below reads in the results report and attaches the menu item name to record as well as appending a quantity column.

Food-Trak Micros 9700 data integration exception report

The Micros 9700 transformation is run automatically using Pentaho Data Integration Kettle application. The automated transformation saves a date stamp variable which is inserted into the SQL code of the table input object. A batch containing a one line kettle command is scheduled via Windows Task Scheduler.

My company owns several food and beverage services- restaurants, banquent and catering, concessions, and cafeterias. In order to control costs and manage assets, we implemented the Food-Trak Food and Beverage Management System. For my manufacturing friends, Food-Trak is the equivalent of Enterprise Resource Planning for the Food and Beverage Industry.  And while we think of Food-Trak this way, consider the thought that a restaurant is a manufacturing facility which manufactures product on demand and in real time.

Our goal as a company is to understand and manage our food and beverage assets to a very granular level. Thus, we (attempt) to count every fraction of an ounce of what is served. To this end, we must record every serving in our Point of Sales system(POS), Micros 9700 HMS, and feed these transactions into our Food and Beverage Management System, Food-Trak.

In operations the POS workstation must be programmed to capture the servings.  The challenges we faced were in the realm of modifiers- options, sides, condiments, add-ons, extras. Because of the complexity in programming modifiers and condiments and because of a few years of decentralized programming, many things were neglected. Operations had gotten used to messaging things, using Open transactions, and making assumptions. After a couple months of reconfiguring and testing, we were able to program the Micros 9700 system to capture transactions to the level required without hampering operations. In fact, the changes improved operations as servers had more automated choices presented in a logical way and the kitchen had to do less interpretation of orders, i.e better communications between front of house and the kitchen.

The bigger challenge was extracting the sales data from the Micros 9700 HMS. Organizations that want to integrate their Micros 9700 sales data into Food-Trak have 3 options-

  1. Contract with Micros to do it
  2. Contract with their Micros vendor to do it.
  3. Do it yourself.

Contracting with Micros or a vendor requires significant investment. If you are reading this because you want to integrate Micros 9700 sales data with Food-Trak data, and you have talked to your vendor, you know this is true. Also, Micros doesn’t provide the best customer support in the world. If you are a Micros customer, and not IHG, you know this is true.  Also, the interfaces I was able to review were nothing more than scripts written in SIMM, a proprietary Micros language. Ingenious? Yes. Practical and user friendly? Not at all.

Since I have been writing data extracts since the early 1990s, the decision to do it ourselves was a very easy decision for me to make, although it was met with a bit of skepticism from some of the decision makers. My first attempt was to extract data from the Netvupoint database. Easy right? Micros already has a built in data warehouse. All I had to do was extract the total sales from the Netvupoint tables. Well, during my validation testing, I discovered I couldn’t get to the granularity we required from Netvupoint. I had to dive into the transaction database.

I will save the details of my adventure within the transaction database for another time. When you hire me to setup your extracts, I will share the story with you over coffee! In a few weeks of focused effort, I was able to extract the transactions at the granular level we required and today have a hit rate of >98%. The exceptions are mostly open items and the rest are a data entry errors. I use an open source ETL called Pentaho Data Integration. The extract procedure is automated and has a graphical user interface. To aid in validation testing and monitoring of extracts, I have also created an ETL procedure that reads the exception reports from Food-Trak and matches the “Not Found” menu item number with its name entry in the menu item master table in Micros.

Because of our integration of Micros 9700 sales data and our Food-Trak production data, we are able to create industry standard sales analysis reports on demand and to an incredible depth. I would enjoy talking to you about this experience and look forward to helping you integrate your Micros 9700 sales data with Food-Trak.

Indeed, the car dashboard analogy is used extensively in the business performance dashboard business. In my quest to create meaningful, valuable dashboards, I must always keep an open mind. While I thought the business tachometer is a good idea, and maybe it is ok, I must take heed of Dr. Nicholas Bissantz’s critique of the car metaphor.

Dr. Bissantz’s work is extraordinary. I will be adding sparklines to my list of graphs to work on.

Because the car dashboard metaphor is still ubiquitous in the business performance dashboard world, dials and guages will for the time being will remain as a dashboard component widget just as pie charts and bar charts are still used. However, killer business performance dashboards must adopt sparklines. 

Why do things break down? Are there predictable patterns in malfunctions that can be expressed in terms of mercury retrograde? My personal experience says yes. So do the headlines.