Real Time Reporting Databases

Reporting projects are the kind of projects which never seem to end. After a couple iterations I’ve come to the following conclusions:

  1. Absolutely no reports should run on a production database.
  2. Moving/aggregating data from a production database to a reporting database using ETL tools prone to synchronization issues and pretty unreliable.
  3. The best option is to set up real time replication of the data and build additional views on that.

Unfortunately, if you need to get data from heterogeneous databases, ie. Oracle, MySQL, SQL Server, etc. into a single reporting database, replication is not a simple solution. If you are running expensive database software in production, it may not be cost effective to run the same database for reporting.

Of course there are cross database replication solutions like Golden Gate or SharePlex but they are very expensive. I had already given up on getting data from Oracle into MySQL for reports when I stumbled across Tungsten Replicator.

According to the website, Tungsten Replicator provides open source database-neutral master/slave replication. Master/slave replication is a highly flexible technology that can solve a wide variety of problems including Cross DBMS Integration, ie. replication from Oracle to MySQL.

I’m looking forward to testing this product in the near future and I’d be happy to get anyone’s input if they’ve used it.

Yonah Russ

Yonah Russ
Production tested jack of all trades and master of some. Forgotten more than I remember, some of which is written here. Occasional BOFH but only because I care. KISS is Bliss!

Being a startup shouldn't mean starting from scratch

In an open retrospective, Eden Shochat, from Aleph VC, asks what they could do better to help the companies they invest in. I really appr...… Continue reading

Triton Bare Metal Containers FTW!

Published on June 22, 2015