Reporting projects are the kind of projects which never seem to end. After a couple iterations I’ve come to the following conclusions:
- Absolutely no reports should run on a production database.
- Moving/aggregating data from a production database to a reporting database using ETL tools prone to synchronization issues and pretty unreliable.
- The best option is to set up real time replication of the data and build additional views on that.
Unfortunately, if you need to get data from heterogeneous databases, ie. Oracle, MySQL, SQL Server, etc. into a single reporting database, replication is not a simple solution. If you are running expensive database software in production, it may not be cost effective to run the same database for reporting.
Of course there are cross database replication solutions like Golden Gate or SharePlex but they are very expensive. I had already given up on getting data from Oracle into MySQL for reports when I stumbled across Tungsten Replicator.
According to the website, Tungsten Replicator provides open source database-neutral master/slave replication. Master/slave replication is a highly flexible technology that can solve a wide variety of problems including Cross DBMS Integration, ie. replication from Oracle to MySQL.
I’m looking forward to testing this product in the near future and I’d be happy to get anyone’s input if they’ve used it.